LM: Difference between revisions

From Fundamental Ramen
Jump to navigation Jump to search
 
(6 intermediate revisions by the same user not shown)
Line 20: Line 20:
: [[LM/llama.cpp for SYCL|llama.cpp for SYCL]]
: [[LM/llama.cpp for SYCL|llama.cpp for SYCL]]
: [[LM/llama.cpp for CUDA|llama.cpp for CUDA]]
: [[LM/llama.cpp for CUDA|llama.cpp for CUDA]]
: [[LM/OVMS|OVMS]]
: [[LM/llama.cpp for OpenVINO|llama.cpp for OpenVINO]]
: [[LM/llama.cpp for OpenVINO|llama.cpp for OpenVINO]]
: [[LM/llama.cpp for TurboQuant|llama.cpp for TurboQuant]]
: [[LM/llama.cpp for TurboQuant|llama.cpp for TurboQuant]]
: [[LM/lemonade for AMD NPU|lemonade for AMD NPU]]
: [[LM/SGLang|SGLang]]
: [[LM/SGLang|SGLang]]
| style="width:200px;vertical-align:top" |
| style="width:200px;vertical-align:top" |
Line 40: Line 38:
:: [[LM/lact|lact]]
:: [[LM/lact|lact]]
:: [[LM/PCIe Trouble Shooting|PCIe Trouble Shooting]]
:: [[LM/PCIe Trouble Shooting|PCIe Trouble Shooting]]
: Comparison
:: [[LM/GPU Comparison]]
| style="width:200px;vertical-align:top" |
| style="width:200px;vertical-align:top" |
: [[LM/Performance|Performance]]
: [[LM/Performance|Performance]]
: [[LM/Security|Security]]
: [[LM/Hallucination|Hallucination]]
|}
|}


Line 56: Line 54:
flowchart LR
flowchart LR
   A(LiteLLM)
   A(LiteLLM)
   B(llama-swap)
   B1(llama-swap)
   C1(llama.cpp)
   B2(vLLM<br>Qwen 3.8 27B)
   C2(vLLM)
   C(llama.cpp<br>Gemma 4 E4B)


   A --> B
   A --> B1 & B2
   B --> C1 & C2
   B1 --> C
</quickmmd>
</quickmmd>



Latest revision as of 09:49, 21 September 2026

Topics

🤖 Agent  🧠 Model  🚀 Engine  ⚙️ GPU & Driver  Evaluation
ZooCode
axe
MCP
hf (Model Management)
Qwen 3.8 27B
Gemma 4
Devstral Small 2 24B
Experience
llama-swap
vLLM
ComfyUI
llama.cpp for SYCL
llama.cpp for CUDA
llama.cpp for OpenVINO
llama.cpp for TurboQuant
SGLang
Intel Arc Pro B70
Install driver
Install oneAPI (SYCL)
Install OpenVINO
.bashrc for B70
Experiment
NVIDIA RTX 3060
Install driver
AMD Ryzen AI 7 350
Experiment
Tools
Install nvtop
lact
PCIe Trouble Shooting
Comparison
LM/GPU Comparison
Performance

News

Models management

  • llama-swap can launch model, cannot manage users.
  • LiteLLM cannot launch model, can manage users.

Decision Tree for tinkering with Intel Arc Pro B70