LM: Difference between revisions
Jump to navigation
Jump to search
| (208 intermediate revisions by the same user not shown) | |||
| Line 1: | Line 1: | ||
= | = Topics = | ||
== | {| class="wikitable" | ||
! 🤖 Agent || 🧠 Model || 🚀 Engine || ⚙️ GPU & Driver || Evaluation | |||
|- | |||
| style="width:200px;vertical-align:top" | | |||
: [[LM/ZooCode|ZooCode]] | |||
: [[LM/jrswab_axe|axe]] | |||
: [[LM/MCP|MCP]] | |||
| style="width:200px;vertical-align:top" | | |||
: [[LM/hf|hf (Model Management)]] | |||
: [[LM/Qwen 3.8 27B|Qwen 3.8 27B]] | |||
: [[LM/Gemma 4|Gemma 4]] | |||
: [[LM/Devstral Small 2 24B|Devstral Small 2 24B]] | |||
: [[LM/Experience|Experience]] | |||
| style="width:200px;vertical-align:top" | | |||
: [[LM/llama-swap|llama-swap]] | |||
: [[LM/vLLM|vLLM]] | |||
: [[LM/ComfyUI|ComfyUI]] | |||
: [[LM/llama.cpp for SYCL|llama.cpp for SYCL]] | |||
: [[LM/llama.cpp for CUDA|llama.cpp for CUDA]] | |||
: [[LM/llama.cpp for OpenVINO|llama.cpp for OpenVINO]] | |||
: [[LM/llama.cpp for TurboQuant|llama.cpp for TurboQuant]] | |||
: [[LM/SGLang|SGLang]] | |||
| style="width:200px;vertical-align:top" | | |||
: Intel Arc Pro B70 | |||
:: [[LM/Install driver|Install driver]] | |||
:: [[LM/Install oneAPI|Install oneAPI (SYCL)]] | |||
:: [[LM/Install OpenVINO|Install OpenVINO]] | |||
:: [[LM/.bashrc for B70|.bashrc for B70]] | |||
:: [[LM/Intel Pro Arc B70|Experiment]] | |||
: NVIDIA RTX 3060 | |||
:: [[LM/Install NVIDIA driver|Install driver]] | |||
: AMD Ryzen AI 7 350 | |||
:: [[LM/Ryzen AI 7 350|Experiment]] | |||
: Tools | |||
:: [[LM/Install nvtop|Install nvtop]] | |||
:: [[LM/lact|lact]] | |||
:: [[LM/PCIe Trouble Shooting|PCIe Trouble Shooting]] | |||
: Comparison | |||
:: [[LM/GPU Comparison]] | |||
| style="width:200px;vertical-align:top" | | |||
: [[LM/Performance|Performance]] | |||
|} | |||
= News = | |||
== | * [https://huggingface.co/datasets/ScaleAI/SWE-bench_Pro?leaderboard_max_params=32B SWE Pro Benchmark for models < 32B] | ||
* [https://storage.openvinotoolkit.org/repositories/openvino/packages/2026.3/linux/ OpenVINO 2026.3 supports Ubuntu 26] - 2026-08-04 | |||
= Models management = | |||
<quickmmd name="models-handling-infra"> | |||
flowchart LR | |||
A(LiteLLM) | |||
B1(llama-swap) | |||
B2(vLLM<br>Qwen 3.8 27B) | |||
C(llama.cpp<br>Gemma 4 E4B) | |||
A --> B1 & B2 | |||
B1 --> C | |||
</quickmmd> | |||
* llama-swap can launch model, cannot manage users. | |||
* LiteLLM cannot launch model, can manage users. | |||
= Decision Tree for tinkering with Intel Arc Pro B70 = | |||
<quickmmd name="tinkering-b70"> | |||
flowchart TD | |||
subgraph Hardware | |||
A1(Intel Arc Pro B70):::node-okay | |||
A2(Asus ProArt B850 WiFi NEO):::node-okay | |||
A3(AMD Ryzen 5 9600X):::node-okay | |||
end | |||
subgraph OS | |||
B1(Install<br>Ubuntu 26.04):::node-okay | |||
A1 & A2 & A3 --> B1 | |||
end | |||
subgraph Driver | |||
C1(Install PPA driver<br>26.27.39122.14):::node-okay | |||
C2(Install oneAPI<br>2026.1.1.20260724):::node-okay | |||
C3(Install OpenVINO<br>2026.3.0<br>hard to install):::node-warn | |||
C1 --> C2 & C3 | |||
B1 --> C1 | |||
end | |||
subgraph Runtime | |||
D1[Build llama.cpp for<br>SYCL]:::node-okay | |||
D2[Build llama.cpp for<br>OpenVINO<br>not stable yet]:::node-warn | |||
%% D3[Build SGLang for<br>SYCL]:::node-todo | |||
%% D4[Build SGLang for<br>OpenVINO]:::node-todo | |||
C2 --> D1 | |||
C3 --> D2 | |||
%% C2 --> D3 | |||
%% C3 --> D4 | |||
end | |||
subgraph Tools | |||
T1(Install<br>pipx) | |||
T2(Install<br>hf) | |||
T3(Install<br>nvtop) | |||
T4(Install<br>tmux) | |||
T5(Install<br>lact) | |||
T6(Install<br>llama-swap) | |||
T1 --> T2 | |||
B1 --> T1 | |||
B1 --> T3 | |||
B1 --> T4 | |||
B1 --> T5 | |||
T5 --> T6 | |||
end | |||
subgraph Models | |||
M1(Qwen3.8-27B) | |||
M2(Devstral-Small-2-24B) | |||
T2 --> M1 & M2 | |||
end | |||
subgraph config | |||
V1(Optimize<br>.bashrc) | |||
V2(Optimize<br>lmsw.yaml) | |||
C2 --> V1 | |||
C3 --> V1 | |||
T2 --> V1 | |||
T4 ---> V1 | |||
T6 ---> V2 | |||
end | |||
subgraph Service | |||
Z(service<br>llama-swap \<br> -config lmsw.yaml \<br> -listen 0.0.0.0:9876) | |||
M1 --> Z | |||
D1 --> Z | |||
V2 --> Z | |||
end | |||
classDef node-okay fill:#efe,stroke:#393 | |||
classDef node-warn fill:#fff0e0,stroke:#d63 | |||
classDef node-fail fill:#fee,stroke:#d33 | |||
classDef node-todo fill:#eee,stroke:#777 | |||
</quickmmd> | |||
Latest revision as of 09:49, 21 September 2026
Topics
| 🤖 Agent | 🧠 Model | 🚀 Engine | ⚙️ GPU & Driver | Evaluation |
|---|---|---|---|---|
|
News
Models management
- llama-swap can launch model, cannot manage users.
- LiteLLM cannot launch model, can manage users.
Decision Tree for tinkering with Intel Arc Pro B70
