LM: Difference between revisions
Jump to navigation
Jump to search
(→Topics) |
|||
| (24 intermediate revisions by the same user not shown) | |||
| Line 11: | Line 11: | ||
: [[LM/hf|hf (Model Management)]] | : [[LM/hf|hf (Model Management)]] | ||
: [[LM/Qwen 3.8 27B|Qwen 3.8 27B]] | : [[LM/Qwen 3.8 27B|Qwen 3.8 27B]] | ||
: [[LM/Gemma 4|Gemma 4]] | |||
: [[LM/Devstral Small 2 24B|Devstral Small 2 24B]] | : [[LM/Devstral Small 2 24B|Devstral Small 2 24B]] | ||
: [[LM/Experience|Experience]] | : [[LM/Experience|Experience]] | ||
| style="width:200px;vertical-align:top" | | | style="width:200px;vertical-align:top" | | ||
: [[LM/llama-swap|llama-swap]] | : [[LM/llama-swap|llama-swap]] | ||
: [[LM/ | : [[LM/vLLM|vLLM]] | ||
: [[LM/ComfyUI|ComfyUI]] | |||
: [[LM/llama.cpp for SYCL|llama.cpp for SYCL]] | : [[LM/llama.cpp for SYCL|llama.cpp for SYCL]] | ||
: [[LM/ | : [[LM/llama.cpp for CUDA|llama.cpp for CUDA]] | ||
: [[LM/llama.cpp for OpenVINO|llama.cpp for OpenVINO]] | : [[LM/llama.cpp for OpenVINO|llama.cpp for OpenVINO]] | ||
: [[LM/llama.cpp for TurboQuant|llama.cpp for TurboQuant]] | : [[LM/llama.cpp for TurboQuant|llama.cpp for TurboQuant]] | ||
: [[LM/SGLang|SGLang]] | : [[LM/SGLang|SGLang]] | ||
| style="width:200px;vertical-align:top" | | | style="width:200px;vertical-align:top" | | ||
| Line 32: | Line 30: | ||
:: [[LM/.bashrc for B70|.bashrc for B70]] | :: [[LM/.bashrc for B70|.bashrc for B70]] | ||
:: [[LM/Intel Pro Arc B70|Experiment]] | :: [[LM/Intel Pro Arc B70|Experiment]] | ||
: NVIDIA RTX 3060 | |||
:: [[LM/Install NVIDIA driver|Install driver]] | |||
: AMD Ryzen AI 7 350 | : AMD Ryzen AI 7 350 | ||
:: [[LM/Ryzen AI 7 350|Experiment]] | :: [[LM/Ryzen AI 7 350|Experiment]] | ||
: Tools | : Tools | ||
:: [[LM/Install nvtop|Install nvtop]] | :: [[LM/Install nvtop|Install nvtop]] | ||
:: [[LM/lact|lact]] | |||
:: [[LM/PCIe Trouble Shooting|PCIe Trouble Shooting]] | |||
: Comparison | |||
:: [[LM/GPU Comparison]] | |||
| style="width:200px;vertical-align:top" | | | style="width:200px;vertical-align:top" | | ||
: [[LM/Performance|Performance]] | : [[LM/Performance|Performance]] | ||
|} | |} | ||
| Line 46: | Line 48: | ||
* [https://huggingface.co/datasets/ScaleAI/SWE-bench_Pro?leaderboard_max_params=32B SWE Pro Benchmark for models < 32B] | * [https://huggingface.co/datasets/ScaleAI/SWE-bench_Pro?leaderboard_max_params=32B SWE Pro Benchmark for models < 32B] | ||
* [https://storage.openvinotoolkit.org/repositories/openvino/packages/2026.3/linux/ OpenVINO 2026.3 supports Ubuntu 26] - 2026-08-04 | * [https://storage.openvinotoolkit.org/repositories/openvino/packages/2026.3/linux/ OpenVINO 2026.3 supports Ubuntu 26] - 2026-08-04 | ||
= Models management = | |||
<quickmmd name="models-handling-infra"> | |||
flowchart LR | |||
A(LiteLLM) | |||
B1(llama-swap) | |||
B2(vLLM<br>Qwen 3.8 27B) | |||
C(llama.cpp<br>Gemma 4 E4B) | |||
A --> B1 & B2 | |||
B1 --> C | |||
</quickmmd> | |||
* llama-swap can launch model, cannot manage users. | |||
* LiteLLM cannot launch model, can manage users. | |||
= Decision Tree for tinkering with Intel Arc Pro B70 = | = Decision Tree for tinkering with Intel Arc Pro B70 = | ||
Latest revision as of 09:49, 21 September 2026
Topics
| 🤖 Agent | 🧠 Model | 🚀 Engine | ⚙️ GPU & Driver | Evaluation |
|---|---|---|---|---|
|
News
Models management
- llama-swap can launch model, cannot manage users.
- LiteLLM cannot launch model, can manage users.
Decision Tree for tinkering with Intel Arc Pro B70
