LM: Difference between revisions

From Fundamental Ramen
Jump to navigation Jump to search
 
(33 intermediate revisions by the same user not shown)
Line 1: Line 1:
= Topics =
= Topics =
{| class="wikitable"
{| class="wikitable"
! Agent || Model || Runner || Driver
! 🤖 Agent || 🧠 Model || 🚀 Runner || ⚙️ GPU & Driver
|-
|-
| valign="top" |
| style="width:200px;vertical-align:top" |
* [[LLM/jrswab_axe|axe]]
: [[LM/jrswab_axe|axe]]
* SKILL
: SKILL
* MCP
: MCP
| valign="top" |
| style="width:200px;vertical-align:top" |
* [[LLM/hf|hf]]
: [[LM/hf|hf (Model Management)]]
| valign="top" |
: Muse Glimmer
* SGLang
: Devstral Small 2
* llama.cpp for SYCL
: Qwen 3.6 27B
* llama.cpp for OpenVINO
: Gemma 4
* lemonade for AMD NPU
| style="width:200px;vertical-align:top" |
| valign="top" |
: [[LM/SGLang|SGLang]]
* xe Driver
: [[LM/llama-swap|llama-swap]]
* oneAPI
: [[LM/llama.cpp for SYCL|llama.cpp for SYCL]]
* OpenVINO
: [[LM/llama.cpp for OpenVINO|llama.cpp for OpenVINO]]
: [[LM/lemonade for AMD NPU|lemonade for AMD NPU]]
| style="width:200px;vertical-align:top" |
: Intel Arc Pro B70
:: [[LM/Install driver|Install driver]]
:: [[LM/Install oneAPI|Install oneAPI (SYCL)]]
:: [[LM/Install OpenVINO|Install OpenVINO]]
:: [[LM/Intel Pro Arc B70|Experiment]]
: AMD Ryzen AI 7 350
:: [[LM/Ryzen AI 7 350|Experiment]]
: Tools
:: [[LM/Install nvtop|Install nvtop]]
|}
|}


= News =
= News =


* https://huggingface.co/docs/huggingface_hub/package_reference/environment_variables
* [https://huggingface.co/datasets/ScaleAI/SWE-bench_Pro?leaderboard_max_params=32B SWE Pro Benchmark for models < 32B]
* https://www.datalearner.com/en/leaderboards/category/code?benchmark=SWE-bench+Verified&modelSize=34b&licenseType=open
* [https://storage.openvinotoolkit.org/repositories/openvino/packages/2026.3/linux/ OpenVINO 2026.3 supports Ubuntu 26] - 2026-08-04
* https://huggingface.co/datasets/ScaleAI/SWE-bench_Pro


= Decision Tree for tinkering with Intel Arc Pro B70 =
= Decision Tree for tinkering with Intel Arc Pro B70 =
Line 46: Line 57:
     C3[Install<br>OpenVINO]:::node-okay
     C3[Install<br>OpenVINO]:::node-okay
     C4[Install<br>PPA driver]:::node-warn
     C4[Install<br>PPA driver]:::node-warn
     C5[Cannot install<br>oneAPI]:::node-fail
     C5[Client GPU not listed in<br>oneAPI 2026.1.0]:::node-fail
     C6[Cannot install<br>OpenVINO]:::node-fail
     C6[Cannot install<br>OpenVINO 2026.2.1]:::node-fail
     C1 --> C2 & C3
     C1 --> C2 & C3
     C4 --> C5 & C6
     C4 --> C5 & C6
Line 108: Line 119:
   classDef node-todo fill:#eee,stroke:#777
   classDef node-todo fill:#eee,stroke:#777
</quickmmd>
</quickmmd>
= Build environment =
== hf (model management) ==
{| class="wikitable"
! Purpose || Command
|-
| cache management ||
<syntaxhighlight lang="bash">
hf cache list
hf cache rm <model id>
hf cache prune
</syntaxhighlight>
|-
| fix WiFi problem ||
<syntaxhighlight lang="bash">
HF_XET_FIXED_DOWNLOAD_CONCURRENCY=10 hf download "unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF" --include "*UD-Q4_K_XL*"
HF_XET_FIXED_DOWNLOAD_CONCURRENCY=10 hf download "unsloth/Devstral-Small-2-24B-Instruct-2512-GGUF" --include "*UD-Q4_K_XL*"
</syntaxhighlight>
|-
| search ||
<syntaxhighlight lang="bash">
# search by population
hf models ls --search "Muse-Glimmer" --apps llama.cpp --expand "downloads,likes,createdAt,lastModified" --sort downloads --no-truncate --limit 25
# search for latest publish
hf models ls --search "Muse-Glimmer" --apps llama.cpp --expand "downloads,likes,createdAt,lastModified" --sort created_at --no-truncate --limit 25
# search for latest tunning
hf models ls --search "Muse-Glimmer" --apps llama.cpp --expand "downloads,likes,createdAt,lastModified" --sort last_modified --no-truncate --limit 25
</syntaxhighlight>
|-
| optimize Qwen3-Coder ||
<syntaxhighlight lang="bash">
hf models ls --search "coder" --apps llama.cpp --sort downloads --limit 1 --format json | jq .
hf models ls -h "unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF"
hf download "unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF" --include "*UD-Q4_K_XL*"
</syntaxhighlight>
|-
| optimize gemma-4-E4B ||
<syntaxhighlight lang="bash">
hf models ls -h unsloth/gemma-4-E4B-it-qat-GGUF
hf download unsloth/gemma-4-E4B-it-qat-GGUF --include "*UD-Q4_K_XL*"
hf download unsloth/gemma-4-E4B-it-qat-GGUF --include "mmproj-BF16.gguf"
hf download unsloth/gemma-4-E4B-it-qat-GGUF --include "mtp-gemma-4-E4B-it.gguf"
</syntaxhighlight>
|-
| optimize gemma-4-12B ||
<syntaxhighlight lang="bash">
hf models ls -h unsloth/gemma-4-12B-it-qat-GGUF
hf download unsloth/gemma-4-12B-it-qat-GGUF --include "*UD-Q4_K_XL*"
hf download unsloth/gemma-4-12B-it-qat-GGUF --include "mmproj-BF16.gguf"
hf download unsloth/gemma-4-12B-it-qat-GGUF --include "mtp-gemma-4-12B-it.gguf"
</syntaxhighlight>
|-
| optimize Muse Glimmer ||
<syntaxhighlight lang="bash">
hf models ls -h unsloth/Muse-Glimmer-30B-GGUF
hf download unsloth/Muse-Glimmer-30B-GGUF --include "*UD-Q4_K_XL*"
hf download unsloth/Muse-Glimmer-30B-GGUF --include "*UD-Q5_K_XL*"
hf download unsloth/Muse-Glimmer-30B-GGUF --include "*UD-Q6_K_XL*"
hf download unsloth/Muse-Glimmer-30B-GGUF --include "dflash-kquant.gguf"
hf download unsloth/Muse-Glimmer-30B-GGUF --include "mmproj-kquant.gguf"
</syntaxhighlight>
|-
|  ||
<syntaxhighlight lang="bash">
tree ~/.cache/huggingface/hub/models--unsloth--Devstral-Small-2-24B-Instruct-2512-GGUF/snapshots
tree ~/.cache/huggingface/hub/models--unsloth--Qwen3-Coder-30B-A3B-Instruct-GGUF/snapshots
</syntaxhighlight>
|}
== Ubuntu ==
* [[LLM/Intel Pro Arc B70]]
* [[LLM/Ryzen AI 7 350]] (XDNA)

Latest revision as of 11:16, 13 August 2026

Topics

🤖 Agent 🧠 Model 🚀 Runner ⚙️ GPU & Driver
axe
SKILL
MCP
hf (Model Management)
Muse Glimmer
Devstral Small 2
Qwen 3.6 27B
Gemma 4
SGLang
llama-swap
llama.cpp for SYCL
llama.cpp for OpenVINO
lemonade for AMD NPU
Intel Arc Pro B70
Install driver
Install oneAPI (SYCL)
Install OpenVINO
Experiment
AMD Ryzen AI 7 350
Experiment
Tools
Install nvtop

News

Decision Tree for tinkering with Intel Arc Pro B70

Retrieved from "https://wiki.fundamental-ramen.com/index.php?title=LM&oldid=2826"