LM: Difference between revisions

From Fundamental Ramen
Jump to navigation Jump to search
 
(47 intermediate revisions by the same user not shown)
Line 1: Line 1:
= Topics =
{| class="wikitable"
! 🤖 Agent || 🧠 Model || 🚀 Runner || ⚙️ GPU & Driver
|-
| style="width:200px;vertical-align:top" |
: [[LM/jrswab_axe|axe]]
: SKILL
: MCP
| style="width:200px;vertical-align:top" |
: [[LM/hf|hf (Model Management)]]
: Muse Glimmer
: Devstral Small 2
: Qwen 3.6 27B
: Gemma 4
| style="width:200px;vertical-align:top" |
: [[LM/SGLang|SGLang]]
: [[LM/llama-swap|llama-swap]]
: [[LM/llama.cpp for SYCL|llama.cpp for SYCL]]
: [[LM/llama.cpp for OpenVINO|llama.cpp for OpenVINO]]
: [[LM/lemonade for AMD NPU|lemonade for AMD NPU]]
| style="width:200px;vertical-align:top" |
: Intel Arc Pro B70
:: [[LM/Install driver|Install driver]]
:: [[LM/Install oneAPI|Install oneAPI (SYCL)]]
:: [[LM/Install OpenVINO|Install OpenVINO]]
:: [[LM/Intel Pro Arc B70|Experiment]]
: AMD Ryzen AI 7 350
:: [[LM/Ryzen AI 7 350|Experiment]]
: Tools
:: [[LM/Install nvtop|Install nvtop]]
|}
= News =
* [https://huggingface.co/datasets/ScaleAI/SWE-bench_Pro?leaderboard_max_params=32B SWE Pro Benchmark for models < 32B]
* [https://storage.openvinotoolkit.org/repositories/openvino/packages/2026.3/linux/ OpenVINO 2026.3 supports Ubuntu 26] - 2026-08-04
= Decision Tree for tinkering with Intel Arc Pro B70 =
= Decision Tree for tinkering with Intel Arc Pro B70 =


Line 19: Line 57:
     C3[Install<br>OpenVINO]:::node-okay
     C3[Install<br>OpenVINO]:::node-okay
     C4[Install<br>PPA driver]:::node-warn
     C4[Install<br>PPA driver]:::node-warn
     C5[Cannot install<br>oneAPI]:::node-fail
     C5[Client GPU not listed in<br>oneAPI 2026.1.0]:::node-fail
     C6[Cannot install<br>OpenVINO]:::node-fail
     C6[Cannot install<br>OpenVINO 2026.2.1]:::node-fail
     C1 --> C2 & C3
     C1 --> C2 & C3
     C4 --> C5 & C6
     C4 --> C5 & C6
Line 49: Line 87:
     B1 --> T4
     B1 --> T4
     B1 --> T5
     B1 --> T5
  end
  subgraph models
    M1[Muse-Glimmer-30B]
    M2[Devstral-Small-2]
    M3[Qwen3-Coder-30B]
    M4[Gemma-4-E4B]
    T2 --> M1 & M2 & M3 & M4
   end
   end


Line 63: Line 109:
   subgraph service
   subgraph service
     Z[service]
     Z[service]
    M1 --> Z
     D1 --> Z
     D1 --> Z
     V2 --> Z
     V2 --> Z
Line 72: Line 119:
   classDef node-todo fill:#eee,stroke:#777
   classDef node-todo fill:#eee,stroke:#777
</quickmmd>
</quickmmd>
= Build environment =
== hf (model management) ==
* https://huggingface.co/docs/huggingface_hub/package_reference/environment_variables
* https://www.datalearner.com/en/leaderboards/category/code?benchmark=SWE-bench+Verified&modelSize=34b&licenseType=open
* https://huggingface.co/datasets/ScaleAI/SWE-bench_Pro
{| class="wikitable"
! Purpose || Command
|-
| cache management ||
<syntaxhighlight lang="bash">
hf cache list
hf cache rm <model id>
hf cache prune
</syntaxhighlight>
|-
| fix WiFi problem ||
<syntaxhighlight lang="bash">
HF_XET_FIXED_DOWNLOAD_CONCURRENCY=10 hf download "unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF" --include "*UD-Q4_K_XL*"
HF_XET_FIXED_DOWNLOAD_CONCURRENCY=10 hf download "unsloth/Devstral-Small-2-24B-Instruct-2512-GGUF" --include "*UD-Q4_K_XL*"
</syntaxhighlight>
|-
| search ||
<syntaxhighlight lang="bash">
# search by population
hf models ls --search "Muse-Glimmer" --apps llama.cpp --expand "downloads,likes,createdAt,lastModified" --sort downloads --no-truncate --limit 25
# search for latest publish
hf models ls --search "Muse-Glimmer" --apps llama.cpp --expand "downloads,likes,createdAt,lastModified" --sort created_at --no-truncate --limit 25
# search for latest tunning
hf models ls --search "Muse-Glimmer" --apps llama.cpp --expand "downloads,likes,createdAt,lastModified" --sort last_modified --no-truncate --limit 25
</syntaxhighlight>
|-
| optimize Qwen3-Coder ||
<syntaxhighlight lang="bash">
hf models ls --search "coder" --apps llama.cpp --sort downloads --limit 1 --format json | jq .
hf models ls -h "unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF"
hf download "unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF" --include "*UD-Q4_K_XL*"
</syntaxhighlight>
|-
| optimize gemma-4-E4B ||
<syntaxhighlight lang="bash">
hf models ls -h unsloth/gemma-4-E4B-it-qat-GGUF
hf download unsloth/gemma-4-E4B-it-qat-GGUF --include "*UD-Q4_K_XL*"
hf download unsloth/gemma-4-E4B-it-qat-GGUF --include "mmproj-BF16.gguf"
hf download unsloth/gemma-4-E4B-it-qat-GGUF --include "mtp-gemma-4-E4B-it.gguf"
</syntaxhighlight>
|-
| optimize gemma-4-12B ||
<syntaxhighlight lang="bash">
hf models ls -h unsloth/gemma-4-12B-it-qat-GGUF
hf download unsloth/gemma-4-12B-it-qat-GGUF --include "*UD-Q4_K_XL*"
hf download unsloth/gemma-4-12B-it-qat-GGUF --include "mmproj-BF16.gguf"
hf download unsloth/gemma-4-12B-it-qat-GGUF --include "mtp-gemma-4-12B-it.gguf"
</syntaxhighlight>
|-
| optimize Muse Glimmer ||
<syntaxhighlight lang="bash">
hf models ls -h unsloth/Muse-Glimmer-30B-GGUF
hf download unsloth/Muse-Glimmer-30B-GGUF --include "*UD-Q4_K_XL*"
hf download unsloth/Muse-Glimmer-30B-GGUF --include "*UD-Q5_K_XL*"
hf download unsloth/Muse-Glimmer-30B-GGUF --include "*UD-Q6_K_XL*"
hf download unsloth/Muse-Glimmer-30B-GGUF --include "dflash-kquant.gguf"
hf download unsloth/Muse-Glimmer-30B-GGUF --include "mmproj-kquant.gguf"
</syntaxhighlight>
|-
|  ||
<syntaxhighlight lang="bash">
tree ~/.cache/huggingface/hub/models--unsloth--Devstral-Small-2-24B-Instruct-2512-GGUF/snapshots
tree ~/.cache/huggingface/hub/models--unsloth--Qwen3-Coder-30B-A3B-Instruct-GGUF/snapshots
</syntaxhighlight>
|}
== Ubuntu ==
* [[LLM/Intel Pro Arc B70]]
* [[LLM/Ryzen AI 7 350]] (XDNA)
== Agents ==
* [[LLM/jrswab_axe]]
== Wishlist ==
* [[LLM/prompt for coding]]

Latest revision as of 11:16, 13 August 2026

Topics

🤖 Agent 🧠 Model 🚀 Runner ⚙️ GPU & Driver
axe
SKILL
MCP
hf (Model Management)
Muse Glimmer
Devstral Small 2
Qwen 3.6 27B
Gemma 4
SGLang
llama-swap
llama.cpp for SYCL
llama.cpp for OpenVINO
lemonade for AMD NPU
Intel Arc Pro B70
Install driver
Install oneAPI (SYCL)
Install OpenVINO
Experiment
AMD Ryzen AI 7 350
Experiment
Tools
Install nvtop

News

Decision Tree for tinkering with Intel Arc Pro B70

Retrieved from "https://wiki.fundamental-ramen.com/index.php?title=LM&oldid=2826"