LM: Difference between revisions

From Fundamental Ramen
Jump to navigation Jump to search
 
(68 intermediate revisions by the same user not shown)
Line 1: Line 1:
= Topics =
{| class="wikitable"
! 🤖 Agent || 🧠 Model || 🚀 Runner || ⚙️ GPU & Driver
|-
| style="width:200px;vertical-align:top" |
: [[LM/jrswab_axe|axe]]
: SKILL
: MCP
| style="width:200px;vertical-align:top" |
: [[LM/hf|hf (Model Management)]]
: Muse Glimmer
: Devstral Small 2
: Qwen 3.6 27B
: Gemma 4
| style="width:200px;vertical-align:top" |
: [[LM/SGLang|SGLang]]
: [[LM/llama-swap|llama-swap]]
: [[LM/llama.cpp for SYCL|llama.cpp for SYCL]]
: [[LM/llama.cpp for OpenVINO|llama.cpp for OpenVINO]]
: [[LM/lemonade for AMD NPU|lemonade for AMD NPU]]
| style="width:200px;vertical-align:top" |
: Intel Arc Pro B70
:: [[LM/Install driver|Install driver]]
:: [[LM/Install oneAPI|Install oneAPI (SYCL)]]
:: [[LM/Install OpenVINO|Install OpenVINO]]
:: [[LM/Intel Pro Arc B70|Experiment]]
: AMD Ryzen AI 7 350
:: [[LM/Ryzen AI 7 350|Experiment]]
: Tools
:: [[LM/Install nvtop|Install nvtop]]
|}
= News =
* [https://huggingface.co/datasets/ScaleAI/SWE-bench_Pro?leaderboard_max_params=32B SWE Pro Benchmark for models < 32B]
* [https://storage.openvinotoolkit.org/repositories/openvino/packages/2026.3/linux/ OpenVINO 2026.3 supports Ubuntu 26] - 2026-08-04
= Decision Tree for tinkering with Intel Arc Pro B70 =
= Decision Tree for tinkering with Intel Arc Pro B70 =


<quickmmd name="tinkering-b70">
<quickmmd name="tinkering-b70">
flowchart LR
flowchart LR
   subgraph zero
   subgraph hardware
     A[Got B70]
     A[Got B70]
   end
   end


   subgraph os
   subgraph os
     B1[Install Ubuntu 24.04]
     B1[Install<br>Ubuntu 24.04]:::node-okay
     B2[Install Ubuntu 26.04]
     B2[Install<br>Ubuntu 26.04]:::node-warn
     A --> B1
     A --> B1
     A --> B2
     A --> B2
Line 15: Line 53:


   subgraph drvier
   subgraph drvier
     C1[Install onAPI]
     C1[Install<br>PPA driver]:::node-okay
     C2[Install OpenVINO]
     C2[Install<br>oneAPI]:::node-okay
     C3[Install onAPI]
     C3[Install<br>OpenVINO]:::node-okay
     C4[Install OpenVINO]
     C4[Install<br>PPA driver]:::node-warn
    C5[Client GPU not listed in<br>oneAPI 2026.1.0]:::node-fail
    C6[Cannot install<br>OpenVINO 2026.2.1]:::node-fail
    C1 --> C2 & C3
    C4 --> C5 & C6
     B1 --> C1
     B1 --> C1
    B1 --> C2
    B2 --> C3
     B2 --> C4
     B2 --> C4
   end
   end
</quickmmd>


= Build environment =
  subgraph runtime
    D1[Build llama.cpp for<br>SYCL]:::node-okay
    D2[Build llama.cpp for<br>OpenVINO]:::node-okay
    D3[Build SGLang for<br>SYCL]:::node-todo
    D4[Build SGLang for<br>OpenVINO]:::node-todo
    C2 --> D1
    C3 --> D2
    C2 --> D3
    C3 --> D4
  end


== hf (model management) ==
  subgraph tools
* https://huggingface.co/docs/huggingface_hub/package_reference/environment_variables
    T1[Install<br>pipx]
* https://www.datalearner.com/en/leaderboards/category/code?benchmark=SWE-bench+Verified&modelSize=34b&licenseType=open
    T2[Install<br>hf]
* https://huggingface.co/datasets/ScaleAI/SWE-bench_Pro
    T3[Install<br>nvtop]
    T4[Install<br>screen]
    T5[Install<br>llama-swap]
    T1 --> T2
    B1 --> T1
    B1 --> T3
    B1 --> T4
    B1 --> T5
  end


{| class="wikitable"
  subgraph models
! Purpose || Command
    M1[Muse-Glimmer-30B]
|-
    M2[Devstral-Small-2]
| cache management ||
    M3[Qwen3-Coder-30B]
<syntaxhighlight lang="bash">
    M4[Gemma-4-E4B]
hf cache list
    T2 --> M1 & M2 & M3 & M4
hf cache rm <model id>
  end
hf cache prune
</syntaxhighlight>
|-
| fix WiFi problem ||
<syntaxhighlight lang="bash">
HF_XET_FIXED_DOWNLOAD_CONCURRENCY=10 hf download "unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF" --include "*UD-Q4_K_XL*"
HF_XET_FIXED_DOWNLOAD_CONCURRENCY=10 hf download "unsloth/Devstral-Small-2-24B-Instruct-2512-GGUF" --include "*UD-Q4_K_XL*"
</syntaxhighlight>
|-
| search ||
<syntaxhighlight lang="bash">
hf models ls --search "gemma-4" --apps llama.cpp --sort downloads --limit 10
hf models ls --search "coder" --apps llama.cpp --sort downloads --limit 10
hf models ls --search "unsloth" --apps llama.cpp --sort downloads --limit 10
</syntaxhighlight>
|-
| optimize Qwen3-Coder ||
<syntaxhighlight lang="bash">
hf models ls --search "coder" --apps llama.cpp --sort downloads --limit 1 --format json | jq .
hf models ls -h "unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF"
hf download "unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF" --include "*UD-Q4_K_XL*"
</syntaxhighlight>
|-
| optimize gemma-4-E4B ||
<syntaxhighlight lang="bash">
hf models ls -h unsloth/gemma-4-E4B-it-qat-GGUF
hf download unsloth/gemma-4-E4B-it-qat-GGUF --include "*UD-Q4_K_XL*"
hf download unsloth/gemma-4-E4B-it-qat-GGUF --include "mmproj-BF16.gguf"
hf download unsloth/gemma-4-E4B-it-qat-GGUF --include "mtp-gemma-4-E4B-it.gguf"
</syntaxhighlight>
|-
| optimize gemma-4-12B ||
<syntaxhighlight lang="bash">
hf models ls -h unsloth/gemma-4-12B-it-qat-GGUF
hf download unsloth/gemma-4-12B-it-qat-GGUF --include "*UD-Q4_K_XL*"
hf download unsloth/gemma-4-12B-it-qat-GGUF --include "mmproj-BF16.gguf"
hf download unsloth/gemma-4-12B-it-qat-GGUF --include "mtp-gemma-4-12B-it.gguf"
</syntaxhighlight>
|-
| optimize Muse Glimmer ||
<syntaxhighlight lang="bash">
hf models ls -h unsloth/Muse-Glimmer-30B-GGUF
hf download unsloth/Muse-Glimmer-30B-GGUF --include "*UD-Q4_K_XL*"
hf download unsloth/Muse-Glimmer-30B-GGUF --include "*UD-Q5_K_XL*"
hf download unsloth/Muse-Glimmer-30B-GGUF --include "*UD-Q6_K_XL*"
hf download unsloth/Muse-Glimmer-30B-GGUF --include "dflash-kquant.gguf"
hf download unsloth/Muse-Glimmer-30B-GGUF --include "mmproj-kquant.gguf"
</syntaxhighlight>
|-
|  ||
<syntaxhighlight lang="bash">
tree ~/.cache/huggingface/hub/models--unsloth--Devstral-Small-2-24B-Instruct-2512-GGUF/snapshots
tree ~/.cache/huggingface/hub/models--unsloth--Qwen3-Coder-30B-A3B-Instruct-GGUF/snapshots
</syntaxhighlight>
|}


== Ubuntu ==
  subgraph env
    V1[Optimize<br>.bashrc]
    V2[Optimize<br>llama-swap.yaml]
    C2 --> V1
    C3 --> V1
    T2 --> V1
    T4 ---> V1
    T5 ---> V2
  end


* [[LLM/Intel Pro Arc B70]]
  subgraph service
* [[LLM/Ryzen AI 7 350]] (XDNA)
    Z[service]
    M1 --> Z
    D1 --> Z
    V2 --> Z
  end


== Agents ==
  classDef node-okay fill:#efe,stroke:#393
 
  classDef node-warn fill:#fff0e0,stroke:#d63
* [[LLM/jrswab_axe]]
  classDef node-fail fill:#fee,stroke:#d33
 
  classDef node-todo fill:#eee,stroke:#777
== Wishlist ==
</quickmmd>
 
* [[LLM/prompt for coding]]

Latest revision as of 11:16, 13 August 2026

Topics

🤖 Agent 🧠 Model 🚀 Runner ⚙️ GPU & Driver
axe
SKILL
MCP
hf (Model Management)
Muse Glimmer
Devstral Small 2
Qwen 3.6 27B
Gemma 4
SGLang
llama-swap
llama.cpp for SYCL
llama.cpp for OpenVINO
lemonade for AMD NPU
Intel Arc Pro B70
Install driver
Install oneAPI (SYCL)
Install OpenVINO
Experiment
AMD Ryzen AI 7 350
Experiment
Tools
Install nvtop

News

Decision Tree for tinkering with Intel Arc Pro B70

Retrieved from "https://wiki.fundamental-ramen.com/index.php?title=LM&oldid=2826"