LM: Difference between revisions

From Fundamental Ramen
Jump to navigation Jump to search
 
(95 intermediate revisions by the same user not shown)
Line 1: Line 1:
= Build environment =
= Topics =
 
== hf ==
* https://huggingface.co/docs/huggingface_hub/package_reference/environment_variables


{| class="wikitable"
{| class="wikitable"
! Purpose || Command
! 🤖 Agent || 🧠 Model || 🚀 Runner || ⚙️ GPU & Driver
|-
|-
| ||
| style="width:200px;vertical-align:top" |
<syntaxhighlight lang="bash">
: [[LM/jrswab_axe|axe]]
hf cache list
: SKILL
hf cache rm <model id>
: MCP
hf cache prune
| style="width:200px;vertical-align:top" |
</syntaxhighlight>
: [[LM/hf|hf (Model Management)]]
|-
: Muse Glimmer
| ||
: Devstral Small 2
<syntaxhighlight lang="bash">
: Qwen 3.6 27B
hf models ls --search "gemma-4" --apps llama.cpp --sort downloads --limit 10
: Gemma 4
hf models ls --search "coder" --apps llama.cpp --sort downloads --limit 10
| style="width:200px;vertical-align:top" |
hf models ls --search "unsloth" --apps llama.cpp --sort downloads --limit 10
: [[LM/SGLang|SGLang]]
</syntaxhighlight>
: [[LM/llama-swap|llama-swap]]
|-
: [[LM/llama.cpp for SYCL|llama.cpp for SYCL]]
| ||
: [[LM/llama.cpp for OpenVINO|llama.cpp for OpenVINO]]
<syntaxhighlight lang="bash">
: [[LM/lemonade for AMD NPU|lemonade for AMD NPU]]
hf models ls --search "coder" --apps llama.cpp --sort downloads --limit 1 --format json | jq .
| style="width:200px;vertical-align:top" |
hf models ls -h "unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF"
: Intel Arc Pro B70
hf download "unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF" --include "*UD-Q4_K_XL*"
:: [[LM/Install driver|Install driver]]
HF_XET_FIXED_DOWNLOAD_CONCURRENCY=10 hf download "unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF" --include "*UD-Q4_K_XL*"
:: [[LM/Install oneAPI|Install oneAPI (SYCL)]]
HF_XET_FIXED_DOWNLOAD_CONCURRENCY=10 hf download "unsloth/Devstral-Small-2-24B-Instruct-2512-GGUF" --include "*UD-Q4_K_XL*"
:: [[LM/Install OpenVINO|Install OpenVINO]]
:: [[LM/Intel Pro Arc B70|Experiment]]
: AMD Ryzen AI 7 350
:: [[LM/Ryzen AI 7 350|Experiment]]
: Tools
:: [[LM/Install nvtop|Install nvtop]]
|}
 
= News =
 
* [https://huggingface.co/datasets/ScaleAI/SWE-bench_Pro?leaderboard_max_params=32B SWE Pro Benchmark for models < 32B]
* [https://storage.openvinotoolkit.org/repositories/openvino/packages/2026.3/linux/ OpenVINO 2026.3 supports Ubuntu 26] - 2026-08-04
 
= Decision Tree for tinkering with Intel Arc Pro B70 =
 
<quickmmd name="tinkering-b70">
flowchart LR
  subgraph hardware
    A[Got B70]
  end
 
  subgraph os
    B1[Install<br>Ubuntu 24.04]:::node-okay
    B2[Install<br>Ubuntu 26.04]:::node-warn
    A --> B1
    A --> B2
  end


hf download unsloth/gemma-4-E4B-it-qat-GGUF --include "*UD-Q4_K_XL*"
  subgraph drvier
hf download unsloth/gemma-4-E4B-it-qat-GGUF --include "mmproj-BF16.gguf"
    C1[Install<br>PPA driver]:::node-okay
hf download unsloth/gemma-4-E4B-it-qat-GGUF --include "mtp-gemma-4-E4B-it.gguf"
    C2[Install<br>oneAPI]:::node-okay
</syntaxhighlight>
    C3[Install<br>OpenVINO]:::node-okay
|}
    C4[Install<br>PPA driver]:::node-warn
    C5[Client GPU not listed in<br>oneAPI 2026.1.0]:::node-fail
    C6[Cannot install<br>OpenVINO 2026.2.1]:::node-fail
    C1 --> C2 & C3
    C4 --> C5 & C6
    B1 --> C1
    B2 --> C4
  end


== Ubuntu ==
  subgraph runtime
    D1[Build llama.cpp for<br>SYCL]:::node-okay
    D2[Build llama.cpp for<br>OpenVINO]:::node-okay
    D3[Build SGLang for<br>SYCL]:::node-todo
    D4[Build SGLang for<br>OpenVINO]:::node-todo
    C2 --> D1
    C3 --> D2
    C2 --> D3
    C3 --> D4
  end


* [[LLM/Intel Pro Arc B70]]
  subgraph tools
* [[LLM/Ryzen AI 7 350]] (XDNA)
    T1[Install<br>pipx]
    T2[Install<br>hf]
    T3[Install<br>nvtop]
    T4[Install<br>screen]
    T5[Install<br>llama-swap]
    T1 --> T2
    B1 --> T1
    B1 --> T3
    B1 --> T4
    B1 --> T5
  end


== Agents ==
  subgraph models
    M1[Muse-Glimmer-30B]
    M2[Devstral-Small-2]
    M3[Qwen3-Coder-30B]
    M4[Gemma-4-E4B]
    T2 --> M1 & M2 & M3 & M4
  end


* [[LLM/jrswab_axe]]
  subgraph env
    V1[Optimize<br>.bashrc]
    V2[Optimize<br>llama-swap.yaml]
    C2 --> V1
    C3 --> V1
    T2 --> V1
    T4 ---> V1
    T5 ---> V2
  end


== Wishlist ==
  subgraph service
    Z[service]
    M1 --> Z
    D1 --> Z
    V2 --> Z
  end


* [[LLM/prompt for coding]]
  classDef node-okay fill:#efe,stroke:#393
  classDef node-warn fill:#fff0e0,stroke:#d63
  classDef node-fail fill:#fee,stroke:#d33
  classDef node-todo fill:#eee,stroke:#777
</quickmmd>

Latest revision as of 11:16, 13 August 2026

Topics

🤖 Agent 🧠 Model 🚀 Runner ⚙️ GPU & Driver
axe
SKILL
MCP
hf (Model Management)
Muse Glimmer
Devstral Small 2
Qwen 3.6 27B
Gemma 4
SGLang
llama-swap
llama.cpp for SYCL
llama.cpp for OpenVINO
lemonade for AMD NPU
Intel Arc Pro B70
Install driver
Install oneAPI (SYCL)
Install OpenVINO
Experiment
AMD Ryzen AI 7 350
Experiment
Tools
Install nvtop

News

Decision Tree for tinkering with Intel Arc Pro B70

Retrieved from "https://wiki.fundamental-ramen.com/index.php?title=LM&oldid=2826"