LM: Difference between revisions

From Fundamental Ramen
Jump to navigation Jump to search
 
(48 intermediate revisions by the same user not shown)
Line 2: Line 2:


{| class="wikitable"
{| class="wikitable"
! 🤖 Agent || 🧠 Model || 🚀 Runner || ⚙️ GPU & Driver
! 🤖 Agent　 || 🧠 Model　 || 🚀 Engine　 || ⚙️ GPU & Driver　|| Evaluation
|-
|-
| style="width:200px;vertical-align:top" |
| style="width:200px;vertical-align:top" |
: [[LM/Continue.dev|Continue.dev]]
: [[LM/ZooCode|ZooCode]]
: [[LM/jrswab_axe|axe]]
: [[LM/jrswab_axe|axe]]
: SKILL
: [[LM/MCP|MCP]]
: MCP
| style="width:200px;vertical-align:top" |
| style="width:200px;vertical-align:top" |
: [[LM/hf|hf (Model Management)]]
: [[LM/hf|hf (Model Management)]]
: [[LM/Devstral Small 2]]
: [[LM/Qwen 3.8 27B|Qwen 3.8 27B]]
: [[LM/starcoder2 3B|starcoder2 3B]]
: [[LM/Gemma 4|Gemma 4]]
: [[LM/Qwen 3.6 27B]]
: [[LM/Devstral Small 2 24B|Devstral Small 2 24B]]
: [[LM/Muse Glimmer]]
: [[LM/Muse Glimmer|Muse Glimmer]]
: [[LM/Gemma 4]]
: [[LM/Experience|Experience]]
: [[LM/Experience]]
| style="width:200px;vertical-align:top" |
| style="width:200px;vertical-align:top" |
: [[LM/vLLM|vLLM]]
: [[LM/llama-swap|llama-swap]]
: [[LM/llama-swap|llama-swap]]
: [[LM/lact|lact]]
: [[LM/llama.cpp for SYCL|llama.cpp for SYCL]]
: [[LM/llama.cpp for SYCL|llama.cpp for SYCL]]
: [[LM/llama.cpp for CUDA|llama.cpp for CUDA]]
: [[LM/SGLang|SGLang]]
: [[LM/OVMS|OVMS]]
: [[LM/llama.cpp for OpenVINO|llama.cpp for OpenVINO]]
: [[LM/llama.cpp for OpenVINO|llama.cpp for OpenVINO]]
: [[LM/llama.cpp for TurboQuant|llama.cpp for TurboQuant]]
: [[LM/lemonade for AMD NPU|lemonade for AMD NPU]]
: [[LM/lemonade for AMD NPU|lemonade for AMD NPU]]
: [[LM/SGLang|SGLang]]
: [[LM/vLLM|vLLM]]
| style="width:200px;vertical-align:top" |
| style="width:200px;vertical-align:top" |
: Intel Arc Pro B70
: Intel Arc Pro B70
Line 29: Line 31:
:: [[LM/Install oneAPI|Install oneAPI (SYCL)]]
:: [[LM/Install oneAPI|Install oneAPI (SYCL)]]
:: [[LM/Install OpenVINO|Install OpenVINO]]
:: [[LM/Install OpenVINO|Install OpenVINO]]
:: [[LM/.bashrc for B70|.bashrc for B70]]
:: [[LM/Intel Pro Arc B70|Experiment]]
:: [[LM/Intel Pro Arc B70|Experiment]]
: NVIDIA RTX 3060
:: [[LM/Install NVIDIA driver|Install driver]]
: AMD Ryzen AI 7 350
: AMD Ryzen AI 7 350
:: [[LM/Ryzen AI 7 350|Experiment]]
:: [[LM/Ryzen AI 7 350|Experiment]]
: Tools
: Tools
:: [[LM/Install nvtop|Install nvtop]]
:: [[LM/Install nvtop|Install nvtop]]
:: [[LM/PCIe Trouble Shooting|PCIe Trouble Shooting]]
| style="width:200px;vertical-align:top" |
: [[LM/Performance|Performance]]
: [[LM/Security|Security]]
: [[LM/Hallucination|Hallucination]]
|}
|}


Line 44: Line 54:


<quickmmd name="tinkering-b70">
<quickmmd name="tinkering-b70">
flowchart LR
flowchart TD
   subgraph hardware
   subgraph Hardware
     A(Got B70)
     A1(Intel Arc Pro B70):::node-okay
    A2(Asus ProArt B850 WiFi NEO):::node-okay
    A3(AMD Ryzen 5 9600X):::node-okay
   end
   end


   subgraph os
   subgraph OS
     B1(Install<br>Ubuntu 24.04):::node-okay
     B1(Install<br>Ubuntu 26.04):::node-okay
     B2(Install<br>Ubuntu 26.04):::node-warn
     A1 & A2 & A3 --> B1
    A --> B1
    A --> B2
   end
   end


   subgraph drvier
   subgraph Driver
     C1(Install<br>PPA driver):::node-okay
     C1(Install PPA driver<br>26.27.39122.14):::node-okay
     C2(Install<br>oneAPI):::node-okay
     C2(Install oneAPI<br>2026.1.1.20260724):::node-okay
     C3(Install<br>OpenVINO):::node-okay
     C3(Install OpenVINO<br>2026.3.0<br>hard to install):::node-warn
    C4(Install<br>PPA driver):::node-warn
    C5(Client GPU not listed in<br>oneAPI 2026.1.0):::node-fail
    C6(Cannot install<br>OpenVINO 2026.2.1):::node-fail
     C1 --> C2 & C3
     C1 --> C2 & C3
    C4 --> C5 & C6
     B1 --> C1
     B1 --> C1
    B2 --> C4
   end
   end


   subgraph runtime
   subgraph Runtime
     D1[Build llama.cpp for<br>SYCL]:::node-okay
     D1[Build llama.cpp for<br>SYCL]:::node-okay
     D2[Build llama.cpp for<br>OpenVINO]:::node-okay
     D2[Build llama.cpp for<br>OpenVINO<br>not stable yet]:::node-warn
     D3[Build SGLang for<br>SYCL]:::node-todo
     %% D3[Build SGLang for<br>SYCL]:::node-todo
     D4[Build SGLang for<br>OpenVINO]:::node-todo
     %% D4[Build SGLang for<br>OpenVINO]:::node-todo
     C2 --> D1
     C2 --> D1
     C3 --> D2
     C3 --> D2
     C2 --> D3
     %% C2 --> D3
     C3 --> D4
     %% C3 --> D4
   end
   end


   subgraph tools
   subgraph Tools
     T1(Install<br>pipx)
     T1(Install<br>pipx)
     T2(Install<br>hf)
     T2(Install<br>hf)
     T3(Install<br>nvtop)
     T3(Install<br>nvtop)
     T4(Install<br>tmux)
     T4(Install<br>tmux)
     T5(Install<br>llama-swap)
     T5(Install<br>lact)
    T6(Install<br>llama-swap)
     T1 --> T2
     T1 --> T2
     B1 --> T1
     B1 --> T1
Line 91: Line 97:
     B1 --> T4
     B1 --> T4
     B1 --> T5
     B1 --> T5
    T5 --> T6
   end
   end


   subgraph models
   subgraph Models
     M1(Muse-Glimmer-30B)
     M1(Qwen3.8-27B)
     M2(Devstral-Small-2)
     M2(Devstral-Small-2-24B)
    M3(Qwen3-Coder-30B)
     T2 --> M1 & M2
    M4(Gemma-4-E4B)
     T2 --> M1 & M2 & M3 & M4
   end
   end


   subgraph env
   subgraph config
     V1(Optimize<br>.bashrc)
     V1(Optimize<br>.bashrc)
     V2(Optimize<br>llama-swap.yaml)
     V2(Optimize<br>lmsw.yaml)
     C2 --> V1
     C2 --> V1
     C3 --> V1
     C3 --> V1
     T2 --> V1
     T2 --> V1
     T4 ---> V1
     T4 ---> V1
     T5 ---> V2
     T6 ---> V2
   end
   end


   subgraph service
   subgraph Service
     Z(service)
     Z(service<br>llama-swap \<br>  -config lmsw.yaml \<br>  -listen 0.0.0.0:9876)
     M1 --> Z
     M1 --> Z
     D1 --> Z
     D1 --> Z

Latest revision as of 03:50, 9 September 2026

Topics

🤖 Agent　 🧠 Model　 🚀 Engine　 ⚙️ GPU & Driver　 Evaluation
ZooCode
axe
MCP
hf (Model Management)
Qwen 3.8 27B
Gemma 4
Devstral Small 2 24B
Muse Glimmer
Experience
llama-swap
lact
llama.cpp for SYCL
llama.cpp for CUDA
SGLang
OVMS
llama.cpp for OpenVINO
llama.cpp for TurboQuant
lemonade for AMD NPU
vLLM
Intel Arc Pro B70
Install driver
Install oneAPI (SYCL)
Install OpenVINO
.bashrc for B70
Experiment
NVIDIA RTX 3060
Install driver
AMD Ryzen AI 7 350
Experiment
Tools
Install nvtop
PCIe Trouble Shooting
Performance
Security
Hallucination

News

Decision Tree for tinkering with Intel Arc Pro B70

Retrieved from "https://wiki.fundamental-ramen.com/index.php?title=LM&oldid=3302"