LLM/Ryzen AI 7 350/2026-07-09: Difference between revisions
< LLM | Ryzen AI 7 350
Jump to navigation
Jump to search
(Created page with "= Install lemonade from PPA =") |
|||
| (49 intermediate revisions by the same user not shown) | |||
| Line 1: | Line 1: | ||
= Install lemonade from PPA = | = Install lemonade from PPA = | ||
<syntaxhighlight lang="bash"> | |||
sudo add-apt-repository ppa:lemonade-team/stable | |||
sudo apt update | |||
sudo apt install lemonade-server | |||
</syntaxhighlight> | |||
== Run lemonade server == | |||
<syntaxhighlight lang="bash"> | |||
sudo systemctl start lemond | |||
sudo systemctl enable lemond | |||
</syntaxhighlight> | |||
== Install Backends == | |||
Open http://localhost:13305, then ... | |||
=== Install llama.cpp cuda === | |||
* Click backends at left sidebar. | |||
* Find Llama.cpp GPU | |||
* Find child item cuda | |||
* Click download | |||
<syntaxhighlight lang="bash"> | |||
sudo vim /etc/gdm3/custom.conf | |||
</syntaxhighlight> | |||
<pre> | |||
[daemon] | |||
# 強制 GDM 的 Wayland 渲染走 AMD 的基底 | |||
Environment=VK_LOADER_DRIVERS_SELECT=*radeon* | |||
Environment=MESA_VK_DEVICE_SELECT=pci-0000_66_00_0 | |||
</pre> | |||
<syntaxhighlight lang="bash"> | |||
sudo vim /etc/default/grub | |||
</syntaxhighlight> | |||
<pre> | |||
GRUB_CMDLINE_LINUX_DEFAULT="quiet splash nvidia-drm.modeset=0" | |||
</pre> | |||
<syntaxhighlight lang="bash"> | |||
sudo update-grub | |||
</syntaxhighlight> | |||
=== Install Fastflow === | |||
* Click backends at left sidebar. | |||
* Find FastFlowLM NPU | |||
* Find child item npu | |||
* Click download | |||
* Install XDNA driver | |||
<syntaxhighlight lang="bash"> | |||
sudo apt install amdxdna-dkms | |||
sudo reboot | |||
</syntaxhighlight> | |||
* Get FastFlowLM release from https://github.com/FastFlowLM/FastFlowLM/releases/ | |||
* Download the latest db for Ubuntu 26.04 | |||
<syntaxhighlight lang="bash"> | |||
cd 下載 | |||
sudo apt install ./fastflowlm_0.9.44_ubuntu26.04_amd64.deb | |||
flm --version | |||
flm validate | |||
</syntaxhighlight> | |||
<pre> | |||
Memlock limit is too low (8MB). Please ... | |||
</pre> | |||
=== Fix memory limit problem === | |||
<syntaxhighlight lang="bash"> | |||
sudo nano /etc/security/limits.conf | |||
</syntaxhighlight> | |||
Add before `# End of file` | |||
<pre> | |||
* soft memlock unlimited | |||
* hard memlock unlimited | |||
</pre> | |||
<syntaxhighlight lang="bash"> | |||
sudo reboot | |||
</syntaxhighlight> | |||
<syntaxhighlight lang="bash"> | |||
ulimit -l | |||
flm validate | |||
</syntaxhighlight> | |||
=== Enable NPU and set default models === | |||
<syntaxhighlight lang="bash"> | |||
sudo systemctl edit lemond | |||
</syntaxhighlight> | |||
<pre> | |||
[Service] | |||
# Enable NPU | |||
Environment="LEMONADE_ENABLE_NPU=true" | |||
# Prevent from model unloaded | |||
Environment="LEMONADE_MODEL_TTL=0" | |||
# Default models | |||
ExecStartPost=/usr/bin/lemonade load model --ctx-size 65535 Gemma-4-12B-it-GGUF --backend cuda | |||
ExecStartPost=/usr/bin/lemonade load model gpt-oss-sg-20b-FLM --backend npu | |||
</pre> | |||
<syntaxhighlight lang="bash"> | |||
sudo systemctl daemon-reload | |||
sudo systemctl restart lemond | |||
</syntaxhighlight> | |||
=== Let iGPU can use large memory size === | |||
Default iGPU memory limit is 536.87MB in Ubuntu 26.04 | |||
<syntaxhighlight lang="bash"> | |||
sudo nano /etc/modprobe.d/amdgpu.conf | |||
</syntaxhighlight> | |||
<pre> | |||
options amdgpu ttm_pages_limit=25600000 | |||
options amdgpu page_pool_size=25600000 | |||
</pre> | |||
<syntaxhighlight lang="bash"> | |||
sudo update-initramfs -u | |||
sudo reboot | |||
sudo dmesg | grep -iE "amdgpu.*memory" | |||
</syntaxhighlight> | |||
<pre> | |||
[ 5.304706] amdgpu 0000:66:00.0: Trusted Memory Zone (TMZ) feature disabled as experimental (default) | |||
[ 5.304936] amdgpu 0000:66:00.0: 512M of VRAM memory ready | |||
[ 5.304939] amdgpu 0000:66:00.0: 46677M of GTT memory ready. | |||
</pre> | |||
sudo apt install vulkan-tools | |||
sudo vim /var/lib/lemonade/.cache/lemonade/config.json | |||
== Optimize Lemonade config == | |||
Find Cache dir | |||
<syntaxhighlight lang="bash"> | |||
sudo journalctl -u lemond -f -n 100 | |||
</syntaxhighlight> | |||
<pre> | |||
7月 09 17:37:22 agentlab-zen5 lemond[411649]: 2026-07-09 17:37:22.869 [Info] (main) Starting Lemonade Server... | |||
7月 09 17:37:22 agentlab-zen5 lemond[411649]: 2026-07-09 17:37:22.869 [Info] (main) Version: 10.9.0 | |||
7月 09 17:37:22 agentlab-zen5 lemond[411649]: 2026-07-09 17:37:22.869 [Info] (main) Cache dir: /var/lib/lemonade/.cache/lemonade | |||
7月 09 17:37:22 agentlab-zen5 lemond[411649]: 2026-07-09 17:37:22.869 [Info] (main) Port: 13305 | |||
7月 09 17:37:22 agentlab-zen5 lemond[411649]: 2026-07-09 17:37:22.869 [Info] (main) Host: localhost | |||
7月 09 17:37:22 agentlab-zen5 lemond[411649]: 2026-07-09 17:37:22.869 [Info] (main) Log level: info | |||
7月 09 17:37:22 agentlab-zen5 lemond[411649]: 2026-07-09 17:37:22.869 [Info] (main) Telemetry: disabled | |||
</pre> | |||
Edit config.json | |||
<syntaxhighlight lang="bash"> | |||
sudo vim /var/lib/lemonade/.cache/lemonade/config.json | |||
</syntaxhighlight> | |||
* host: 0.0.0.0 | |||
* max_loaded_models: -1 | |||
<syntaxhighlight lang="bash"> | |||
sudo systemctl restart lemond | |||
</syntaxhighlight> | |||
Now Lemonade can load multiple models. | |||
== Debug == | |||
<syntaxhighlight lang="bash"> | |||
sudo journalctl -u lemond.service -n 50 --no-pager | |||
sudo journalctl -u lemond -f -n 100 | |||
</syntaxhighlight> | |||
Latest revision as of 10:18, 10 July 2026
Install lemonade from PPA
sudo add-apt-repository ppa:lemonade-team/stable
sudo apt update
sudo apt install lemonade-server
Run lemonade server
sudo systemctl start lemond
sudo systemctl enable lemond
Install Backends
Open http://localhost:13305, then ...
Install llama.cpp cuda
- Click backends at left sidebar.
- Find Llama.cpp GPU
- Find child item cuda
- Click download
sudo vim /etc/gdm3/custom.conf
[daemon] # 強制 GDM 的 Wayland 渲染走 AMD 的基底 Environment=VK_LOADER_DRIVERS_SELECT=*radeon* Environment=MESA_VK_DEVICE_SELECT=pci-0000_66_00_0
sudo vim /etc/default/grub
GRUB_CMDLINE_LINUX_DEFAULT="quiet splash nvidia-drm.modeset=0"
sudo update-grub
Install Fastflow
- Click backends at left sidebar.
- Find FastFlowLM NPU
- Find child item npu
- Click download
- Install XDNA driver
sudo apt install amdxdna-dkms
sudo reboot
- Get FastFlowLM release from https://github.com/FastFlowLM/FastFlowLM/releases/
- Download the latest db for Ubuntu 26.04
cd 下載
sudo apt install ./fastflowlm_0.9.44_ubuntu26.04_amd64.deb
flm --version
flm validate
Memlock limit is too low (8MB). Please ...
Fix memory limit problem
sudo nano /etc/security/limits.conf
Add before `# End of file`
* soft memlock unlimited * hard memlock unlimited
sudo reboot
ulimit -l
flm validate
Enable NPU and set default models
sudo systemctl edit lemond
[Service] # Enable NPU Environment="LEMONADE_ENABLE_NPU=true" # Prevent from model unloaded Environment="LEMONADE_MODEL_TTL=0" # Default models ExecStartPost=/usr/bin/lemonade load model --ctx-size 65535 Gemma-4-12B-it-GGUF --backend cuda ExecStartPost=/usr/bin/lemonade load model gpt-oss-sg-20b-FLM --backend npu
sudo systemctl daemon-reload
sudo systemctl restart lemond
Let iGPU can use large memory size
Default iGPU memory limit is 536.87MB in Ubuntu 26.04
sudo nano /etc/modprobe.d/amdgpu.conf
options amdgpu ttm_pages_limit=25600000 options amdgpu page_pool_size=25600000
sudo update-initramfs -u
sudo reboot
sudo dmesg | grep -iE "amdgpu.*memory"
[ 5.304706] amdgpu 0000:66:00.0: Trusted Memory Zone (TMZ) feature disabled as experimental (default) [ 5.304936] amdgpu 0000:66:00.0: 512M of VRAM memory ready [ 5.304939] amdgpu 0000:66:00.0: 46677M of GTT memory ready.
sudo apt install vulkan-tools sudo vim /var/lib/lemonade/.cache/lemonade/config.json
Optimize Lemonade config
Find Cache dir
sudo journalctl -u lemond -f -n 100
7月 09 17:37:22 agentlab-zen5 lemond[411649]: 2026-07-09 17:37:22.869 [Info] (main) Starting Lemonade Server... 7月 09 17:37:22 agentlab-zen5 lemond[411649]: 2026-07-09 17:37:22.869 [Info] (main) Version: 10.9.0 7月 09 17:37:22 agentlab-zen5 lemond[411649]: 2026-07-09 17:37:22.869 [Info] (main) Cache dir: /var/lib/lemonade/.cache/lemonade 7月 09 17:37:22 agentlab-zen5 lemond[411649]: 2026-07-09 17:37:22.869 [Info] (main) Port: 13305 7月 09 17:37:22 agentlab-zen5 lemond[411649]: 2026-07-09 17:37:22.869 [Info] (main) Host: localhost 7月 09 17:37:22 agentlab-zen5 lemond[411649]: 2026-07-09 17:37:22.869 [Info] (main) Log level: info 7月 09 17:37:22 agentlab-zen5 lemond[411649]: 2026-07-09 17:37:22.869 [Info] (main) Telemetry: disabled
Edit config.json
sudo vim /var/lib/lemonade/.cache/lemonade/config.json
- host: 0.0.0.0
- max_loaded_models: -1
sudo systemctl restart lemond
Now Lemonade can load multiple models.
Debug
sudo journalctl -u lemond.service -n 50 --no-pager
sudo journalctl -u lemond -f -n 100