LM/SGLang: Difference between revisions
< LM
| Line 54: | Line 54: | ||
|url=https://docs.sglang.io/docs/advanced_features/server_arguments#server-arguments | |url=https://docs.sglang.io/docs/advanced_features/server_arguments#server-arguments | ||
|title=SGLang server arguments | |title=SGLang server arguments | ||
|site=docs.sglang.io | |||
}} | |||
</ref> | |||
<ref> | |||
{{ref web | |||
|url=https://docs.sglang.io/cookbook/autoregressive/Qwen/Qwen3#3-1-basic-configuration | |||
|title=SGLang cookbook for Qwen3 + BMG | |||
|site=docs.sglang.io | |site=docs.sglang.io | ||
}} | }} | ||
Revision as of 01:52, 11 November 2026
Engine works fine, but most models cannot loaded in xpu mode.
Rebuild docker image from source
cd sglang
git pull
cd docker
docker build -t sglang-xpu:latest -f xpu.Dockerfile .
Run
docker run \
-it \
--privileged \
--ipc=host \
--network=host \
--user root \
--group-add $(getent group video | cut -d: -f3) \
--device /dev/dri \
-v /dev/dri/by-path:/dev/dri/by-path \
-v /dev/shm:/dev/shm \
-v ~/.cache/huggingface:/root/.cache/huggingface \
-p 30000:30000 \
-e "HF_TOKEN=$HF_TOKEN" \
sglang-xpu:latest /bin/bash
Structure
TODO
- ↑ Official SGLang docker image - hub.docker.com
- ↑ Intel patched SGLang docker image - hub.docker.com
- ↑ Hardware Platforms XPU - docs.sglang.io
- ↑ SGLang server arguments - docs.sglang.io
- ↑ SGLang cookbook for Qwen3 + BMG - docs.sglang.io