LM/SGLang: Difference between revisions
< LM
| Line 102: | Line 102: | ||
= TODO = | = TODO = | ||
{{ref web | |||
|url=https://hub.docker.com/r/lmsysorg/sglang/tags?name=xpu | |||
|title=Official SGLang docker image | |||
|site=hub.docker.com | |||
}} | |||
{{ref web | |||
|url=https://docs.sglang.io/docs/hardware-platforms/xpu#xpu | |||
|title=Hardware Platforms XPU | |||
|site=docs.sglang.io | |||
}} | |||
{{ref web | {{ref web | ||
|url=https://hub.docker.com/r/ | |url=https://hub.docker.com/r/intel/sglang-dev | ||
|title=SGLang docker image | |title=Intel patched SGLang docker image | ||
|site=hub.docker.com | |site=hub.docker.com | ||
}} | |||
{{ref web | |||
|url=https://docs.sglang.io/docs/advanced_features/server_arguments#server-arguments | |||
|title=SGLang server arguments | |||
|site=docs.sglang.io | |||
}} | }} | ||
Revision as of 01:28, 11 November 2026
Engine works fine, but most models cannot loaded in xpu mode.
Rebuild docker image from source
cd sglang
git pull
cd docker
docker build -t sglang-xpu:latest -f xpu.Dockerfile .
Run
docker run \
-it \
--privileged \
--ipc=host \
--network=host \
--user root \
--group-add $(getent group video | cut -d: -f3) \
--device /dev/dri \
-v /dev/dri/by-path:/dev/dri/by-path \
-v /dev/shm:/dev/shm \
-v ~/.cache/huggingface:/root/.cache/huggingface \
-p 30000:30000 \
-e "HF_TOKEN=$HF_TOKEN" \
sglang-xpu:latest /bin/bash
In container ...
sglang serve \
--model-path Intel/gemma-4-12B-it-int4-AutoRound \
--device xpu \
--attention-backend intel_xpu
Structure
TODO
Official SGLang docker image - hub.docker.com
Hardware Platforms XPU - docs.sglang.io
Intel patched SGLang docker image - hub.docker.com
SGLang server arguments - docs.sglang.io