LM/SGLang

Rebuild docker image from source

cd sglang
git pull
cd docker
docker build -t sglang-xpu:latest -f xpu.Dockerfile .

Run

docker run \
    -it \
    --privileged \
    --ipc=host \
    --network=host \
    --user root \
    --group-add $(getent group video | cut -d: -f3) \
    --device /dev/dri \
    -v /dev/dri/by-path:/dev/dri/by-path \
    -v /dev/shm:/dev/shm \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    -p 30000:30000 \
    -e "HF_TOKEN=<secret>" \
    sglang-xpu:latest /bin/bash

Structure

TODO

• https://docs.sglang.io/docs/hardware-platforms/xpu#xpu
• https://hub.docker.com/r/intel/sglang-dev
• https://docs.pytorch.org/docs/2.13/notes/get_start_xpu.html#