LM/vLLM: Difference between revisions
< LM
Jump to navigation
Jump to search
(→TODO) |
(→TODO) |
||
| Line 44: | Line 44: | ||
* https://sergiiob.dev/posts/intel-arc-b70-vllm-initial-mxfp4-test/ | * https://sergiiob.dev/posts/intel-arc-b70-vllm-initial-mxfp4-test/ | ||
* https://forum.level1techs.com/t/intel-b70-launch-unboxed-and-tested/247873 | * https://forum.level1techs.com/t/intel-b70-launch-unboxed-and-tested/247873 | ||
* https://huggingface.co/blog/MatrixYao/intel-gpu | |||
Revision as of 02:56, 14 August 2026
Experiment
uv venv --python 3.12 --seed --managed-python
source .venv/bin/activate
git clone https://github.com/vllm-project/vllm.git
cd vllm
pip install --upgrade pip
pip install -v -r requirements/xpu.txt
pip uninstall -y triton triton-xpu
pip install triton-xpu==3.7.2 --extra-index-url https://download.pytorch.org/whl/xpu
VLLM_TARGET_DEVICE=xpu pip install --no-build-isolation -e . -v
# check device
python -c "import torch; print(torch.xpu.is_available()); print(torch.xpu.device_count())"
# download model
hf download ciocan/gemma-4-E4B-it-W4A16
# Run
vllm serve unsloth/gemma-4-E4B-it-qat-GGUF --device xpu
Path
TODO
- https://docs.vllm.ai/en/stable/getting_started/installation/gpu/#requirements
- https://www.pugetsystems.com/labs/articles/intel-arc-pro-b70-multi-gpu-ai-inference-performance/
- https://github.com/Hal9000AIML/arc-pro-b70-inference-setup-ubuntu-server
- https://sergiiob.dev/posts/intel-arc-b70-vllm-initial-mxfp4-test/
- https://forum.level1techs.com/t/intel-b70-launch-unboxed-and-tested/247873
- https://huggingface.co/blog/MatrixYao/intel-gpu