LM/Inference Engine Comparison: Difference between revisions
< LM
Created page with "{| class="wikitable" ! || Model format || TP || ROCM || XPU |- | vLLM || || || || |- | llama.cpp || || || || |- | SGLang || || || || |- | Strata || || || || |- | Ninfer || || || || |}" |
No edit summary |
||
| Line 2: | Line 2: | ||
! || Model format || TP || ROCM || XPU | ! || Model format || TP || ROCM || XPU | ||
|- | |- | ||
| vLLM || || || || | | vLLM || Hugging Face Transformers / Safetensors || ✅ || ✅ || ✅ | ||
|- | |- | ||
| llama.cpp || || || || | | llama.cpp || GGUF || ✅ || ✅ || ✅ | ||
|- | |- | ||
| SGLang || || || || | | SGLang || Hugging Face Transformers / Safetensors || ✅ || ✅ || ✅ | ||
|- | |- | ||
| Strata || || || || | | Strata || Project-specific format / GGUF || ✅ || ✅ || ❌ | ||
|- | |- | ||
| Ninfer || || || || | | Ninfer || Project-specific model artifacts || ✅ || ✅ || ❌ | ||
|} | |} | ||
Latest revision as of 09:01, 7 October 2026
| Model format | TP | ROCM | XPU | |
|---|---|---|---|---|
| vLLM | Hugging Face Transformers / Safetensors | ✅ | ✅ | ✅ |
| llama.cpp | GGUF | ✅ | ✅ | ✅ |
| SGLang | Hugging Face Transformers / Safetensors | ✅ | ✅ | ✅ |
| Strata | Project-specific format / GGUF | ✅ | ✅ | ❌ |
| Ninfer | Project-specific model artifacts | ✅ | ✅ | ❌ |