LM/Inference Engine Comparison

From Fundamental Ramen
< LM
Jump to navigation Jump to search
Model format TP ROCM XPU
vLLM Hugging Face Transformers / Safetensors ✅ ✅ ✅
llama.cpp GGUF ✅ ✅ ✅
SGLang Hugging Face Transformers / Safetensors ✅ ✅ ✅
Strata Project-specific format / GGUF ✅ ✅ ❌
Ninfer Project-specific model artifacts ✅ ✅ ❌