LM/Qwen 3.8 27B

vLLM

Best-Fit Models for B70

The B70 delivers the best performance with W4A8 quantized models.

The underlying rationale can be referenced from TechPowerUp’s spec data, and the listed models show solid performance.

Model	Last-Modified	W	A	TP2 / C1		TP2 / C2		TP2 / C4		TP2 / C8		Single
				PP 512	TG 128	PP 512	TG 128	PP 512	TG 128	PP 512	TG 128	PP 512	TG 128
cyankiwi/Swift-1.5-Qwen3.8-27B-AWQ-BF16-INT4	2026-10-19	4	16	1737	65	2087	97	2121	139	2664	240
cyankiwi/Swift-1.5-Qwen3.8-27B-AWQ-INT4	2026-10-19	4	16	1908	79	2062	115	2196	158	2632	269
Launch80/Swift-1.5-Qwen3.8-27B-Uncensored-Int4-baked	2026-10-13	4	16	1899	102	2084	165	2595	241	2758	307
davetha/Huihui-Qwen3.8-27B-abliterated-AutoRound-Int4-baked-embed-int8	2026-10-09	4	16	2026	118	2411	164	2801	266	2785	319
🔥 Wrapzii/Swift-1.5-Qwen3.8-27b-GPTQ-Int4-baked-v1-embed-int8 🔥	2026-10-01	4	16	2335	110	2429	190	2822	286	2928	506

hf operations

# search latest vllm models
hf models ls --apps vllm --expand "downloads,likes,lastModified" --sort last_modified --no-truncate --no-gated --limit 25 --search "Qwen3.8 27B int4"
hf models ls --apps vllm --expand "downloads,likes,lastModified" --sort last_modified --no-truncate --no-gated --limit 25 --search "Qwen3.8 27B int8 int4"
hf models ls --apps vllm --expand "downloads,likes,lastModified" --sort last_modified --no-truncate --no-gated --limit 25 --search "Qwen3.8 27B autoround"
hf models ls --apps vllm --expand "downloads,likes,lastModified" --sort last_modified --no-truncate --no-gated --limit 25 --search "Qwen3.8 27B w4a8" 

# Wrapzii/Swift-1.5-Qwen3.8-27b-GPTQ-Int4-baked-v1-embed-int8 
hf models ls --tree -R -h Wrapzii/Swift-1.5-Qwen3.8-27b-GPTQ-Int4-baked-v1-embed-int8
hf download Wrapzii/Swift-1.5-Qwen3.8-27b-GPTQ-Int4-baked-v1-embed-int8
tree -h ~/.cache/huggingface/hub/models--Wrapzii--Swift-1.5-Qwen3.8-27b-GPTQ-Int4-baked-v1-embed-int8

llama.cpp

• https://unsloth.ai/docs/models/qwen3.8
• https://unsloth.ai/docs/basics/dynamic-3.0-ggufs

Running

The five options are important, especially --chat-template-kwargs '{"reasoning_effort":"..."}'.

The option reasoning_effort can be set to xhigh, medium, and low, but xhigh causes significant performance degradation.

• --temp 1.0
• --top-p 0.95
• --top-k 20
• --min-p 0.0
• --chat-template-kwargs '{"reasoning_effort":"low"}'

# works
llama-server -hf unsloth/Qwen3.8-27B-GGUF:UD-Q5_K_XL
  --spec-type draft-mtp --spec-draft-n-max 2 --spec-draft-p-min 0.7 -ngld 99
  -fa 1 -c 128000 -ctk f16 -ctv f16
  --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0
  --chat-template-kwargs '{"reasoning_effort":"low"}'
  -b 2048 -ub 1024
  -ngl 99 -np 1 -t 6
  --load-mode mlock --jinja --reasoning-preserve

Benchmark for me

# control group
./llama.cpp/sycl/bin/llama-bench -hf unsloth/Qwen3.8-27B-GGUF:UD-Q5_K_XL -fa 1 -ctk q8_0 -ctv q8_0

| model                          |       size |     params | backend    | ngl | type_k | type_v |  fa |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -----: | -----: | --: | --------------: | -------------------: |
| qwen35 27B Q5_K - Medium       |  19.43 GiB |    27.32 B | SYCL       |  99 |   q8_0 |   q8_0 |   1 |           pp512 |        951.05 ± 7.72 |
| qwen35 27B Q5_K - Medium       |  19.43 GiB |    27.32 B | SYCL       |  99 |   q8_0 |   q8_0 |   1 |           tg128 |         22.17 ± 0.06 |

# Test -ctv q4_0
./llama.cpp/sycl/bin/llama-bench -hf unsloth/Qwen3.8-27B-GGUF:UD-Q5_K_XL -fa 1 -ctk q8_0 -ctv q4_0

# Test -ub 2048
./llama.cpp/sycl/bin/llama-bench -hf unsloth/Qwen3.8-27B-GGUF:UD-Q5_K_XL -fa 1 -ctk q8_0 -ctv q8_0 -ub 2048

# Test OpenVINO
GGML_OPENVINO_DEVICE=GPU GGML_OPENVINO_STATEFUL_EXECUTION=1 ./llama.cpp/openvino/bin/llama-bench -hf unsloth/Qwen3.8-27B-GGUF:UD-Q5_K_XL -fa 0

Experiment

# search
hf models ls --search "Qwen3.8-27B" --apps llama.cpp --expand "downloads,likes,lastModified" --sort downloads --no-truncate --limit 10

# unsloth
hf models ls --tree -R -h unsloth/Qwen3.8-27B-GGUF

hf download unsloth/Qwen3.8-27B-GGUF --include "*UD-IQ4_XS*"
hf download unsloth/Qwen3.8-27B-GGUF --include "*UD-Q4_K_XL*"
hf download unsloth/Qwen3.8-27B-GGUF --include "*UD-Q5_K_XL*"
hf download unsloth/Qwen3.8-27B-GGUF --include "*UD-Q6_K.gguf"
hf download unsloth/Qwen3.8-27B-GGUF --include "*UD-Q6_K_M*"
hf download unsloth/Qwen3.8-27B-GGUF --include "*UD-Q6_K_L*"
hf download unsloth/Qwen3.8-27B-GGUF --include "*UD-Q6_K_XL*"
hf download unsloth/Qwen3.8-27B-GGUF --include "*UD-Q8_K_L*"
hf download unsloth/Qwen3.8-27B-GGUF --include "*Q4_0*"
hf download unsloth/Qwen3.8-27B-GGUF --include "*Q8_0*"
hf download unsloth/Qwen3.8-27B-GGUF --include "mmproj-BF16*"
hf download unsloth/Qwen3.8-27B-GGUF --include "*mtp-Qwen3.8-27B-Q4_0.gguf"

find ~/.cache/huggingface/hub -name "*Qwen3.8-27B-UD-Q8_K_L.gguf"
find ~/.cache/huggingface/hub -name "*Qwen3.8-27B-UD-Q8_K_L.gguf" -exec sh -c 'rm -f "$1" "$(readlink -f "$1")"' _ {} \;

tree -h ~/.cache/huggingface/hub/models--unsloth--Qwen3.8-27B-GGUF