LM/Qwen 3.8 27B
< LM
vLLM
Best-Fit Models for B70
The B70 delivers the best performance with W4A8 quantized models.
The underlying rationale can be referenced from TechPowerUp’s spec data, and the listed models show solid performance.
| Model | Last-Modified | W | A | Single | TP2 / C1 | TP2 / C2 | TP2 / C4 | TP2 / C8 | |||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| PP 512 | TG 128 | PP 512 | TG 128 | PP 512 | TG 128 | PP 512 | TG 128 | PP 512 | TG 128 | ||||
| cyankiwi/Swift-1.5-Qwen3.8-27B-AWQ-BF16-INT4 | 2026-10-19 | 4 | ? | ||||||||||
| cyankiwi/Swift-1.5-Qwen3.8-27B-AWQ-INT4 | 2026-10-19 | 4 | ? | 1908 | 79 | 2062 | 115 | 2196 | 158 | 2632 | 269 | ||
| Launch80/Swift-1.5-Qwen3.8-27B-Uncensored-Int4-baked | 2026-10-13 | 4 | ? | 1899 | 102 | 2084 | 165 | 2595 | 241 | 2758 | 307 | ||
| davetha/Huihui-Qwen3.8-27B-abliterated-AutoRound-Int4-baked-embed-int8 | 2026-10-09 | 4 | 8 | 2026 | 118 | 2411 | 164 | 2801 | 266 | 2785 | 319 | ||
| Wrapzii/Swift-1.5-Qwen3.8-27b-GPTQ-Int4-baked-v1-embed-int8 | 2026-10-01 | 4 | 8 | 2335 | 110 | 2429 | 190 | 2822 | 286 | 2928 | 506 | ||
hf operations
# search latest vllm models
hf models ls --search "Qwen3.8 27B w4a8" --apps vllm --expand "downloads,likes,lastModified" --sort last_modified --no-truncate --no-gated --limit 25
hf models ls --search "Qwen3.8 27B int8 int4" --apps vllm --expand "downloads,likes,lastModified" --sort last_modified --no-truncate --no-gated --limit 25
hf models ls --search "Qwen3.8 27B int4" --apps vllm --expand "downloads,likes,lastModified" --sort last_modified --no-truncate --no-gated --limit 25
hf models ls --search "Qwen3.8 27B autoround" --apps vllm --expand "downloads,likes,lastModified" --sort last_modified --no-truncate --no-gated --limit 25
hf models ls --search "Bonsai 2" --apps vllm --expand "downloads,likes,lastModified" --sort last_modified --no-truncate --no-gated --limit 25
# SergiioB
hf models ls --tree -R -h SergiioB/Qwen3.8-27B-GPTQ-Int4-sym-G128-MTP-BF16
hf download SergiioB/Qwen3.8-27B-GPTQ-Int4-sym-G128-MTP-BF16
tree -h ~/.cache/huggingface/hub/models--SergiioB--Qwen3.8-27B-GPTQ-Int4-sym-G128-MTP-BF16
llama.cpp
Running
The five options are important, especially --chat-template-kwargs '{"reasoning_effort":"..."}'.
The option reasoning_effort can be set to xhigh, medium, and low, but xhigh causes significant performance degradation.
--temp 1.0--top-p 0.95--top-k 20--min-p 0.0--chat-template-kwargs '{"reasoning_effort":"low"}'
# works
llama-server -hf unsloth/Qwen3.8-27B-GGUF:UD-Q5_K_XL
--spec-type draft-mtp --spec-draft-n-max 2 --spec-draft-p-min 0.7 -ngld 99
-fa 1 -c 128000 -ctk f16 -ctv f16
--temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0
--chat-template-kwargs '{"reasoning_effort":"low"}'
-b 2048 -ub 1024
-ngl 99 -np 1 -t 6
--load-mode mlock --jinja --reasoning-preserve
Benchmark for me
# control group
./llama.cpp/sycl/bin/llama-bench -hf unsloth/Qwen3.8-27B-GGUF:UD-Q5_K_XL -fa 1 -ctk q8_0 -ctv q8_0
| model | size | params | backend | ngl | type_k | type_v | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -----: | -----: | --: | --------------: | -------------------: |
| qwen35 27B Q5_K - Medium | 19.43 GiB | 27.32 B | SYCL | 99 | q8_0 | q8_0 | 1 | pp512 | 951.05 ± 7.72 |
| qwen35 27B Q5_K - Medium | 19.43 GiB | 27.32 B | SYCL | 99 | q8_0 | q8_0 | 1 | tg128 | 22.17 ± 0.06 |
# Test -ctv q4_0
./llama.cpp/sycl/bin/llama-bench -hf unsloth/Qwen3.8-27B-GGUF:UD-Q5_K_XL -fa 1 -ctk q8_0 -ctv q4_0
# Test -ub 2048
./llama.cpp/sycl/bin/llama-bench -hf unsloth/Qwen3.8-27B-GGUF:UD-Q5_K_XL -fa 1 -ctk q8_0 -ctv q8_0 -ub 2048
# Test OpenVINO
GGML_OPENVINO_DEVICE=GPU GGML_OPENVINO_STATEFUL_EXECUTION=1 ./llama.cpp/openvino/bin/llama-bench -hf unsloth/Qwen3.8-27B-GGUF:UD-Q5_K_XL -fa 0
Benchmark for community
./llama.cpp/sycl/bin/llama-bench -hf unsloth/Qwen3.8-27B-GGUF:UD-Q6_K -fa 0,1
./llama.cpp/sycl/bin/llama-bench -hf unsloth/Qwen3.8-27B-GGUF:UD-Q5_K_XL -fa 0,1
./llama.cpp/sycl/bin/llama-bench -hf unsloth/Qwen3.8-27B-GGUF:UD-Q5_K_M -fa 0,1
./llama.cpp/sycl/bin/llama-bench -hf unsloth/Qwen3.8-27B-GGUF:UD-Q5_K_S -fa 0,1
./llama.cpp/sycl/bin/llama-bench -hf unsloth/Qwen3.8-27B-GGUF:UD-Q4_K_XL -fa 0,1
./llama.cpp/sycl/bin/llama-bench -hf unsloth/Qwen3.8-27B-GGUF:UD-Q4_K_XL -fa 1 -ctk q8_0 -ctv q8_0
./llama.cpp/sycl/bin/llama-bench -hf unsloth/Qwen3.8-27B-GGUF:Q4_0 -fa 0,1
./llama.cpp/sycl/bin/llama-bench -hf unsloth/Qwen3.8-27B-GGUF:Q4_0 -fa 1 -ctk q8_0 -ctv q8_0
./llama.cpp/sycl/bin/llama-bench -hf unsloth/Qwen3.8-27B-GGUF:UD-IQ4_XS -fa 0,1
./llama.cpp/sycl-nodnn/bin/llama-bench -hf unsloth/Qwen3.8-27B-GGUF:UD-Q5_K_XL -fa 0,1
./llama.cpp/sycl-nodnn/bin/llama-bench -hf unsloth/Qwen3.8-27B-GGUF:Q4_0 -fa 0,1
./llama.cpp/sycl-nodnn/bin/llama-bench -hf unsloth/Qwen3.8-27B-GGUF:UD-IQ4_XS -fa 0,1
./llama.cpp/sycl-f32/bin/llama-bench -hf unsloth/Qwen3.8-27B-GGUF:UD-Q5_K_XL -fa 0,1
./llama.cpp/sycl-f32/bin/llama-bench -hf unsloth/Qwen3.8-27B-GGUF:Q4_0 -fa 0,1
./llama.cpp/sycl-f32/bin/llama-bench -hf unsloth/Qwen3.8-27B-GGUF:UD-IQ4_XS -fa 0,1
## unsloth/Qwen3.8-27B-GGUF:UD-Q6_K | model | size | params | backend | ngl | fa | test | t/s | | ------------------------------ | ---------: | ---------: | ---------- | --: | --: | --------------: | -------------------: | | qwen35 27B Q6_K | 20.46 GiB | 27.32 B | SYCL | -1 | 0 | pp512 | 979.33 ± 4.64 | | qwen35 27B Q6_K | 20.46 GiB | 27.32 B | SYCL | -1 | 0 | tg128 | 21.25 ± 0.05 | | qwen35 27B Q6_K | 20.46 GiB | 27.32 B | SYCL | -1 | 1 | pp512 | 793.28 ± 1.67 | | qwen35 27B Q6_K | 20.46 GiB | 27.32 B | SYCL | -1 | 1 | tg128 | 21.15 ± 0.02 | build: 9723942ad (10711) ## F16 unsloth/Qwen3.8-27B-GGUF:UD-Q5_K_XL | model | size | params | backend | ngl | fa | test | t/s | | ------------------------------ | ---------: | ---------: | ---------- | --: | --: | --------------: | -------------------: | | qwen35 27B Q5_K - Medium | 19.43 GiB | 27.32 B | SYCL | -1 | 0 | pp512 | 970.49 ± 3.90 | | qwen35 27B Q5_K - Medium | 19.43 GiB | 27.32 B | SYCL | -1 | 0 | tg128 | 22.30 ± 0.01 | | qwen35 27B Q5_K - Medium | 19.43 GiB | 27.32 B | SYCL | -1 | 1 | pp512 | 840.86 ± 4.11 | | qwen35 27B Q5_K - Medium | 19.43 GiB | 27.32 B | SYCL | -1 | 1 | tg128 | 22.22 ± 0.01 | build: 9723942ad (10711) ## F16 unsloth/Qwen3.8-27B-GGUF:UD-Q5_K_M | model | size | params | backend | ngl | fa | test | t/s | | ------------------------------ | ---------: | ---------: | ---------- | --: | --: | --------------: | -------------------: | | qwen35 27B Q5_K - Medium | 18.40 GiB | 27.32 B | SYCL | -1 | 0 | pp512 | 858.88 ± 22.25 | | qwen35 27B Q5_K - Medium | 18.40 GiB | 27.32 B | SYCL | -1 | 0 | tg128 | 22.22 ± 0.03 | | qwen35 27B Q5_K - Medium | 18.40 GiB | 27.32 B | SYCL | -1 | 1 | pp512 | 815.38 ± 6.21 | | qwen35 27B Q5_K - Medium | 18.40 GiB | 27.32 B | SYCL | -1 | 1 | tg128 | 22.25 ± 0.05 | build: 9723942ad (10711) ## F16 unsloth/Qwen3.8-27B-GGUF:UD-Q5_K_S | model | size | params | backend | ngl | fa | test | t/s | | ------------------------------ | ---------: | ---------: | ---------- | --: | --: | --------------: | -------------------: | | qwen35 27B Q5_K - Small | 17.37 GiB | 27.32 B | SYCL | -1 | 0 | pp512 | 914.77 ± 2.72 | | qwen35 27B Q5_K - Small | 17.37 GiB | 27.32 B | SYCL | -1 | 0 | tg128 | 22.40 ± 0.02 | | qwen35 27B Q5_K - Small | 17.37 GiB | 27.32 B | SYCL | -1 | 1 | pp512 | 845.78 ± 1.73 | | qwen35 27B Q5_K - Small | 17.37 GiB | 27.32 B | SYCL | -1 | 1 | tg128 | 22.33 ± 0.02 | build: 9723942ad (10711) ## F16 unsloth/Qwen3.8-27B-GGUF:UD-Q4_K_XL | model | size | params | backend | ngl | fa | test | t/s | | ------------------------------ | ---------: | ---------: | ---------- | --: | --: | --------------: | -------------------: | | qwen35 27B Q4_K - Medium | 16.34 GiB | 27.32 B | SYCL | -1 | 0 | pp512 | 811.78 ± 1.40 | | qwen35 27B Q4_K - Medium | 16.34 GiB | 27.32 B | SYCL | -1 | 0 | tg128 | 21.39 ± 0.02 | | qwen35 27B Q4_K - Medium | 16.34 GiB | 27.32 B | SYCL | -1 | 1 | pp512 | 784.04 ± 1.72 | | qwen35 27B Q4_K - Medium | 16.34 GiB | 27.32 B | SYCL | -1 | 1 | tg128 | 21.33 ± 0.02 | build: 9723942ad (10711) ## F16 unsloth/Qwen3.8-27B-GGUF:Q4_0 | model | size | params | backend | ngl | fa | test | t/s | | ------------------------------ | ---------: | ---------: | ---------- | --: | --: | --------------: | -------------------: | | qwen35 27B Q4_0 | 14.94 GiB | 27.32 B | SYCL | -1 | 0 | pp512 | 813.56 ± 1.92 | | qwen35 27B Q4_0 | 14.94 GiB | 27.32 B | SYCL | -1 | 0 | tg128 | 27.48 ± 0.02 | | qwen35 27B Q4_0 | 14.94 GiB | 27.32 B | SYCL | -1 | 1 | pp512 | 355.72 ± 0.43 | | qwen35 27B Q4_0 | 14.94 GiB | 27.32 B | SYCL | -1 | 1 | tg128 | 27.37 ± 0.03 | build: 9723942ad (10711) ## F16 unsloth/Qwen3.8-27B-GGUF:UD-IQ4_XS | model | size | params | backend | ngl | fa | test | t/s | | ------------------------------ | ---------: | ---------: | ---------- | --: | --: | --------------: | -------------------: | | qwen35 27B IQ4_XS - 4.25 bpw | 13.26 GiB | 27.32 B | SYCL | -1 | 0 | pp512 | 583.02 ± 0.61 | | qwen35 27B IQ4_XS - 4.25 bpw | 13.26 GiB | 27.32 B | SYCL | -1 | 0 | tg128 | 16.40 ± 0.03 | | qwen35 27B IQ4_XS - 4.25 bpw | 13.26 GiB | 27.32 B | SYCL | -1 | 1 | pp512 | 586.82 ± 0.83 | | qwen35 27B IQ4_XS - 4.25 bpw | 13.26 GiB | 27.32 B | SYCL | -1 | 1 | tg128 | 16.34 ± 0.03 | build: 9723942ad (10711) ## F16 without DNN unsloth/Qwen3.8-27B-GGUF:UD-Q5_K_XL | model | size | params | backend | ngl | fa | test | t/s | | ------------------------------ | ---------: | ---------: | ---------- | --: | --: | --------------: | -------------------: | | qwen35 27B Q5_K - Medium | 19.43 GiB | 27.32 B | SYCL | -1 | 0 | pp512 | 948.86 ± 4.32 | | qwen35 27B Q5_K - Medium | 19.43 GiB | 27.32 B | SYCL | -1 | 0 | tg128 | 22.31 ± 0.03 | | qwen35 27B Q5_K - Medium | 19.43 GiB | 27.32 B | SYCL | -1 | 1 | pp512 | 791.62 ± 1.74 | | qwen35 27B Q5_K - Medium | 19.43 GiB | 27.32 B | SYCL | -1 | 1 | tg128 | 22.17 ± 0.02 | build: 9723942ad (10711) ## F16 without DNN unsloth/Qwen3.8-27B-GGUF:Q4_0 | model | size | params | backend | ngl | fa | test | t/s | | ------------------------------ | ---------: | ---------: | ---------- | --: | --: | --------------: | -------------------: | | qwen35 27B Q4_0 | 14.94 GiB | 27.32 B | SYCL | -1 | 0 | pp512 | 792.89 ± 2.42 | | qwen35 27B Q4_0 | 14.94 GiB | 27.32 B | SYCL | -1 | 0 | tg128 | 27.53 ± 0.02 | | qwen35 27B Q4_0 | 14.94 GiB | 27.32 B | SYCL | -1 | 1 | pp512 | 353.01 ± 1.91 | | qwen35 27B Q4_0 | 14.94 GiB | 27.32 B | SYCL | -1 | 1 | tg128 | 27.36 ± 0.01 | build: 9723942ad (10711) ## F16 without DNN unsloth/Qwen3.8-27B-GGUF:UD-IQ4_XS | model | size | params | backend | ngl | fa | test | t/s | | ------------------------------ | ---------: | ---------: | ---------- | --: | --: | --------------: | -------------------: | | qwen35 27B IQ4_XS - 4.25 bpw | 13.26 GiB | 27.32 B | SYCL | -1 | 0 | pp512 | 571.95 ± 0.63 | | qwen35 27B IQ4_XS - 4.25 bpw | 13.26 GiB | 27.32 B | SYCL | -1 | 0 | tg128 | 16.43 ± 0.04 | | qwen35 27B IQ4_XS - 4.25 bpw | 13.26 GiB | 27.32 B | SYCL | -1 | 1 | pp512 | 560.78 ± 0.38 | | qwen35 27B IQ4_XS - 4.25 bpw | 13.26 GiB | 27.32 B | SYCL | -1 | 1 | tg128 | 16.40 ± 0.01 | build: 9723942ad (10711) ## F32 unsloth/Qwen3.8-27B-GGUF:UD-Q5_K_XL | model | size | params | backend | ngl | fa | test | t/s | | ------------------------------ | ---------: | ---------: | ---------- | --: | --: | --------------: | -------------------: | | qwen35 27B Q5_K - Medium | 19.43 GiB | 27.32 B | SYCL | -1 | 0 | pp512 | 311.00 ± 0.43 | | qwen35 27B Q5_K - Medium | 19.43 GiB | 27.32 B | SYCL | -1 | 0 | tg128 | 22.26 ± 0.03 | | qwen35 27B Q5_K - Medium | 19.43 GiB | 27.32 B | SYCL | -1 | 1 | pp512 | 304.89 ± 0.13 | | qwen35 27B Q5_K - Medium | 19.43 GiB | 27.32 B | SYCL | -1 | 1 | tg128 | 22.17 ± 0.02 | build: 9723942ad (10711) ## F32 unsloth/Qwen3.8-27B-GGUF:Q4_0 | model | size | params | backend | ngl | fa | test | t/s | | ------------------------------ | ---------: | ---------: | ---------- | --: | --: | --------------: | -------------------: | | qwen35 27B Q4_0 | 14.94 GiB | 27.32 B | SYCL | -1 | 0 | pp512 | 239.50 ± 2.02 | | qwen35 27B Q4_0 | 14.94 GiB | 27.32 B | SYCL | -1 | 0 | tg128 | 27.07 ± 0.04 | | qwen35 27B Q4_0 | 14.94 GiB | 27.32 B | SYCL | -1 | 1 | pp512 | 270.79 ± 0.47 | | qwen35 27B Q4_0 | 14.94 GiB | 27.32 B | SYCL | -1 | 1 | tg128 | 27.05 ± 0.05 | build: 9723942ad (10711) ## F32 unsloth/Qwen3.8-27B-GGUF:UD-IQ4_XS | model | size | params | backend | ngl | fa | test | t/s | | ------------------------------ | ---------: | ---------: | ---------- | --: | --: | --------------: | -------------------: | | qwen35 27B IQ4_XS - 4.25 bpw | 13.26 GiB | 27.32 B | SYCL | -1 | 0 | pp512 | 274.41 ± 0.40 | | qwen35 27B IQ4_XS - 4.25 bpw | 13.26 GiB | 27.32 B | SYCL | -1 | 0 | tg128 | 16.51 ± 0.06 | | qwen35 27B IQ4_XS - 4.25 bpw | 13.26 GiB | 27.32 B | SYCL | -1 | 1 | pp512 | 273.20 ± 0.13 | | qwen35 27B IQ4_XS - 4.25 bpw | 13.26 GiB | 27.32 B | SYCL | -1 | 1 | tg128 | 16.42 ± 0.01 | build: 9723942ad (10711)
Experiment
# search
hf models ls --search "Qwen3.8-27B" --apps llama.cpp --expand "downloads,likes,lastModified" --sort downloads --no-truncate --limit 10
# unsloth
hf models ls --tree -R -h unsloth/Qwen3.8-27B-GGUF
hf download unsloth/Qwen3.8-27B-GGUF --include "*UD-IQ4_XS*"
hf download unsloth/Qwen3.8-27B-GGUF --include "*UD-Q4_K_XL*"
hf download unsloth/Qwen3.8-27B-GGUF --include "*UD-Q5_K_XL*"
hf download unsloth/Qwen3.8-27B-GGUF --include "*UD-Q6_K.gguf"
hf download unsloth/Qwen3.8-27B-GGUF --include "*UD-Q6_K_M*"
hf download unsloth/Qwen3.8-27B-GGUF --include "*UD-Q6_K_L*"
hf download unsloth/Qwen3.8-27B-GGUF --include "*UD-Q6_K_XL*"
hf download unsloth/Qwen3.8-27B-GGUF --include "*UD-Q8_K_L*"
hf download unsloth/Qwen3.8-27B-GGUF --include "*Q4_0*"
hf download unsloth/Qwen3.8-27B-GGUF --include "*Q8_0*"
hf download unsloth/Qwen3.8-27B-GGUF --include "mmproj-BF16*"
hf download unsloth/Qwen3.8-27B-GGUF --include "*mtp-Qwen3.8-27B-Q4_0.gguf"
find ~/.cache/huggingface/hub -name "*Qwen3.8-27B-UD-Q8_K_L.gguf"
find ~/.cache/huggingface/hub -name "*Qwen3.8-27B-UD-Q8_K_L.gguf" -exec sh -c 'rm -f "$1" "$(readlink -f "$1")"' _ {} \;
tree -h ~/.cache/huggingface/hub/models--unsloth--Qwen3.8-27B-GGUF
