LM/Qwen 3.8 27B - Revision history

Tacoball: /* Running */

2026-09-07T10:09:25Z

Running

← Older revision		Revision as of 10:09, 7 September 2026
Line 6:		Line 6:
	=== Running ===		=== Running ===

	The five options are important, especially <code>--chat-template-kwargs</code>.		The five options are important, especially <code>--chat-template-kwargs '{"reasoning_effort":"..."}'</code>.

	The option <code>reasoning_effort</code> can be set to <code>xhigh</code>, <code>medium</code>, and <code>low</code>, but <code>xhigh</code> causes significant performance degradation.		The option <code>reasoning_effort</code> can be set to <code>xhigh</code>, <code>medium</code>, and <code>low</code>, but <code>xhigh</code> causes significant performance degradation.

Tacoball: /* Running */

2026-09-07T10:08:10Z

Running

← Older revision		Revision as of 10:08, 7 September 2026
Line 6:		Line 6:
	=== Running ===		=== Running ===

	The five options are important, especially ~~'''~~--chat-template-kwargs~~'''~~.		The five options are important, especially <code>--chat-template-kwargs</code>.

	~~Option~~ reasoning_effort can be set as xhigh, medium and low, but xhigh ~~would cause bad~~ performance.		The option <code>reasoning_effort</code> can be set to <code>xhigh</code>, <code>medium</code>, and <code>low</code>, but <code>xhigh</code> causes significant performance degradation.

	* --temp 1.0		* <code>--temp 1.0</code>
	* --top-p 0.95		* <code>--top-p 0.95</code>
	* --top-k 20		* <code>--top-k 20</code>
	* --min-p 0.0		* <code>--min-p 0.0</code>
	* --chat-template-kwargs '{"reasoning_effort":"low"}'		* <code>--chat-template-kwargs '{"reasoning_effort":"low"}'</code>

	<syntaxhighlight lang="bash">		<syntaxhighlight lang="bash">

Tacoball: /* Running */

2026-09-07T10:05:06Z

Running

← Older revision		Revision as of 10:05, 7 September 2026
Line 7:		Line 7:

	The five options are important, especially '''--chat-template-kwargs'''.		The five options are important, especially '''--chat-template-kwargs'''.

	Option reasoning_effort can be set as xhigh, medium and low, but xhigh would cause bad performance.		Option reasoning_effort can be set as xhigh, medium and low, but xhigh would cause bad performance.

Tacoball: /* llama.cpp */

2026-09-07T10:04:57Z

llama.cpp

← Older revision		Revision as of 10:04, 7 September 2026
Line 5:		Line 5:

	=== Running ===		=== Running ===

			The five options are important, especially '''--chat-template-kwargs'''.
			Option reasoning_effort can be set as xhigh, medium and low, but xhigh would cause bad performance.

			* --temp 1.0
			* --top-p 0.95
			* --top-k 20
			* --min-p 0.0
			* --chat-template-kwargs '{"reasoning_effort":"low"}'

	<syntaxhighlight lang="bash">		<syntaxhighlight lang="bash">

Tacoball: /* Running */

2026-09-07T10:01:55Z

Running

← Older revision		Revision as of 10:01, 7 September 2026
Line 8:		Line 8:
	<syntaxhighlight lang="bash">		<syntaxhighlight lang="bash">
	# works		# works
	~~./llama.cpp/sycl/bin/~~llama-server \		llama-server -hf unsloth/Qwen3.8-27B-GGUF:UD-Q5_K_XL
	-hf unsloth/Qwen3.8-27B-GGUF:UD-~~Q6_K_M \~~		--spec-type draft-mtp --spec-draft-n-max 2 --spec-draft-p-min 0.7 -ngld 99
	--spec-type draft-mtp --spec-draft-n-max 2 -ngld 99 \		-fa 1 -c 128000 -ctk f16 -ctv f16
	-c ~~64000~~ -ctk ~~q8_0~~ -ctv ~~q8_0 \~~		--temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0
	-~~fa on~~ -~~ngl 99~~ --~~load~~-~~mode mlock -np 1~~ --~~jinja~~ --~~reasoning~~-~~preserve~~ -~~t 8~~		--chat-template-kwargs '{"reasoning_effort":"low"}'
			-b 2048 -ub 1024
	~~# with turboquant~~		-ngl 99 -np 1 -t 6
	~~./llama~~.~~cpp/vulkan/bin/llama-server \~~		--load-mode mlock --jinja --reasoning-preserve
	-~~hf unsloth/Qwen3.8~~-~~27B~~-~~GGUF~~:~~UD-Q6_K_M \~~
	--~~spec-type draft-mtp --spec-draft-n-max 2 -ngld 99 \~~
	-~~c 64000~~ -~~ctk q8_0~~ -~~ctv turbo4 \~~
	~~-fa on -ngl 99~~ --load-mode mlock ~~-np 1~~ --jinja --reasoning-preserve ~~-t 8~~
	</syntaxhighlight>		</syntaxhighlight>

Tacoball: /* Experiment */

2026-09-01T08:42:51Z

Experiment

← Older revision		Revision as of 08:42, 1 September 2026
Line 234:		Line 234:

	tree -h ~/.cache/huggingface/hub/models--unsloth--Qwen3.8-27B-GGUF		tree -h ~/.cache/huggingface/hub/models--unsloth--Qwen3.8-27B-GGUF

			# SergiioB
			hf models ls --tree -R -h SergiioB/Qwen3.8-27B-GPTQ-Int4-sym-G128-MTP-BF16
			hf dpwnload SergiioB/Qwen3.8-27B-GPTQ-Int4-sym-G128-MTP-BF16
			tree -h ~/.cache/huggingface/hub/models--SergiioB--Qwen3.8-27B-GPTQ-Int4-sym-G128-MTP-BF16
	</syntaxhighlight>		</syntaxhighlight>

Tacoball: /* Benchmark for community */

2026-09-01T07:39:16Z

Benchmark for community

@@ Line 75: / Line 75: @@
 build: 9723942ad (10711)
-## unsloth/Qwen3.8-27B-GGUF:UD-Q5_K_XL
+## F16 unsloth/Qwen3.8-27B-GGUF:UD-Q5_K_XL
 | model                          |       size |     params | backend    | ngl |  fa |            test |                  t/s |
@@ Line 86: / Line 86: @@
 build: 9723942ad (10711)
-## unsloth/Qwen3.8-27B-GGUF:UD-Q5_K_M
+## F16 unsloth/Qwen3.8-27B-GGUF:UD-Q5_K_M
 | model                          |       size |     params | backend    | ngl |  fa |            test |                  t/s |
@@ Line 97: / Line 97: @@
 build: 9723942ad (10711)
-## unsloth/Qwen3.8-27B-GGUF:UD-Q5_K_S
+## F16 unsloth/Qwen3.8-27B-GGUF:UD-Q5_K_S
 | model                          |       size |     params | backend    | ngl |  fa |            test |                  t/s |
@@ Line 108: / Line 108: @@
 build: 9723942ad (10711)
-## unsloth/Qwen3.8-27B-GGUF:UD-Q4_K_XL
+## F16 unsloth/Qwen3.8-27B-GGUF:UD-Q4_K_XL
 | model                          |       size |     params | backend    | ngl |  fa |            test |                  t/s |
@@ Line 119: / Line 119: @@
 build: 9723942ad (10711)
-## unsloth/Qwen3.8-27B-GGUF:Q4_0
+## F16 unsloth/Qwen3.8-27B-GGUF:Q4_0
 | model                          |       size |     params | backend    | ngl |  fa |            test |                  t/s |
@@ Line 130: / Line 130: @@
 build: 9723942ad (10711)
-## unsloth/Qwen3.8-27B-GGUF:UD-IQ4_XS
+## F16 unsloth/Qwen3.8-27B-GGUF:UD-IQ4_XS
 | model                          |       size |     params | backend    | ngl |  fa |            test |                  t/s |
@@ Line 138: / Line 138: @@
 | qwen35 27B IQ4_XS - 4.25 bpw   |  13.26 GiB |    27.32 B | SYCL       |  -1 |   1 |           pp512 |        586.82 ± 0.83 |
 | qwen35 27B IQ4_XS - 4.25 bpw   |  13.26 GiB |    27.32 B | SYCL       |  -1 |   1 |           tg128 |         16.34 ± 0.03 |
 </pre>

Tacoball: /* Benchmark for community */

2026-09-01T07:30:33Z

Benchmark for community

← Older revision		Revision as of 07:30, 1 September 2026
Line 64:		Line 64:

	<pre>		<pre>
			## unsloth/Qwen3.8-27B-GGUF:UD-Q6_K

	\| model \| size \| params \| backend \| ngl \| fa \| test \| t/s \|		\| model \| size \| params \| backend \| ngl \| fa \| test \| t/s \|
	\| ------------------------------ \| ---------: \| ---------: \| ---------- \| --: \| --: \| --------------: \| -------------------: \|		\| ------------------------------ \| ---------: \| ---------: \| ---------- \| --: \| --: \| --------------: \| -------------------: \|
Line 72:		Line 74:

	build: 9723942ad (10711)		build: 9723942ad (10711)

			## unsloth/Qwen3.8-27B-GGUF:UD-Q5_K_XL

	\| model \| size \| params \| backend \| ngl \| fa \| test \| t/s \|		\| model \| size \| params \| backend \| ngl \| fa \| test \| t/s \|
Line 81:		Line 85:

	build: 9723942ad (10711)		build: 9723942ad (10711)

			## unsloth/Qwen3.8-27B-GGUF:UD-Q5_K_M

	\| model \| size \| params \| backend \| ngl \| fa \| test \| t/s \|		\| model \| size \| params \| backend \| ngl \| fa \| test \| t/s \|
Line 90:		Line 96:

	build: 9723942ad (10711)		build: 9723942ad (10711)

			## unsloth/Qwen3.8-27B-GGUF:UD-Q5_K_S

	\| model \| size \| params \| backend \| ngl \| fa \| test \| t/s \|		\| model \| size \| params \| backend \| ngl \| fa \| test \| t/s \|
Line 99:		Line 107:

	build: 9723942ad (10711)		build: 9723942ad (10711)

			## unsloth/Qwen3.8-27B-GGUF:UD-Q4_K_XL

	\| model \| size \| params \| backend \| ngl \| fa \| test \| t/s \|		\| model \| size \| params \| backend \| ngl \| fa \| test \| t/s \|
Line 108:		Line 118:

	build: 9723942ad (10711)		build: 9723942ad (10711)

			## unsloth/Qwen3.8-27B-GGUF:Q4_0

	\| model \| size \| params \| backend \| ngl \| fa \| test \| t/s \|		\| model \| size \| params \| backend \| ngl \| fa \| test \| t/s \|
Line 117:		Line 129:

	build: 9723942ad (10711)		build: 9723942ad (10711)

			## unsloth/Qwen3.8-27B-GGUF:UD-IQ4_XS

			\| model \| size \| params \| backend \| ngl \| fa \| test \| t/s \|
			\| ------------------------------ \| ---------: \| ---------: \| ---------- \| --: \| --: \| --------------: \| -------------------: \|
			\| qwen35 27B IQ4_XS - 4.25 bpw \| 13.26 GiB \| 27.32 B \| SYCL \| -1 \| 0 \| pp512 \| 583.02 ± 0.61 \|
			\| qwen35 27B IQ4_XS - 4.25 bpw \| 13.26 GiB \| 27.32 B \| SYCL \| -1 \| 0 \| tg128 \| 16.40 ± 0.03 \|
			\| qwen35 27B IQ4_XS - 4.25 bpw \| 13.26 GiB \| 27.32 B \| SYCL \| -1 \| 1 \| pp512 \| 586.82 ± 0.83 \|
			\| qwen35 27B IQ4_XS - 4.25 bpw \| 13.26 GiB \| 27.32 B \| SYCL \| -1 \| 1 \| tg128 \| 16.34 ± 0.03 \|
	</pre>		</pre>

Tacoball: /* Benchmark for community */

2026-09-01T07:26:49Z

Benchmark for community

@@ Line 62: / Line 62: @@
 ./llama.cpp/sycl-f32/bin/llama-bench -hf unsloth/Qwen3.8-27B-GGUF:UD-IQ4_XS -fa 0,1
 </syntaxhighlight>
 == Experiment ==

Tacoball at 02:50, 1 September 2026

2026-09-01T02:50:03Z

← Older revision		Revision as of 02:50, 1 September 2026
Line 1:		Line 1:
	== llama.cpp ==		== llama.cpp ==

			* https://unsloth.ai/docs/models/qwen3.8
	* https://unsloth.ai/docs/basics/dynamic-3.0-ggufs		* https://unsloth.ai/docs/basics/dynamic-3.0-ggufs