LM/Devstral Small 2 24B: Difference between revisions
< LM
Jump to navigation
Jump to search
m (Tacoball moved page LM/Devstral Small 2 to LM/Devstral Small 2 24B without leaving a redirect) |
No edit summary |
||
| Line 1: | Line 1: | ||
== Benchmark == | |||
<syntaxhighlight lang="bash"> | |||
./llama.cpp/sycl/bin/llama-bench \ | |||
-hf unsloth/Devstral-Small-2-24B-Instruct-2512-GGUF:Q4_0 \ | |||
-fa on -ctk q8_0 -ctv q8_0 -ngl 99 | |||
</syntaxhighlight> | |||
== Experiment == | == Experiment == | ||
Revision as of 02:00, 27 August 2026
Benchmark
./llama.cpp/sycl/bin/llama-bench \
-hf unsloth/Devstral-Small-2-24B-Instruct-2512-GGUF:Q4_0 \
-fa on -ctk q8_0 -ctv q8_0 -ngl 99
Experiment
hf models ls --search "Devstral" --apps llama.cpp --expand "downloads,likes,lastModified" --sort downloads --no-truncate --limit 10
hf models ls --tree -R -h unsloth/Devstral-Small-2-24B-Instruct-2512-GGUF
hf download unsloth/Devstral-Small-2-24B-Instruct-2512-GGUF --include "*UD-Q4*"
hf download unsloth/Devstral-Small-2-24B-Instruct-2512-GGUF --include "*UD-Q5*"
hf download unsloth/Devstral-Small-2-24B-Instruct-2512-GGUF --include "*UD-Q6*"
hf download unsloth/Devstral-Small-2-24B-Instruct-2512-GGUF --include "*Q8_0*"
hf download unsloth/Devstral-Small-2-24B-Instruct-2512-GGUF --include "mmproj-BF16*"
tree -h ~/.cache/huggingface/hub/models--unsloth--Devstral-Small-2-24B-Instruct-2512-GGUF