LM/Ornith 1.5 35B A3B: Difference between revisions
< LM
Created page with "== vLLM == === Best-Fit Models for B70 === {| class="wikitable" style="text-align:center;" ! rowspan="2" | Model ! rowspan="2" | Last-Modified ! rowspan="2" | W ! rowspan="2" | A ! colspan="2" | TP2 / C1 ! colspan="2" | TP2 / C2 ! colspan="2" | TP2 / C4 ! colspan="2" | TP2 / C8 ! colspan="2" | Single |- ! PP 512 || TG 128 || PP 512 || TG 128 || PP 512 || TG 128 || PP 512 || TG 128 || PP 512 || TG 128 |- | style="font-size:10pt; text-align:left;" | TODO | 2026-10-19 ||..." |
|||
| Line 28: | Line 28: | ||
hf models ls --apps vllm --sort last_modified --no-truncate --no-gated \ | hf models ls --apps vllm --sort last_modified --no-truncate --no-gated \ | ||
--expand "downloads,likes,lastModified" \ | --expand "downloads,likes,lastModified" \ | ||
--limit 25 --search " | --limit 25 --search "Ornith 1.5 35B int4" | ||
hf models ls --apps vllm --sort last_modified --no-truncate --no-gated \ | |||
--expand "downloads,likes,lastModified" \ | |||
--limit 25 --search "Ornith 1.5 35B autoround" | |||
# list / download / tree | # list / download / tree | ||
Latest revision as of 09:23, 22 October 2026
vLLM
Best-Fit Models for B70
| Model | Last-Modified | W | A | TP2 / C1 | TP2 / C2 | TP2 / C4 | TP2 / C8 | Single | |||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| PP 512 | TG 128 | PP 512 | TG 128 | PP 512 | TG 128 | PP 512 | TG 128 | PP 512 | TG 128 | ||||
| TODO | 2026-10-19 | 4 | 16 | 1737 | 65 | 2087 | 97 | 2121 | 139 | 2664 | 240 | ||
hf operations
# search latest vllm models
hf models ls --apps vllm --sort last_modified --no-truncate --no-gated \
--expand "downloads,likes,lastModified" \
--limit 25 --search "Ornith 1.5 35B int4"
hf models ls --apps vllm --sort last_modified --no-truncate --no-gated \
--expand "downloads,likes,lastModified" \
--limit 25 --search "Ornith 1.5 35B autoround"
# list / download / tree
hf models ls --tree -R -h Wrapzii/Swift-1.5-Qwen3.8-27b-GPTQ-Int4-baked-v1-embed-int8
hf download Wrapzii/Swift-1.5-Qwen3.8-27b-GPTQ-Int4-baked-v1-embed-int8
tree -h ~/.cache/huggingface/hub/models--Wrapzii--Swift-1.5-Qwen3.8-27b-GPTQ-Int4-baked-v1-embed-int8