LM/Ornith 1.5 35B A3B: Difference between revisions
< LM
| Line 21: | Line 21: | ||
| || | | || | ||
|} | |} | ||
=== TODO === | |||
* cyankiwi/Ornith-1.5-35B-A3B-AWQ-INT4 | |||
* wiesion/Ornith-1.5-35B-A3B-int4-AutoRound-GPTQ-XPU | |||
=== hf operations === | === hf operations === | ||
Latest revision as of 06:38, 28 October 2026
vLLM
Best-Fit Models for B70
| Model | Last-Modified | W | A | TP2 / C1 | TP2 / C2 | TP2 / C4 | TP2 / C8 | Single | |||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| PP 512 | TG 128 | PP 512 | TG 128 | PP 512 | TG 128 | PP 512 | TG 128 | PP 512 | TG 128 | ||||
| TODO | 2026-10-19 | 4 | 16 | 1737 | 65 | 2087 | 97 | 2121 | 139 | 2664 | 240 | ||
TODO
- cyankiwi/Ornith-1.5-35B-A3B-AWQ-INT4
- wiesion/Ornith-1.5-35B-A3B-int4-AutoRound-GPTQ-XPU
hf operations
# search latest vllm models
hf models ls --apps vllm --sort last_modified --no-truncate --no-gated \
--expand "downloads,likes,lastModified" \
--limit 25 --search "Ornith 1.5 35B int4"
hf models ls --apps vllm --sort last_modified --no-truncate --no-gated \
--expand "downloads,likes,lastModified" \
--limit 25 --search "Ornith 1.5 35B autoround"
# list / download / tree
hf models ls --tree -R -h Wrapzii/Swift-1.5-Qwen3.8-27b-GPTQ-Int4-baked-v1-embed-int8
hf download Wrapzii/Swift-1.5-Qwen3.8-27b-GPTQ-Int4-baked-v1-embed-int8
tree -h ~/.cache/huggingface/hub/models--Wrapzii--Swift-1.5-Qwen3.8-27b-GPTQ-Int4-baked-v1-embed-int8