LM/Ornith 1.5 35B A3B

From Fundamental Ramen
< LM
Revision as of 09:20, 22 October 2026 by Tacoball (talk | contribs) (Created page with "== vLLM == === Best-Fit Models for B70 === {| class="wikitable" style="text-align:center;" ! rowspan="2" | Model ! rowspan="2" | Last-Modified ! rowspan="2" | W ! rowspan="2" | A ! colspan="2" | TP2 / C1 ! colspan="2" | TP2 / C2 ! colspan="2" | TP2 / C4 ! colspan="2" | TP2 / C8 ! colspan="2" | Single |- ! PP 512 || TG 128 || PP 512 || TG 128 || PP 512 || TG 128 || PP 512 || TG 128 || PP 512 || TG 128 |- | style="font-size:10pt; text-align:left;" | TODO | 2026-10-19 ||...")
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

vLLM

Best-Fit Models for B70

Model Last-Modified W A TP2 / C1 TP2 / C2 TP2 / C4 TP2 / C8 Single
PP 512 TG 128 PP 512 TG 128 PP 512 TG 128 PP 512 TG 128 PP 512 TG 128
TODO 2026-10-19 4 16 1737 65 2087 97 2121 139 2664 240

hf operations

# search latest vllm models
hf models ls --apps vllm --sort last_modified --no-truncate --no-gated \
  --expand "downloads,likes,lastModified" \
  --limit 25 --search "Qwen3.8 27B int4"

# list / download / tree
hf models ls --tree -R -h Wrapzii/Swift-1.5-Qwen3.8-27b-GPTQ-Int4-baked-v1-embed-int8
hf download Wrapzii/Swift-1.5-Qwen3.8-27b-GPTQ-Int4-baked-v1-embed-int8
tree -h ~/.cache/huggingface/hub/models--Wrapzii--Swift-1.5-Qwen3.8-27b-GPTQ-Int4-baked-v1-embed-int8
Retrieved from "http://192.168.25.90:53000/index.php?title=LM/Ornith_1.5_35B_A3B&oldid=3989"