Qwen model family
Qwen is Alibaba’s open-weight LLM family (Qwen team / Tongyi Lab) — the lineage most local stacks converge on, and the base for most finetunes on the doombox. The releases in active rotation locally:
- Qwen3.8-27B — dense 27B, hybrid attention (Gated DeltaNet linear + full attention), native vision-language (mmproj adapter), flexible thinking control, tool calling, MTP speculative-decoding head. The current workhorse: 14 of 59 llama-swap entries point at
unsloth/Qwen3.8-27B-UD-Q4_K_M.gguf. - Qwen3.6-27B — dense predecessor; MTP-capable builds (
unsloth/Qwen3.6-27B-MTP-GGUF, Q4_K_M). - Qwen3.6-35B-A3B — sparse MoE sibling: 35B total, ~3B active per token — speed follows the 3B, memory follows the 35B. Served as
Qwen3.6-35B-A3B-UD-Q3_K_XL. - Family traits worth noting: bilingual EN/ZH, Apache-2.0 (or MIT for some finetunes), GGUF + MLX ecosystems, reasoning traces returned separately from the answer.
The family is what everything else on the box is built on: GestaltLabs’ post-training pipeline (Gestalt Labs), OrcaRouter’s abliterated build, the DFlash2 draft model — all Qwen3.x-27B at the core.
Local deployment shape (llama-swap (OSS) on the 3090, single GPU, one model hot-swapped at a time): every model registered in think-off and think-on (-think) variants, MTP speculative variants at draft depth 2/3/4/6, and the DFlash2 pairing as the Hermes iris profile default: Qwen38-27B-dflash2-think — Q4_K_M target + Q2_K block-diffusion drafter (--spec-type draft-dflash --spec-draft-n-max 3, 245k context, q4_0 KV cache). The naming map: Qwen36→Qwen3.6, Qwen38→Qwen3.8, Orca38→orcarouter uncensored, Ornstein36→GestaltLabs Ornstein, Ornith15→AtomicChat Ornith 1.5. DFlash pairing available (announcement, 2026-10-01): ornith-ai published DFlash draft checkpoints for Ornith-1.5 9B / 35B-A3B / 397B (ornith-ai/Ornith-1.5-35B-A3B-DFlash, 386M-param BF16 block-diffusion drafter, ~0.77 GB, MIT) — up to 2.54× speedup, lossless per their eval; the same --spec-type draft-dflash pattern as the iris Qwen38-27B default, now first-party for the Ornith llama-swap entries.
Adjacent: ThinkingCap-Qwen3.8-27B (37% fewer reasoning tokens), Speech to text inference-side Qwen3-Embedding is the qmd fallback embedder candidate per the Local LM roadmap. The post-train ecosystem around the 27B dense lives in Qwen3.8 finetunes (Ornstein, OrcaRouter, Swift 1.5), and the “midpoint of two generations” question gets its own answer in Model blends (JetBrains’ 50/50 3.6×3.8 interpolation); the Flash-Next big-MoE branch has its own section below.
Finetunes and post-trains in the local rotation
- Unsloth — quantizers/finetuners; every unsloth/ GGUF on the box.
- Gestalt labs — Ornstein3.6-27B-MTP-NSC-ACE-SABER: SFT → NSC-ACE (GRPO + latent steering) → SABER.
- OrcaRouter — Qwen3.8-27B-Uncensored, abliterated.
- ThinkingCap (BottleCap AI) — captured model card in clippings; not on the box.
- Holo4-27B (H Company, computer-use post-train; GGUF card captured) — Qwen3.8-27B base turned into a computer-use VLM: screenshots + tool results in, clicks/typing/code/tool calls out via the hai-agents harness; 262k context; web/desktop/mobile targets. Official 16.9 GB Q4_K_M GGUF fits the 3090. The @TeksEdge head-to-head: same base vs post-train on a Godot Pac-Man build — base 197 calls/11.4M tokens, Holo4 68 calls/2.4M tokens (~65% fewer calls, ~79% fewer tokens). OSWorld 85.2% at 1.22. Family also spans Holo4-35B-A3B (Qwen3.6 MoE base) and Holotron4-30B-A3B (NemotronH Nano Omni). Open agent-trajectory dataset published. Not on the box yet — candidate for the computer-use slot.
Qwen3.8-Flash-Next
The family’s big-MoE branch and an early preview of the Qwen4 architecture (same role Qwen3-Next played for Qwen3.5): 125B total / 6B activated, plus 51B n-gram embeddings and a 4B MTP head. Architecture: 48 layers of 12 × (3 × (Gated DeltaNet → MoE) → 1 × (Qwen Sparse Attention → MoE)), 512 experts, 10 routed + 1 shared per token, 262k context natively extensible to 1M. The n-gram embedding idea is the novel scaling axis — parameter capacity that’s cheap to compute and offloadable to SSD (read a few rows per token), which is exactly what makes it consumable-PC-runnable. Benchmarks: beats the 27B dense on agentic coding (DeepSWE 58.7 vs 42.2), SWE-bench Multilingual 81.0, JobBench 55.7 vs 33.4.
- Strata (OSS) runs it on one 12-24 GB card + 64 GB RAM at 60-95 tok/s (GPU: dense + hot experts; RAM: all 512 experts; SSD: the n-gram table; MTP speculation 1.6-1.8x). The demo of Local LLM inference’s offload frontier.
- Swift 1.5 (UkisAI fine-tune, GSQ-RCO quants) — thinks much shorter before answering, same speed per token and quality; bundled by Strata.
- Local-fit note: 125B/6B-active is a different beast from the 27B dense the box runs — offload-tier serving, not llama-swap single-GGUF hot-swap.