DFlash2

DFlash 2 (Inco AI, August 2026; ICML 2026 paper “DFlash: Block Diffusion for Flash Speculative Decoding” by Chen, Liang, Liu) is a block-diffusion drafter for speculative decoding. Unlike a classic small draft model that predicts tokens autoregressively, the block-diffusion drafter drafts a block of tokens in parallel, and the target model verifies them — acceptance rides on the drafter matching the target’s distribution.

The drafter is tiny because it only needs to match one target: the local analogalok/Qwen3.8-27B-DFlash2-Q2_K.gguf is 673 MB for a 27B target. Community numbers: 2.26x on 100 real coding prompts, up to 4.68x (best case) on Qwen3.8-27B via the llama.cpp PR build (PR #27342); the r/LocalLLaMA 3-day benchmark thread compared it against all speculative methods. vLLM support exists separately; in llama.cpp it runs as --spec-type draft-dflash with a -md draft file.

On the doombox this is the Hermes iris profile default: Qwen38-27B-dflash2-think — Q4_K_M target + Q2_K drafter, --spec-draft-n-max 3, 245k context, q4_0 KV cache, served through a custom dflash2-llama-server build. It beat the MTP-head variants for the default slot.

Contrast with MTP (multi-token prediction): MTP uses the model’s own extra decoding head (shipped in the weights, llama.cpp PR #22673) — no separate drafter download, but limited to the heads the lab trained; DFlash2 needs its paired drafter but drafts in parallel. Both are represented in the llama-swap config (mtp2/3/4/6 variants vs the dflash2 entries), plus ngram lookup variants on top of MTP.

Links: Qwen model family (the target) · llama-swap (OSS) (serving) · Speech to text inference-adjacent speed work

Primer reference (@ratishtwts day-03 writeup, captured → clippings/learn-ratishfolio-com-llms-speculative-decoding.md + clippings/x-2107004322235978087.md): clean diagrams-first explanation of the classic small-drafter/big-verifier form (draft model guesses several tokens; target verifies in ONE forward pass; “same output, fewer sequential steps”), plus the KV-cache distinction — the clarification most primers skip. Useful as the entry-level reference under this note’s block-diffusion and MTP variants.