Model blends

Checkpoint interpolation — averaging the weights of N same-architecture models into one: blended = a·A + (1−a)·B per tensor. Cheap (no training, one pass over weights), and surprisingly effective when the sources share a base lineage: interpolation stays inside the family’s loss basin, so capabilities combine rather than collide. Known variously as model merging, checkpoint averaging, or model soups.

The mechanics that make it legitimate rather than alchemy:

• Same-architecture requirement is hard (tensor shapes must match); same lineage is what makes the result coherent — two finetunes of one base average to something that behaves like a mid-point finetune
• Reproducibility discipline: serious releases pin source revisions and record merge settings — a merge-manifest.json in the blend below; the analog of lockfiles for weights
• The result is a new single model: full inference speed, no routing overhead, no memory doubling — unlike MoE-routing or ensemble approaches

The reference: JetBrains Qwen3.8-3.6-27B-blend

(Model card captured; the Junie Local release story behind it.) JetBrains’ 50/50 linear interpolation of Qwen3.6-27B and Qwen3.8-27B — config/tokenizer/chat template from 3.8, exact source revisions + merge settings in merge-manifest.json. Released as the engine of Junie Local: on their 100-task internal coding benchmark it sits at a better quality-per-token point than either parent — the blend inherited 3.8’s coding quality with 3.6’s token efficiency (their tradeoff chart shows both parents dominated).

Packaging details worth stealing: GGUFs with the MTP head embedded in the main file (no separate drafter — --spec-type draft-mtp --spec-draft-n-max 3, tested llama.cpp revision pinned), a shared BF16 vision encoder GGUF for image input, quants straight from one BF16 intermediate (no imatrix), and the exact llama-server invocation with source sampling settings documented.

Where it sits in the local stack

• Drop-in for llama-swap (OSS): one GGUF, standard llama.cpp — the Qwen38 slot can host it directly (Q4_K_M 16.81 GB fits the 3090; same native-MTP runtime as the DFlash2 setup)
• Blend vs Qwen3.8 finetunes: finetunes train a behavior change; blends interpolate existing checkpoints — orthogonal axes, composable in principle (blend two finetunes)
• Blend vs Qwen model family generations: this is the answer to “3.6 or 3.8?” — sometimes the midpoint is the better model, and it costs nothing to test
• The reproducibility pattern (merge manifests, pinned revisions, documented sampling settings) is the Golden repo discipline applied to weights

Open questions: does 50/50 generalize, or does the coefficient need task-tuning (the soups literature says uniform often wins, but per-task search helps)? Do reduced-thinking finetunes (Qwen3.8 finetunes) blend cleanly with non-reduced parents?

Related: Qwen model family · Qwen3.8 finetunes · llama-swap (OSS) · Local LLM inference · Unsloth