LLM rigs
The hardware lane of Local LLM inference: complete builds — parts, prices, measured performance — for serving models locally. Entry bar: a rig is a system (GPU + host RAM + CPU/platform + storage), not a single card; and captured numbers must be measured, not vendor claims.
The ~$7k single-Arc rig: one B70, 262K context, 33–41 tok/s (2026-10-19)
0xSero / Sybil AI (122❤/9.1k views; tweet + benchmark image vision-transcribed, “MEASURED” badge): the no-CUDA lane gets its reference build — one Intel Arc Pro B70 (the XeStrata (OSS) GPU) carrying Qwen3.8-Flash-Next at 262K context with vision + MTP, needle found at 233K+.
Parts list (his prices):
- 1× Intel Arc Pro B70 — $1,300–1,600
- 256 GB DDR4 — $2,500
- Epyc 7443p (24c) — $1,100
- Romed8-2t (ASRock Rack ROMED8-2T) — $850
- 4 TB NVMe Gen4 — $1,000
- **Total ~3,000 as demonstrated in the rows”
The measured table (decode tok/s, one user, 262K ctx; dashed floor = 25 tok/s; pf = prefill @32K):
| Config | decode | prefill |
|---|---|---|
| Flash-Next, 63 GB host RAM | 41.0 | 1,145 pf |
| Flash-Next, 54 GB RAM + NVMe | 37.8 | 1,055 pf |
| Flash-Next, 25 GB RAM + NVMe | 32.9 | 1,077 pf |
| Flash-Next, 12 GB RAM + NVMe | 33.2 | 914 pf |
| Nemotron 3.5 Bolt 74B (SYCL) | 26.2 | 728 pf |
| GLM-5.3-Flash, 260 GB RAM | 17.1 ⚠ | 147 pf |
The load-bearing finding: offload to NVMe barely costs decode speed — 12 GB host RAM runs within ~20% of a full-RAM fit (33.2 vs 41.0 tok/s). That’s the economics of the whole rig: the 4.5k. The GLM-5.3-Flash row is the honest-control row — below the 25 tok/s floor even at 260 GB RAM (“fix in, recipe pending”), i.e. per-model recipe quality matters as much as the metal. Doombox comparison line: a single 3090 (24 GB, used ~$700) runs the same 27B-class models at similar context only with aggressive quant + no vision lane; this rig’s argument is headroom — 262K ctx with vision + MTP as a supported configuration, on new parts with a warranty. Kin: Strata-DualGPU (OSS) (143 tok/s dual-GPU, the speed pole), the DGX-Spark effort lore in Local LLM inference (frontier parity at 5–8× slower), and the sasasin field reports there (B70 + OpenCode daily driver, agent-tuned prefill).
Roster format (for future entries)
- Build: GPU / host RAM / CPU+board / storage — total price
- Measured: model × context, decode/prefill, the floor it clears
- The trick: what makes it work (offload recipe, speculative drafter, quant floor)
- Honest control: what doesn’t work on the same metal