Software factory

The agent-era completion of Infrastructure as Code: the factory itself — orchestration, skills, runners, secrets, MCP access, scoring, benchmarks, self-improvement — defined as code in one config tree, versioned and PR-reviewed, editable by humans and agents alike. Ben Holmes’ framing for Warp Factories: “Terraform for agents” (article captured, 2026-09-30).

The config tree (Warp Factories as the reference)

• factory.yaml — the factory definition: orchestration patterns, model selections per agent, secrets and MCP-server grants per agent, automations/triggers
• runners/ — where each sandbox lives: OS, VM size, Docker image. Per-agent isolation separates secret/MCP access; per-agent runners optimize cost (mobile factory: implementation agent on macOS only when compiling Swift, Foreman + triage on Linux)
• skills/ — factory-wide skills following the same skills convention as other harnesses; can live in external repos referenced from factory.yaml
• webhooks/ — third-party event sources as automation triggers (e.g. Sentry with signature-checked managed secrets)
• scorers/ — LLM-as-judge agents grading finished conversations 0-1 against defined labels: code-quality scorer (diff vs team skills/preferences), efficiency scorer (spinning wheels? burning tool calls learning the environment a skill would have saved?). samplingRate for cost control, passingScore cutoffs
• benchmarks/ — change one variable, hold the rest constant: replay your own team’s agent conversations as tasks (agents converted past conversations into task files via the Factories MCP) — tuned to your setup in a way generic benchmarks can’t be; used to pick models and cut cost

Self-improvement is the loop that pays for it: a scheduled agent reviews the window’s failed scored runs, diagnoses what went wrong, and proposes skill/config changes as pull requests. The factory reviews its own setup daily.

The counter-position: too many things (exe.dev, 2026-10-01)

Dex Horthy signal-boosting the exe.dev team: the software-factory demos at conferences were all task management — kanban boards, Slack/email interfaces, dependency graphs, ticketing. Why? Because agents are slow, and the easy fix is hiding latency behind concurrency (start a new agent every time you get blocked — the sketch.dev seven habits). But concurrency is rough on humans: it trashes flow state and thrashes mental page caches. Task management isn’t a solution; it’s a band-aid. (article captured; dexhorthy tweet, 2026-10-01)

The 50-year-old counter-principle, restated: limit work in progress (XP, Lean, Goldratt). Do fewer things in parallel, do them very well, ship incremental value. As models get faster, natural latency reduction shrinks concurrency needs — but exe.dev isn’t waiting:

• Fast model for human comms — a fast, competent small model (Luna 6) owns the conversation: no coding tools, intentionally short context. Real-time check-ins keep attention engaged; when it exhausts its ready info, the big model takes back over with rich feedback in hand. (Limits: chat is still “a tiny pipe”)
• Screen recording as feedback (Shelley show-and-tell) — camera button: think out loud + screen capture, transcribed with word-level timestamps, correlated to what was on screen. “Agents are extraordinarily good at understanding rambling.” Deeper, richer, longer interactions than chat — especially for UI work
• DOM recording — agent emits a structured HTML artifact (decisions, open questions, examples, diffs) instead of text walls; in-page JS tracks scroll, pointer, highlights, clicks with timestamps, correlated with audio. Human I/O optimized, not agent-shaped
• Feedback livestreaming — send feedback prefixes while still recording; answers to inevitable questions are waiting by the time you finish reading. “Sneaks in an extra round trip without breaking concentration”

Net: not yet one-or-two-agents throughput parity with many, “but my attention is being restored to me, little by little.”

Why this pattern matters

• The IaC ladder extends to agents: infra → CI (Golden repo, Self-hosted CI-CD) → the agent pipeline itself. Same discipline: everything reviewed, everything revertible, experiments as branches
• Agents as first-class editors of their own factory is the genuinely new move — the ACP-worker pattern’s orchestration layer made declarative. Skills conventions converging across harnesses make the skills/ folder portable
• Scorers + traces = the observability layer agent pipelines were missing: grading what actually happened in conversations (wasted tool calls, review-loop thrash, human back-and-forth a subagent could absorb), not just build outcomes
• Benchmarks from your own traces beats public suites for config decisions — and the throughput/attention tradeoff has a real cost side (exe.dev): concurrency is a latency band-aid that taxes the human — the same “fixtures from reality” rule as Golden repo’s no-tautological-tests discipline

Open

• (open: the scorer/judge pattern ports to any agent setup — even without Warp: scheduled review of Hermes session logs against efficiency criteria)
• (open: does factory-config-as-PR-review survive contact with adversarial agents editing their own constraints? — the governance question from Agent Client Protocol)

Related: Infrastructure as Code · Agent knowledge systems (the workplace-as-code counterpart: factory as code for the pipeline, typed knowledge graph for the work) · Agent Client Protocol · Golden repo · Self-hosted CI-CD

Rosen: factories for skeptics — the digestible definition (2026-10-09)

JoshARosen (full article embedded in tweet; captured): the skeptic-friendly walk from “lights-out dark factory” (the version everyone reacts against, citing HumanLayer’s retreat from it — “a system where nobody reads the code… is still a pretty aggressive bet,” “probably too dystopian to be desirable”) to Addy Osmani’s light/dark split — and the load-bearing reframe: “light factories are still factories. Factories of all kinds have always had humans working in them.” The system, not the automation level, is the durable asset — “it might become a competitive advantage in a world where coding agents can spit out code effortlessly.” Six patterns carry the argument: (1) coding agents already contain ephemeral factories (subagent workers spun up per task, decomposition + recipes thrown away after the run — “we keep the resulting code, but surprisingly little of the structure around how the work was produced”); (2) the pane of glass — 100 engineers’ agent sessions are invisible to the org between ticket and PR, and “if you don’t trust your agents (and maybe you shouldn’t), this feels fairly important,” without owning the coding environment (Codex/Claude Code/hand-written all visible through one system); (3) factory work stays iterative — steering moves into shared systems, not away; (4) factories are data factories — mine which work needs human intervention, where agents get stuck, which prompts/models/verification produce what results; (5) factory as code — verification requirements, model routing, tools, context, policies, approval points as versioned, editable, testable repo contents (“a prompt change can be measured against outcomes”); (6) the strategy loop — the self-improving factory (an agent improving the factory from its own data) plus a business-understanding loop feeding context graphs. Closing move: “software factory” is semantics over patterns you likely already partially run — “if you adopt even one of the patterns above, you already have some aspect of a software factory.” For this note: the strongest prose statement yet of the system-is-the-asset thesis this note’s config-tree section argues from Warp’s implementation — and pattern (1) is the vault’s ephemeral-subagent observation, pattern (5) is skills-as-factory-config, pattern (6) is the curator/Hermes-skill arc at org scale.