How OpenAI, Spotify and Vercel ship a third of their code with agents
1,500 PRs with zero hand-written code. Half of all company PRs automated. One in three merges written by the factory. How they built it, and how your team starts next sprint.
A factory is a system, not a product
A software factory is a small system your team builds around the tools it already has. Agents do the repetitive work. Your engineers make the judgment calls. Every run makes the next one better.
The teams running them are not smarter than yours, and they started smaller than you’d think. Three of them published their receipts.
Five months, empty repo to ~1M lines, zero hand‑written code. Throughput per engineer rose as the team grew.
OpenAI →Automated at Spotify since mid‑2024 by Honk, their background coding agent. A recent Java backend migration took 3 days.
Spotify →Written by the factory on Vercel’s AI SDK repo, and every single one is public and verifiable on GitHub.
Vercel →The term is older than the AI wave. Robert Bemer proposed software factories in 1968, Microsoft wrote the book in 2004, and BMW runs one with 12,000 developers today. The AI version keeps the industrial framing and swaps the assembly line for agents. Two things changed to make it work now: agents got good enough to finish narrow tasks end to end, and the cost of an agent’s mistake fell below the cost of a human waiting.
Why the gains flattened after everyone got an agent
Your engineers already have coding agents. That was the easy part, and it’s also why the gains flattened.
One engineer with an agent works differently from the next engineer with the same agent. Nobody can measure what any of it returns. Everyone reaches for the expensive model by default. Warp’s Zach Lloyd hears this from leaders constantly, and his diagnosis is blunt: interactive agents have human operators, and human variability is the thing eating your ROI.
A factory fixes the leadership problems, not the coding problems. It standardizes how agent work happens, makes it observable, and gives you a number to manage: shipped product per token spent. That’s why this guide is written for you and not for your staff engineers. The architecture is simple. The decisions that make it work are yours: where humans sit, what gets automated first, build or buy.
Warp automates 20–30% of issues fully, end to end. The factory still earns its keep on the rest by handling review, verification, and monitoring while a human writes the code. Build for that split. The automated share grows on its own.
What the factory actually does
One loop, run over and over. An issue comes in. Agents decide whether they can handle it, write the plan, write the code, review it, and prove it works. A human approves the plan and owns the merge.
Two rules make the output trustworthy, and every serious operator follows both.
The agent that writes the code never grades it. Vercel runs a separate review agent that risk‑scores every factory PR. OpenAI made generator‑and‑evaluator separation a team discipline. Same reason your engineers don’t approve their own PRs.
Every run gets classified. Vercel labels each agent run success, flawed, blocked, or manual. Success ships. The other three tell you exactly what to improve, which is how the factory earns more trust over time. Their team’s phrase for the new job: “improving the factory becomes the job.”
What one issue looks like inside a factory
Watch a real one. Issue #17898 on Vercel’s public AI SDK repo:
- A classifier agent reads the issue and routes it as a bug.
- An analysis agent reproduces it and writes a failing script. The bug now has evidence.
- A spec comes out of the analysis.
- An implementation agent writes the fix against the spec.
- A review agent scores the risk.
- A human merges.
- A backport agent opens the release‑line PRs on its own.
One human decision in the whole trace. Every other step produced an artifact your team can audit, and in this case actually can, because the repo is public. Four weeks of this took their backlog from 1,022 open issues to 844.
Notice the shape: one agent per task, small agents with narrow permissions. Easier to trust, easier to debug, and safer. Vercel runs each one in an isolated sandbox so a malicious issue can’t talk a powerful agent into anything.
How your team starts, next sprint
Every working factory started embarrassingly small. Spotify’s began as a migration bot. Vercel’s began as an issue classifier. Yours starts with three pieces, and the best public build guide says exactly that: don’t build the full chain on day one.
A read‑only researcher maps the code before anything is built. A written procedure tells the building agent how work is done here: match existing patterns, write tests alongside code, run the checks. A validator compares the result against the plan, and it cannot fix what it finds. That restriction is deliberate. The checker stays honest because it has nothing to defend.
Under the chain sit four kinds of team knowledge. Your engineers will recognize all four; the leadership move is making sure each lives in the right place:
“The worker handles all scheduled jobs. Do not add cron.”
The build‑with‑tests playbook every building agent follows.
A read‑only researcher that can’t write code.
A pre‑commit hook that blocks secrets, no matter what the model decides.
Resist the urge to write a giant rules document up front. Cursor’s team says rules should emerge: add a guardrail after an agent fails the same way twice, not before. Your rules file becomes a record of every wrong assumption the AI made in your codebase, which is worth more than any template.
Then steal one advanced pattern, because it’s buildable this month: Warp’s self‑improving reviewer. One agent reviews every PR. A second agent runs daily, reads how your engineers reacted to those reviews, and opens a PR updating the reviewer itself. The factory maintains its own staff. There’s a public demo repo your team can clone today.
What to automate first, next, and later
The order matters more than the tooling. Every rung below is ranked by one property: how easy the work is to verify. HumanLayer’s Dex Horthy names it as the property that decides everything: the easier a task is to verify, the safer it is to automate.
The queue where work waits longest in most orgs, and the easiest output to verify: a review either catches real defects or it doesn’t. Both Warp and Augment built their first production line here. Hand your team Warp’s self‑improving review build and the demo repo. Working reviewer in a sprint.
Reading, labeling, reproducing, routing. Verifiable, low blast radius, and it feeds every later rung with clean input. Vercel’s factory started as a classifier before it ever wrote a line of code.
Repetitive, well‑specified, and CI verifies the result. This is Spotify’s entire Honk story: half of all company PRs automated, and a recent Java backend migration done in three days.
An agent gathers logs, metrics, and recent deploys, then posts a root‑cause draft before your on‑call engineer has context. Production‑impacting calls stay with people. Augment’s alert‑to‑resolution loop is one framing of this rung.
The rung everyone wants to start on, and the hardest to verify. Climb to it. By the time your factory has earned trust on rungs one through four, it has the specs, tests, and review discipline that feature work needs.
The pattern across every team we studied: pick the constraint that hurts most, build one loop around it, and let the results reshape how you scope the next one. One rung per quarter is a fast pace.
The Code: Your daily unfair advantage in software engineering.
Join 350,000+ software engineers, tech leads, and CTOs who start their morning with The Code.
Buy, almost certainly
Zach Lloyd, who sells this infrastructure, says so with the honest caveat: building makes sense at Stripe or Uber scale, where the factory has to weld onto sprawling internal systems.
Everyone else hits the CI/CD problem: your team prototypes a factory in a week, then the remaining 80% eats quarters that belonged to customers. Buying still leaves you with the decision that matters, which vendor. Judge every one against four red flags.
Today’s best agent loses to next quarter’s. Lock‑in hands your vendor pricing leverage over your roadmap.
The factory’s memory of your codebase is a lock‑in asset. You own it, full stop.
A factory that can’t run on your infrastructure won’t survive your procurement process.
A vendor whose margin is inference has a conflict of interest on your cost optimization. Bring‑your‑own‑endpoint is table stakes.
Ask how the factory is defined. The right answer is code: version‑controlled files describing agents, rules, and workflows. Factory‑as‑code means you diff a change to your review agent like a change to CI, and switching vendors later doesn’t mean starting over.
The part nobody puts in the pitch deck
The factory amplifies your organization. Both directions.
LinearB measured 2.7 million PRs across 253 orgs. In the strongest organizations, agent‑written PRs merge almost as reliably as human ones. In the weakest, they merge at less than half the human rate. Same models. Same tools. The difference was the foundations the factory landed on. Cortex’s telemetry says the same thing from the incident side: teams shipped more PRs and more failures at once, because AI magnifies the practices you already have.
So the real readiness question is about your foundations, and only you can answer it. Tap what’s true today. One point per yes. Your answers stay in this browser.
Tap the items above to see where your team stands.
And keep one rule no matter what you score, because every operator we studied holds it. HumanLayer ran the fully automated version, published where it broke, and came back with this: “Every PR needs a human owner. This is non‑negotiable.” Accountability never transfers to the machine. Your engineers stand behind what merges. The factory just changes what they spend their judgment on.
What to do Monday
Pick the place where work waits longest in your org. Put a three‑piece loop around it.
Find where work waits longest: review, triage, migrations. That’s the first production line.
Researcher, procedure, validator. Classify every run: success, flawed, blocked, manual.
Add a rule after the factory fails twice the same way. Keep a human on every merge.
Shipped product per token, not PR count. Then add the next agent your team keeps wishing existed.
The teams ahead of you don’t have better models. They have six months of flawed, blocked, and manual runs telling them what to fix. Start collecting yours.
The reading list, in the order to read it
Every source this guide is built on. Forward the first three to whoever runs your first loop.
- 01How to build a software factory with Claude CodeThe most complete hands‑on tutorial anywhere. Seven agents, an orchestrator, three human gates. Tool‑agnostic despite the title.freeCodeCamp · build
- 02Build a self‑improving code reviewThe rung‑one build, step by step: an inner reviewer plus a daily outer loop that updates it from your engineers’ reactions.Warp · build
- 03cloud‑factory‑demoThe working repo behind the review build. Clone it, point it at a real project, and your team has a factory to argue with.GitHub · code
- 04A guide to cloud software factories for engineering leadersThe operating model: components, factory‑as‑code, the efficiency equation, and the build‑vs‑buy reasoning this guide draws on.Warp · operate
- 05Building a software factory for AI SDKThe strongest case study on record, because every PR is public. The #17898 trace and the four‑outcome taxonomy live here.Vercel · proof
- 06Harness engineeringFive months, ~1M lines, zero hand‑written code, and honest open questions at the end. Read for the entropy‑control patterns.OpenAI · proof
- 071,500+ PRs later: Spotify’s background coding agentHonk, part one of four. Self‑critical enough to trust, and the definitive rung‑three story on migrations at fleet scale.Spotify · proof
- 08Building your own software factoryCursor’s internal playbook, summarized. The autonomy ladder and the rules‑emerge‑dynamically principle come from this talk.Gist · operate
- 09Everyone building a software factory wants the same proofHumanLayer, Warp, AWS, and Asana on ownership, ROI, and the LinearB merge‑rate data behind the readiness section above.Dev Interrupted · judge
- 10The engineering leader’s guide to building a software factoryAugment’s vendor take on the same territory: three named loops with per‑role breakdowns. Read the loop taxonomy, weigh the numbers yourself.Augment · vendor
- 11Engineering in the age of AI: 2026 benchmark reportThe amplifier data: PRs up 20%, incidents per PR up 23.5%. The case for fixing foundations before scaling the factory.Cortex · judge