Gates that block instead of ask. Roles that can't grade their own work. A commit lint that bans the word "verified" unless a command and its output back it up. We pulled this out of the factory we build our own product with — it's what stuck after an agent told us a change was verified when the check had never run.
Budget controls, run observability, and bounded repair loops — choose how much work your agents can take on.
# Actual behavior of scripts/hooks/test-edit-denial.sh and scripts/selftest/run.sh — not a mock.
Choose a local preview, a single budgeted invocation, or a bounded repair loop. The shared controls support Codex, Claude Code, and OpenCode through their native CLIs. Model execution is opt-in; your existing permissions and factory gates still apply.
Available on main. These additions are not included in the current v0.1.6 release. Use the install or upgrade instructions with --ref main to try them.
Limit task attempts, session runs, execution time, and concurrent factory invocations. Preview the plan locally before enabling a run. Optional estimated-cost thresholds can warn or stop further launches; they are not a guaranteed dollar cap. Choose your budget controls →
See the harness, role, requested model, duration, outcome, reported tokens, and available cost estimates. Missing usage stays unknown. The local, gitignored ledger stores metadata without prompts or responses; export reports as JSON. Native harness history follows its own settings. Explore run reporting →
Manual mode runs your configured check without calling a model. Explicit bounded mode coordinates implementation, checks, separate review, and repair through the same budget. It stops on exhausted limits or stalled progress and leaves evidence and a next action for handoff. Configure a bounded loop →
Use --harness claude or --harness opencode with the same commands. Budget limits cover factory-managed launches, not unrelated interactive sessions or individual model turns inside a harness. Bounded loops require both loop_enabled and budget_enabled; allow for reviewer calls in the budget.
The words everyone's chasing are a ladder. A prompt is a sentence. A loop is a cycle. A harness is the floor the agent stands on. A graph is the shape of the work — nodes that think, edges that carry results. A factory is the floor and the graph. Not the influencer version — the one you can run and enforce.
| Graph concept | What it already is here |
|---|---|
| Node — a bounded job, one in, one out | A role — spec-writer, implementer, reviewer — defined once, generated for every harness |
| Edge — a verified hand-off, not just "and then" | A gate — the hook that checks what crosses, and exits non-zero if it's wrong |
| Verifier — a skeptic before a finding counts | The reviewer — adversarial, and never the same model that wrote the code |
| Tier the models per node | The economy cost profile — cheap model for mechanical roles, frontier for judgment |
You don't become a "graph architect." The graph is the floor: roles are the nodes, gates are the edges, the reviewer is the skeptic, models tier by role. And because the roles are canonical — opencode is the source, Claude Code and Codex are generated — the same graph runs on all three. Why it maps this way →
Across four review rounds of real agent-written code, one pattern held:
consecutive review rounds before the pattern was undeniable
of false "verified" claims were checks that had never been run
checks that actually ran and lied
# INSTRUCTIONS.md - never edit test files - always verify before claiming done compliance: whatever the model remembers today
$ ./scripts/hooks/test-edit-denial.sh DENIED: implementer role cannot edit test files. exit code: 2 compliance: not optional
The underlying idea — computational controls beat inferential ones — comes from Birgitte Böckeler at Thoughtworks. We kept applying it until nothing important was left as prose. Read the full Verification Contract →
A check you've only seen passing proves nothing. Every gate ships with this proof in CI — and the installer runs it on your machine before it will say "done."
To be clear: this is how we build, not how you should. There are other serious takes on agent reliability worth your time. This is just the one we can vouch for, since we live in it daily.
The left column is the common default across today's agent tooling — instruction files, trusted-because-green checks, one session doing everything. It exists for good reasons. The right is what survived our failures.
exit 2 is a fact.WROTE / RAN / OBSERVED — no evidence, no "verified." The commit lint enforces it.Test-edit denial, evidence-checking commit lint, decision gate, drift checks — all configured by one flat factory.yaml. Hooks stay byte-identical across adopters; upgrades are file copies. Hook reference →
Defined once, generated per harness, separated by permissions the hooks enforce — not by politeness. One keeps a cited wiki of your codebase — Karpathy's LLM-wiki pattern, with the provenance enforced rather than hoped for.
The suite violates each rule on purpose and asserts the gate fires. factory-init won't claim success until it passes on your repo.
Score agents on real tasks: a task is a red spec plus an oracle, and the pass rate is measured over N runs — a run only counts if the oracle passes untampered. Each baseline records what it was measured against, so when the oracle or instructions change it reports stale, not a false “no regression.” Eval reference →
A recipe is a graph you can read: nodes are roles, edges are code, findings pass a verifier before they count. workflow-lint enforces that shape — no agent spawned for plumbing an edge could do. One recipe, run natively by all three harnesses. Workflows →
What the gates caught, enforced at zero model tokens, plus one clearly-labeled estimate. It refuses to print “tokens saved” — that compares this run to one that never happened. Cost & models →
Whether the factory is real or installed theatre — gates armed versus inert, what they blocked, how often you route around the same one, how much work got reworked or reverted, and whether your commits cite evidence for what they claim. Most of it is read from git history, so it says something the day you install rather than after months of collecting events.
Every metric names the decision it informs; a number nobody acts on is noise wearing a lab coat. The uncomfortable ones are first-class — inert gates, repeat blocks, rework, stale baselines — for the same reason there is no “tokens saved” headline. And a gate that could block without recording it is reported too, because otherwise a silent gate reads as a quiet one.
Local only. The numbers are computed on your machine, from your repository, for you. Nothing is transmitted — no exporters to configure, no server, no daemon, no account. --json exports a versioned document if you want it somewhere else; --html writes a self-contained page that fetches nothing when you open it. Metrics reference →
| Language pack | Blessed stack | Maturity |
|---|---|---|
| Go | Ginkgo + Gomega · golangci-lint · gosec · govulncheck · gremlins | battle-tested |
| TypeScript / React / Vue | Vitest · Biome (React/Vue-aware) · tsc · Stryker · OSV-Scanner | experimental |
| Java / Spring Boot | JUnit 5 + AssertJ + Testcontainers · Spotless · Error Prone · SpotBugs · OSV-Scanner · PIT | experimental |
battle-tested = a production service shipped under it. experimental = no proven adopter yet. Labels change on real adoption — not before. The core — commit gates, push gates, role separation, the verification contract — is language-agnostic and works with no pack at all. A pack only arms the test-file pattern and check command for your language, so Python, Rust, or Ruby projects get everything except those two.
Frameworks ride on the language pack. React and Vue use the TypeScript pack — Biome auto-applies their rules from your package.json — and Spring Boot uses the Java pack (JUnit 5 + Testcontainers). factory-init detects them. Polyglot? Install several at once: --pack go,typescript.
Each role runs on a cost-appropriate model. Reviewer and spec-writer get a frontier model; refactorer, wiki-maintainer, and background tasks get a cheaper one — routed per harness, so opencode, Claude, and Codex each use their own native model tiers. It's opt-in, and it relaxes no gate: the same hooks check the output whichever model wrote it, which is what lets a cheaper model run safely. Cost & models →
Configuration is one flat file — no nesting, no schema to learn:
Bare, the installer only fetches — a pinned copy into one folder, running nothing it downloaded. Add init and it also sets up the current repo. The word is your consent; the plain command never touches your project.
Needs git, a coding harness (opencode, Claude Code, or Codex), and your language's tools. factory-init runs prereq-check first and tells you what's missing.
installer fetches a pinned copy — and executes none of it
factory-init asks a few questions, writes factory.yaml, installs the gates
init breaks a rule on purpose, watches the gate fire, and only then says "done"
The gates are plain shell scripts, so you can call them from any CI — or from the git pre-push hook, which needs no CI at all. One caveat: the ready-made workflow we ship is GitHub Actions. On GitLab CI, Jenkins, CircleCI, and the like, you invoke the same scripts from your own pipeline config — a handful of lines, but that part is yours to write.
Yes. Every gate is independent. Adopt them one at a time — the adoption guide covers going incremental in a brownfield repo.
From inside the repo: curl -fsSL https://softwareaifactory.sh/install.sh | sh -s -- upgrade — it refreshes the template and applies the update to the repo you're in, exactly like init acts on the current directory (./factory upgrade does the same locally). It refreshes the hooks, scripts, and gates in place, never touches your factory.yaml, code, or customized files, and only reports which of those changed upstream. It upgrades this repo, not every repo on your machine — each owns its committed framework files. Then ./factory doctor confirms every gate still fires. Nothing is committed for you.
The canonical config lives in opencode.json, but a sync script generates equivalent Claude Code and Codex configs. The enforcement layer is plain shell — it doesn't care which harness drives.
Frameworks ride on the language pack — they don't get their own. React and Vue use the TypeScript pack, and Biome auto-applies its react/vue rules when it sees the framework in your package.json. Spring Boot uses the Java pack — its JUnit 5 + Testcontainers stack is Spring Boot's own blessed testing. factory-init detects each and points you at the right pack. A framework-specific rule beyond that is a small custom hook, not a new pack.
The core works anyway. Commit gates, push gates, role separation, and the verification contract are language-agnostic. A pack only arms the test-file pattern and check command — set those two in factory.yaml by hand and you have the full thing.
The gates are the fast local rail — a hook that exits in milliseconds. They run where you'd otherwise wait on a human reviewer to notice the same mistake. That trade goes the other way.
The fastest way to evaluate this template is to try to violate it. That is also the point.