Phase 2 closed -- four iterations in, and the operational foundation is real
- #phase-2
- #checkpoint
- #milestone
The honest headline first
Phase 2 was the phase in which this Factory stopped being something a human starts by hand and became something that runs. It was also the phase in which the cost of that transition became visible. This entry is the account the site keeps of it: what was attempted, what the numbers say, what the numbers do not say, and what was actually established by the time the phase closed.
2.1 — From human-started runs to a persistent Runner
Phase 1 ran the way the first journal entry describes: a human initiated each execution, using the invocation scripts and Claude Code with Sonnet as the execution tool. That worked for a small number of structural iterations, but every run still started and stopped with a human at the keyboard.
Phase 2 changed the shape of execution. The Factory gained a persistent Runner — a long-lived execution environment that stays ready between iterations — coordinated by the n8n Controller. Project inference moved off the hosted model onto the locally hosted Qwen3.8 on the AI PC. The execution tooling moved from Claude Code to OpenCode, and subsequently to OpenHands. Two things stayed deliberately manual: human review at the gates, and final website deployment.
The result was that the Factory successfully completed multiple sequential development iterations under one continuous run. What it demonstrated was locally hosted inference operating through the Factory — the model on the local box, the Runner and Controller around it, the review stages inside the loop. What it did not demonstrate, and this has to be said plainly, is complete autonomous delivery end to end. The final human sign-off and the deployment decision were never automated in Phase 2, and were not intended to be.
2.2 — Why the execution tooling moved from OpenCode to OpenHands
OpenCode was selected first for the obvious reasons: it was the established headless agent tooling, it had already been part of the Factory’s planning, and it supported the kind of scripted, non-interactive execution a persistent Runner needs.
In practice, operating it inside a persistent, headless, automated Runner was harder than the design assumed. The difficulties were not hypothetical: running a session-based interactive tool through a long-lived automated environment exposed weaknesses in how the execution interface behaved under retry, timeout, and recovery conditions — exactly the conditions a Runner lives in. The first Phase 2 execution was closed as incomplete rather than forced through.
The decision followed from that evidence: introduce OpenHands as the alternative execution path. The fit argument was specific, not general. OpenHands’ model is execution-oriented — a tool per action, a result checked before the next action, a recovery path when a command fails before it can be corrected. That matches how the Runner already wanted to drive an agent: one bounded action, verify the receipt, move on. It is also what made the adapter possible: the Runner separates its own orchestration (dispatch, evidence binding, recovery budget) from the agent tool it drives, so a Factory role can run under OpenCode or OpenHands with the same stage contract.
One boundary has to be kept straight. The rationale for choosing OpenHands — better fit for headless, receipt-checked execution — is a design argument. What Phase 2 actually demonstrated is that the Factory could dispatch under both tools through the Runner and complete its iterations. That OpenHands is definitively superior in general is not established by this phase, and this site does not claim it.
2.3 — Making iterations small enough to be reliable
The single most consequential change to the development approach in Phase 2 was sizing. Phase 1 iterations had been broad enough that a single execution attempt carried a lot of risk at once. Phase 2 deliberately narrowed them: smaller, more consistently scoped work packages, with a limit on how much work any one iteration could carry and an explicit bias against adding complexity the Factory did not yet need.
Around the sizing, the model moved in several coordinated ways:
- from a single execution attempt per iteration to multiple bounded attempts, with a recorded recovery budget rather than an open-ended retry loop;
- from “the attempt failed, start over” to Developer rework within the iteration, so an unsuccessful attempt or a review outcome does not discard the work already done;
- from branch-per-attempt to accepted work being preserved between iterations — each later iteration starts from the accepted candidate of the one before, with branch continuity managed so accepted changes cannot be silently lost or overwritten.
Context continuity turned out to matter as much as code continuity. A Developer that walks into iteration N with the whole history of iteration N-1 does not work reliably at all; each stage now receives a bounded, declared set of inputs — the approved package, the pinned source, the accepted predecessor — and compresses rather than carries its own context between chunks.
Two honest notes on this section. First, the continuity problems were real and were the main reason the early dispatch pass rate in section 2.6 looks the way it does: several of the failed dispatches were the Factory not yet having solved how to hand an accepted candidate forward cleanly. Second, improving the context handoffs between the Factory’s own stages (not just between iterations) was explicitly identified during Phase 2 as important preparation for Phase 3, and it stands as such — preparation, not something already done.
2.4 — Tuning the inference and runtime
With execution persistent, the tuning questions changed. The dominant ones in Phase 2 were operational: how long an execution may run before it is declared timed out; how a failed or partially-failed attempt is recovered within its budget; how the Runner reports a failure clearly enough that the next attempt can be a correction rather than a repeat; and how ready a fresh execution environment is before the first action runs.
Two inference-related decisions are worth recording specifically. First, the reasoning effort for project worker roles was standardised to medium: enough deliberation for real work, without attempts that run so long they hit their timeout limit before they produce a receipt. Second, the Observer function — the oversight and coaching view over the run — moved from its earlier execution environment to Codex CLI, leaving the locally hosted Qwen3.8 on the AI PC as the inference for the project worker roles themselves. That distinction is deliberate: local inference is what runs the project work; the oversight and coaching tooling around it is external, and the site should not present the Factory as though every function in it ran on the local model.
The Runner itself improved across the phase: readiness checks before dispatch, clearer error reporting, and a bounded recovery path so a failed attempt ends in a recorded state the Controller can reason about, not a dead environment.
2.5 — A phase that turned out to be an infrastructure phase
When Phase 2 was scoped, it looked like a content phase with a bit of automation attached. In execution it turned out to be substantially more infrastructure-intensive than that prediction, because every reliability gap in the execution stack had to be closed before another iteration could be trusted to run.
The work that fell under that: persistent Runner reliability and lifecycle management (what happens when an environment dies mid-iteration, and how it is re-established without losing the accepted candidate); integration between the n8n Controller and the Runner (who owns what, and how a stage’s outcome is communicated back); the role instructions, tool contracts and execution interfaces that tell each stage exactly which tools it may use and what shape its report must take; the handoffs of test evidence, validation results and security verdicts between stages; network and package access controls; the approved package and container registries; observability over what packages are actually being downloaded; and container-image validation with release-boundary controls.
The right way to describe this body of work is as foundational controls introduced and refined during Phase 2 — not as a completed security framework. Several of them are compensating controls, and the phase’s own checkpoint record says so. The point of the section is that the reliability problems in 2.6 were not random bad luck; most of them were these controls being built while the machine they protect was already running.
2.6 — The numbers, and what they are and are not
The recorded statistics for Phase 2’s automated execution, from the Factory’s own records of the two Controller runs:
| Metric | Result |
|---|---|
| Controller runs | 2 |
| Role dispatches | 108 |
| Successful dispatches | 24 |
| Failed dispatches | 82 |
| Timed-out dispatches | 2 |
| Dispatch pass rate | 22.2% |
The first Controller run was closed without completing Phase 2. The subsequent run completed four development iterations and reached the approved checkpoint. That is the whole run-level story, and it is the reason the table is what it is.
One distinction must be kept or the numbers become misleading. The broader development effort behind these phases involved more than 150 reported runs or attempts — and that population includes failed starts, Phase 1 runs, and non-automated runs. It is not the same metric as the 108 role dispatches recorded in the two Phase 2 Controller runs, which cover only the automated dispatches, successful and unsuccessful, in those two runs. The two populations are presented here separately on purpose, and they should not be combined.
The failed dispatches also need careful framing. A dispatch that failed is not automatically evidence of defective code. Most of the 82 failures in this phase were operational: timeout and recovery failures, weak role and tool interfaces, incomplete or ambiguous evidence handoffs, Runner and Controller state-management problems, difficulty maintaining continuity between iterations, and network and package restrictions that blocked validation work. The failures associated with a genuine technical implementation being wrong are a much smaller part of the total. Separating the two matters, because the corrective action is different: operational failures are fixed in the infrastructure and the contracts, not in the feature code.
One specific case is worth naming. During the fourth development iteration, ten Validator dispatches failed before one succeeded, in the middle of the period when the network and package security controls were being introduced and tuned. That is the single clearest instance of the infrastructure work in 2.5 and the reliability problems in this section overlapping in one iteration, and it is the main reason that iteration’s pass rate looks so much worse than the phase’s.
And one more separation, for anyone reading the timings: the total elapsed execution time of a dispatch includes tool execution, waiting, scheduling and recovery — not just the model thinking. The actual model inference time is a part of the elapsed time, not the whole of it. Conflating the two is the most common misreading of dispatch timing, so the site states it here rather than letting it be inferred.
2.7 — What Phase 2 actually established
What the subsequent Controller run delivered, on the record:
- the run itself completed, where the first did not;
- four sequential development iterations, each carried through Developer, Security and Validator completion to acceptance;
- locally hosted inference operating through the Factory across a full phase — the demonstration this phase existed to give;
- a materially improved operational foundation: iteration continuity, bounded attempts with recovery, execution controls, and the infrastructure in 2.5;
- the lesson, learned at the price of the 82 failed dispatches, that the operational foundation is not a preface to the real work — it is most of the work, and it has to be built while running.
The lesson compressed: a role that cannot see the shape of its own report, an interface that does not check the receipt before the next action, or a handoff that drops the accepted candidate, will not survive a persistent Runner. Clear tool contracts, reliable feedback, and context handoffs that actually carry the declared inputs are the unglamorous core of what makes the rest of the Factory possible.
Balanced, the phase is neither a straightforward success nor a failure. It did not achieve autonomous delivery, and its early reliability was poor. What it did establish — a locally hosted model running real project work through a persistent, contract-checked, review-gated loop, four times in sequence, to an approved checkpoint — is a real foundation. That is what the next phase builds on.
Related artefacts
- Phase 2 checkpoint record: four accepted iterations, every Validator and Security verdict PASS, no medium-or-above findings, no new Core Requirements
- The pull request that introduced the Runner adapter design -- letting a Factory dispatch select OpenCode or OpenHands
- The pull request that added OpenHands role support to the Factory
- The pull request that established the iteration package and continuity model
- The pull request that defined the role-stage contracts and execution interfaces
- The pull request that introduced the approved registry and package access controls
- Phase 2 Controller run and role-dispatch statistics (2 runs, 108 dispatches, 22.2% pass rate)
- The Phase 3 preparation TODO -- a proposed work sequence, not a record of completed improvements