The Self-Improvement Loop Is Mostly Just Good Code Review

<p>One of the goals I've been circling for a while is a self-improvement loop inside Nova: agents that read Nova's own codebase, spot problems, and fix them. It sounds like the flashy end of agent work. In practice, almost all of the effort has gone somewhere much less interesting, and I think that's the actual lesson.</p><p>Nova is at 34 agents now, spread across general chat, pipeline execution, ERP specialists, and language tasks. It's in production orchestration — meaning it's not a demo I poke at, it's a thing I depend on during a working day. That changes the risk profile completely. An agent that rewrites part of a toy project is a party trick. An agent that rewrites part of a system you rely on is a liability unless you've done the boring work first.</p><p>The boring work is verification. When I actually sat down and asked myself "what would have to be true for me to let an agent merge a change to Nova unattended?", the answer had nothing to do with the model. It was a list of things about my own codebase: do I have tests that would actually fail if this broke? Can I scope the diff so a bad change can't reach the orchestration layer? Do I have a way to see, after the fact, exactly what was changed and why? Can I roll it back in one command?</p><p>Every one of those is something I should have had anyway. The agent didn't create new requirements — it just made the existing gaps impossible to ignore. I've written plenty of code over the years with the safety net of "I'll notice if it's wrong." That net is me. It doesn't scale, and it definitely doesn't extend to a process running at three in the morning.</p><p>So the loop I'm building looks less like an autonomous engineer and more like a very patient junior with a strict process. It gets a narrow scope — one module, one class of problem. It proposes a diff, not a merge. It has to explain the change in terms of an observable failure or a stated standard, not a vibe about cleanliness. And it runs against the test suite before I ever look at it. The interesting judgement call is still mine; what the agent removes is the part where I have to go looking for the work in the first place.</p><p>That framing has clarified something for me about AI-assisted development generally. The leverage isn't in the generation step. Generating code has been cheap for a while now. The leverage is in how fast and how cheaply you can tell whether generated code is correct. If verification is slow and manual, more generation just gives you a bigger pile to inspect, and you go slower. If verification is fast and automated, generation compounds. Same model, opposite outcome, and the difference is entirely in your own infrastructure.</p><p>This is why I'm sceptical of most of the "AI writes your whole app" pitches. They optimise the half that was already easy. Nobody is selling you better test coverage of your existing untested system, because that's unglamorous and specific to you. But that's the actual bottleneck.</p><p>There's a second-order effect I didn't expect: building for the agent has made the codebase better for me. Modules with clear boundaries are easier to scope an agent to, and they're also easier to reason about at 11pm. Changes that are easy to review automatically are easy to review manually. Writing a spec precise enough for an agent to follow forces me to know what I actually want. I've had a few cases where writing the instruction was enough — once it was clear, the fix took two minutes and I just did it myself.</p><p>So the honest status: the loop isn't autonomous yet, and I'm in no hurry to claim it is. What exists is a narrower thing — agents that find and propose, with me as the gate. The gate comes down as the verification scaffolding goes up, one category of change at a time. That feels like the right order. The alternative is trusting a system because it sounds impressive, and I'd rather trust it because I can prove what it can't break.</p>