The Agents Were the Easy Part

<p>Nova has crossed into what I've been calling the production orchestration phase. It's no longer a pile of scripts I run when I feel like it — it's a system that sits behind my work, behind keirantrace.com, and increasingly behind how I move between projects. There are 34 agents in it now, spread across general chat, pipeline execution, ERP specialists, and language tooling.</p><p>Here's the honest lesson from getting there: the agents were the easy part.</p><p>Writing an agent is close to trivial these days. Pick a model, write a system prompt, give it a few tools, done. I can stand up a competent narrow agent in under an hour. The first ten felt like progress. Every one of them worked in isolation. I'd test it, it'd do the thing, I'd move on feeling productive.</p><p>Then you have thirty of them and you discover that you haven't built a system, you've built a staffing problem. Who answers this request? What does that agent get told about what already happened? Where does its output go? What happens when it's wrong, and who notices? None of those questions are answered by the agents themselves. They're answered by the layer above, and that layer is where all the actual engineering turned out to be.</p><p>Routing was the first thing to bite. Early on I did the obvious thing — let a model look at the request and pick an agent. It works about as well as you'd expect, which is to say most of the time, which is to say not well enough. A router that's right 90% of the time in a multi-step pipeline compounds into something you can't trust. I've ended up much more conservative: explicit routes where the intent is knowable, model-based routing only as the fallback, and a clear log of which agent handled what so I can go back and see where a chain went sideways.</p><p>State was the second. An agent with no memory is a stateless function and easy to reason about. An agent with memory is a database with opinions. Getting the boundaries right — what belongs in a profile that's always present, what belongs in archival storage you go and fetch, what's just conversational noise that should be allowed to die — took more iterations than any individual agent did. Over-remember and you poison every prompt with irrelevance. Under-remember and the system feels like it has a head injury.</p><p>The third thing, which I'm still working through, is that orchestration is fundamentally an ERP problem wearing different clothes. That sounds like a stretch until you've built both. In AIREP I spend my time on multi-tenant boundaries, branch-scoped data, and making sure the right record is visible to the right context and invisible everywhere else. In Nova I'm doing exactly that, except the tenants are agents and the records are context. Same discipline: define the scope, enforce the isolation, don't let convenience leak data across a boundary you'll later regret.</p><p>Which is a nice confirmation of something I already half-believed — the AI part of building AI systems is the part with the least novel engineering in it. The models are commodity. The prompts are a weekend. The differentiator is the boring architecture underneath: routing you can audit, state you can reason about, failure modes you can see.</p><p>Next on the list is the self-improvement loop — agents that review and refactor Nova's own code. I'm deliberately going slowly on that one. The failure mode of an agent that can edit the system it runs on isn't subtle, and I'd rather it be narrow and observable than impressive and opaque. Read the diff, propose the change, let me approve it. That's the whole ambition for version one. If it saves me an hour a week, it's earned its place.</p><p>The takeaway I'd offer anyone building with agents right now: stop counting agents. It's a vanity metric, and I say that as someone with 34 of them. Count the number of decisions your orchestration layer makes that you can explain after the fact. That number is the one that tells you whether you have a system or a collection.</p>