34 Agents in Production: What Building Nova Actually Taught Me

<p>Nova is now in production orchestration. That phrase sounds clean, but the path to it was anything but. Over the past several months I've gone from a handful of experimental agents to 34 running across general chat, pipeline execution, ERP specialists, language tasks, and a growing set of domain-specific tools. It's the most complex system I've personally built, and it's taught me more about software architecture than almost anything else I've worked on.</p><p>The first thing that surprised me was how quickly the coordination problem overtakes the capability problem. Early on, the challenge is getting individual agents to do useful things. That's mostly a prompting and tooling problem — solvable, iterative, satisfying. But once you have a dozen agents that can each do something well, the hard question becomes: who decides what runs, when, and in what order? That's an orchestration problem, and it's closer to distributed systems design than it is to anything I'd associate with "AI development." You're thinking about state, handoffs, failure modes, and idempotency. The AI part almost fades into the background.</p><p>The second thing I learned is that agents need sharp boundaries. When I gave agents broad mandates — "help with ERP tasks" — they became unpredictable and hard to debug. When I tightened the scope to something specific — "validate branch-level data consistency in AIREP" — they became reliable tools I could reason about. This mirrors what I already knew about good software design: small, focused units with clear interfaces. The same principles apply. The novelty of the medium doesn't exempt you from the fundamentals.</p><p>What's genuinely different about building with agents is the feedback loop. In traditional development, you write code, run it, read an error, fix it. With agents, the failure modes are often semantic rather than syntactic. The agent did something — it just wasn't the right thing, and it didn't crash, so you have to go looking. That demands a different kind of observability. I've spent a lot of time building logging and tracing into Nova not because the code was broken, but because I needed to understand what the system was actually deciding at runtime.</p><p>The goal I'm working toward now is a self-improvement loop — agents that can review Nova's own code, identify weak spots, and propose or apply refactors autonomously. That's not a gimmick. The practical value is that Nova touches a lot of my work: client projects, AIREP, Find a Sign, this site. If the system can improve itself incrementally without me driving every change, it compounds. The leverage gets real.</p><p>I want to be honest about where this sits right now though. "Production orchestration" means the architecture is stable and agents are running reliably against real tasks. It does not mean the self-improvement loop exists yet. That's the next phase. I'd rather say that plainly than imply I've built something I haven't.</p><p>The broader shift this work has forced is in how I think about AI's role in what I build. I used to reach for AI as an accelerant — something that made existing workflows faster. That's still true, but it undersells what's actually available. When you build infrastructure around AI rather than bolting it on, the system starts doing things you didn't explicitly program. Not in a magical way — in a compositional way. Agents combining their outputs produce results that neither could produce alone. That's not hype; it's just what happens when you design for it intentionally.</p><p>Nova is also the backend AI layer for this site — it handles memory, context, and agent dispatch for anything I expose here. So in a real sense, the system is already part of how keirantrace.com works, not just a project I'm writing about. That matters to me. I want the things I build to be things I actually use, not demos that exist to look impressive in a portfolio.</p><p>34 agents. Production. More to build. That's where things stand.</p>