34 Agents in Production: What Running a Multi-Agent AI System Actually Looks Like

<p>At some point in the last few months, Nova stopped being a side project and became infrastructure. It's now in what I'm calling the production orchestration phase — 34 agents running across general chat, pipeline execution, ERP specialists, language tasks, and more. That number sounds impressive, but the honest version of this post is about what that actually means day-to-day, and what building it has taught me.</p><p>First, the architecture. Nova isn't a single model with a long system prompt. It's a swarm of purpose-built agents, each scoped to a domain. Some handle general conversation. Some are specialists — there are ERP-focused agents that understand AIREP's data model, branch-scoped tenancy, and Django conventions. There are language task agents for summarisation, rewriting, and extraction. There are pipeline agents that coordinate work across the others. Each agent has a narrow job, and the orchestration layer figures out which one to invoke.</p><p>That design decision — narrow agents over a fat generalist — was the right call, but it created its own complexity. When something goes wrong, debugging a multi-agent system is genuinely harder than debugging a monolith. You're not looking at a stack trace. You're looking at a chain of decisions: which agent was selected, what context it received, what it returned, and whether the orchestrator interpreted that correctly. Observability matters enormously here. I've spent more time building logging and introspection tooling than I expected.</p><p>The second thing I've learned is that agent quality degrades at the edges. An agent that performs well on its core use case will often hallucinate or go sideways when a prompt is ambiguous or slightly out of scope. The fix isn't a smarter model — it's tighter scope definitions and better routing logic. The orchestrator needs to know when a task doesn't fit any agent cleanly, and either ask for clarification or escalate rather than guessing. That's still an area I'm actively tightening.</p><p>Production also means living with the system. Nova runs against my real notes, my real projects, my real calendar. It has memory across conversations. That changes how you think about it — you stop treating it like a chat interface and start treating it like a collaborator that has context. The value compounds over time in a way that a stateless assistant never could. But it also means mistakes persist. A bad memory write, a misclassified note, an incorrect inference that gets stored — those propagate. Building the self-correction loop to catch and fix those is the next major piece of work.</p><p>One goal I've been working toward is a self-improvement loop: agents that can review Nova's own code, flag inconsistencies, suggest refactors, and eventually submit changes for review. That's not science fiction at this point — it's an engineering problem. The hard part isn't getting a model to write code. The hard part is giving it enough context about the system's intent, constraints, and history that its suggestions are actually useful rather than technically valid but architecturally wrong. That's a problem of documentation and structured memory as much as it is of model capability.</p><p>What I keep coming back to is that multi-agent systems aren't primarily an AI problem. They're a software engineering problem. Routing, observability, failure handling, scope management, memory consistency — these are solved problems in distributed systems. The challenge with agents is that the failure modes are fuzzier and the interfaces are natural language instead of typed contracts. But the engineering discipline is the same: define boundaries clearly, log everything, fail loudly, and don't let complexity hide behind "it's AI."</p><p>Nova is useful now. It saves me real time on real work. But it's also a system I maintain, debug, and extend like any other piece of software I own. That's the honest picture of what running 34 agents in production looks like — not a magic layer on top of everything, but a codebase with opinions, edges, and room to grow.</p>