34 Agents in Production: What Running a Real AI System Actually Looks Like

<p>Nova is in production. Not "production" in the demo-video sense — I mean it's the system I actually use every day to manage work across multiple projects, handle reminders, draft documents, query my own notes, and coordinate tasks I'd otherwise context-switch through manually. It has 34 agents running across general chat, pipeline execution, ERP specialists, language tasks, and a few others I've bolted on as the need arose. That number crept up faster than I expected.</p><p>When I started building Nova, the mental model was simple: one agent, one task, done. A glorified wrapper around an LLM with some memory bolted on. What it's become is something closer to a small distributed system — agents that hand off to each other, a shared memory layer that persists context across conversations, and an orchestration layer that figures out which agent should handle what. Getting that right has been the bulk of the work over the last several months.</p><p>The hardest part isn't writing individual agents. Writing an agent is easy. You give it a system prompt, some tools, maybe a few examples, and it does something useful. The hard part is the seams — what happens when one agent finishes and another needs to pick up? What's the contract between them? How do you avoid the situation where context that was obvious in conversation A is completely invisible in conversation B? Memory architecture turns out to be the most important engineering decision in the whole system, and it's the one that gets the least airtime in the discourse around AI agents.</p><p>I've settled on a tiered approach: a core profile that's always injected (stable facts, current goals, active projects), an archival layer for free-form notes and lessons that need to persist but don't need to be in every prompt, and episodic memory for recent conversation context. Each tier has a different retrieval path. Getting the balance right — what to promote to core profile versus what to archive versus what to just let expire — is still something I'm tuning. Promote too aggressively and the always-injected context becomes noise. Promote too conservatively and agents keep rediscovering things they should already know.</p><p>The production orchestration phase, which is where I am now, is less about adding new agents and more about making the existing ones reliable and composable. An agent that works 90% of the time in isolation is not good enough when it's one step in a five-step pipeline. Errors compound. A bad handoff early in a chain produces garbage at the end, and the garbage looks confident. That's the thing about LLM-based systems that you don't fully appreciate until you're running them in anger: failure modes are often fluent. The system doesn't crash — it just quietly goes wrong.</p><p>So a lot of my current work is instrumentation and guardrails. Logging what each agent receives, what it produces, and whether the downstream agent actually used it correctly. Building in checkpoints where I can inspect intermediate state before the pipeline continues. This isn't glamorous, but it's the difference between a system that's a toy and one that's actually useful.</p><p>One thing I've become convinced of through this process: the value of an AI system like Nova is almost entirely in the memory and orchestration layers, not the individual agents. The agents themselves are relatively interchangeable — swap the underlying model, rewrite the system prompt, the agent still does roughly the same thing. But the memory layer that gives an agent context about who I am, what I'm working on, and what was decided last week? That's genuinely hard to replicate and genuinely hard to rebuild if you get it wrong. It's also where the compounding advantage comes from. Each conversation makes the system slightly more useful because the context it carries forward is slightly richer.</p><p>I'm building Nova partly because I need it and partly as a proof of concept for what AI integration actually looks like when it's done seriously — not a chatbot widget, not a one-shot API call, but a system with memory, specialisation, and real orchestration. The goal from here is a self-improvement loop: agents that can review and refactor Nova's own code. That's the next hard problem. I'll write about it when I've got something real to say.</p>