← Back to blog

Why I Don't Trust My Own Code Reviews Anymore

· 3 min read
Why I Don't Trust My Own Code Reviews Anymore Why I Don't Trust My Own Code Reviews Anymore

Building agents to review Nova's own code has made me rethink what a 'review' is actually for — and why a human skimming a diff is often the weakest link in the chain.

<p>I've been working on the self-improvement loop for Nova — agents that read Nova's own code, find things worth changing, and propose fixes. The obvious objection is trust: do you really want AI rewriting the thing that runs your life? Fair question. But working on this has made me more suspicious of something else entirely: my own code review habits.</p><p>When I review a diff by hand, I'm pattern-matching. I look at the shape of a change, check it against what I expect, skim for anything that looks wrong, and move on. That's fine for small changes. It falls apart the moment the diff touches something with second-order effects — a schema change that ripples into three other modules, a refactor that quietly changes behaviour under a specific edge case. I don't always trace those paths. I don't have time to, and honestly, most humans reviewing code don't either. We're trusting vibes dressed up as rigour.</p><p>An agent reviewing code doesn't get tired and doesn't skim. It can actually trace the call graph, check every caller of a changed function, and flag the edge case I would have glossed over on a Friday afternoon. That's a real capability difference, not a marketing one. But it comes with its own failure mode: an agent can be confidently wrong in ways that look exactly like being confidently right. It doesn't have the felt sense of unease a human gets when something seems off but they can't articulate why. That unease is often the first signal that something's actually wrong.</p><p>So the loop I'm building isn't "agent finds problem, agent fixes problem, done." It's agent finds problem, agent proposes fix, and a human — me — has to approve it before it goes anywhere near production. I've written about why a self-improving system still needs a human veto, and this is the sharper version of that point: the veto isn't there because I don't trust the agent's technical judgement. In a lot of cases its technical judgement is better than mine. The veto is there because the review process itself needs a second, different kind of check — not a re-run of the same pattern-matching I'd apply by hand, but a sanity check on intent. Is this the problem actually worth solving right now? Does the fix match what I'd want even if it's technically correct? That's not a code quality question, it's a judgement question, and it's the one place I still have an edge.</p><p>What's changed for me isn't that I trust AI code review more than human review. It's that I've stopped pretending human review was ever as rigorous as it felt. Going through this process with Nova has forced me to be honest about what my own reviews actually catch versus what they just make me feel like I caught. There's a difference between reviewing code and performing the act of reviewing code, and I think a lot of engineering teams — including, for years, me — have been doing more of the latter than they'd admit.</p><p>None of this means I'm handing over the keys. It means the review process itself needs to be redesigned around what each party is actually good at. Agents trace paths exhaustively and never get bored. Humans hold context about what actually matters and can smell intent mismatches that no amount of path-tracing will surface. Building the loop properly means using both for what they're for, instead of either trusting the machine blindly or pretending my own skim-reading was ever the gold standard to begin with.</p>

Comments

No comments yet — be the first!

Leave a comment

Comments are held for moderation before appearing.