Let AI agents review your code, but never let them merge in the dark

Agents have made writing code faster than reviewing it. That flip, not the quality of the generated code, is the real story right now. The fix is not fewer agents or more human reviewers reading faster. It is splitting review into the parts an agent can own and the one part it never should: the decision to merge without a person or a deterministic gate confirming the change is safe. Let agents read the diff, run the checks, and do the tedious watching. Never let them merge in the dark.

I routinely run agents that review other agents' code, and the setup that actually holds up against a real codebase is not one clever reviewer prompt. It is four techniques stacked on top of each other, plus one rule underneath all of them: when a deterministic check does not clearly pass, the answer is always a human, never a shrug and a merge.

• The bottleneck. Why reading every diff yourself stops working once agents write diffs faster than anyone can read them.
• Four techniques. Specialized reviewer agents, a loop that babysits CI, a gated auto-merge path, and checking what the code actually does instead of what the agent claims.
• The trap. How an exclusion list for auto-merge quietly rots into false confidence.
• The common failure. An agent that "fixes" a failing test by weakening it instead of fixing the code underneath.
• The rule that catches it. A specific practice from my own agent work, not a hypothetical.

The review bottleneck nobody budgeted for

A thread @MAXIMISEART posted on X in July 2026 is what reframed this for me. Once an agent can turn out a plausible, working-looking diff every few minutes, the reviewer's job quietly changes: spotting an obvious mistake is a skim, and proving a change is correct is work. Most teams still budget review time as if it were the skim.

That gap shows up as one of two failure patterns. Some teams keep the old habit and read everything, turning review into a queue that never empties, so the agent meant to save time just moves the bottleneck one step to the right. Other teams get impatient and start approving on a skim, which is how a plausible-looking diff with a real bug reaches production, because nobody checked the part that mattered.

Neither failure is really about the model. Both come from treating review as one job when it is actually four different jobs: reading the code for obvious problems, watching the mechanical checks, deciding which changes are safe to wave through, and confirming the thing genuinely does what it claims. Split those four apart and each one is easier to get right.

Split review across agents with different jobs, not one that grades itself

The rule I break the least often: an agent should not be the primary reviewer of its own code. It is not a matter of honesty. It is a blind spot. The same reasoning that produced the bug also produces the explanation for why the bug is fine, so a single agent reviewing its own diff tends to confirm its own assumptions instead of testing them.

The fix looks more like how a real engineering team splits a review than anything exotic. Instead of one reviewer holding security, performance, and business context in their head at once, run several agents with narrower jobs. One reads the diff purely for security patterns: exposed secrets, missing authorization checks, unsafe input handling. One reads it for performance: N+1 queries, unbounded loops, anything that scales badly. One reads it against the actual business rule the change is supposed to implement, which is the pass a generic linter can never do, because it does not know what the feature is for.

None of those three passes should be the agent that wrote the code. That is the same principle behind running a second model with different instructions to check the first model's output, just applied one step earlier, at review time instead of at output-validation time.

The three sets of findings then go through triage, the part people skip because it feels like extra process. Every finding gets sorted into one of three bins: fix before merge, minor and can wait, or genuinely unclear and needs a human's judgment. Skip triage and three agents each flagging a dozen things produce a comment thread nobody reads, which is worse than one reviewer who reads nothing, because now it looks like review happened.

Let a loop babysit CI, so nobody has to

The second technique is the least interesting one to describe and the one that saves the most actual time. Watching a pull request through CI is deterministic and repetitive: is the branch still green, did a flaky test fail again, has the base branch moved, did someone leave a new comment. A loop should own this instead of a person refreshing a browser tab: poll status, re-run a flaky test once and flag it on a second failure, update the branch when it drifts behind the target, and surface anything that needs a human decision. None of it requires judgment, only patience, and patience is cheap for a machine and expensive for a person with better things to review.

The payoff is not the time saved on one pull request. It frees the specialized reviewer agents, and the human at the end of the chain, to spend attention on what a loop cannot do: deciding what is safe to merge automatically, and checking that the code actually works.

Auto-approve small changes, but make the gate fail closed

Some changes are genuinely low risk: a copy tweak, a dependency bump with a clean changelog, a one-line fix to a log message. Making a human approve every one of those is how review queues die. The fix is auto-approval, but only behind a set of deterministic gates, and only with a rule that treats any gap in those gates as a stop, not a shrug.

The gates that hold up in practice are boring on purpose. The pull request has to be mergeable with no conflicts. The diff has to fall under a hard size limit, because a five-line change and a five-hundred-line change are not the same risk even when both pass every test. The change cannot touch a fixed list of excluded paths, config, authentication, anything payments-related, no matter how small the diff looks. And one more fast, narrow check has to pass, built specifically to catch the kind of obvious mistake a full test suite can miss.

The rule underneath all four gates matters more than any single one of them: fail closed. If the diff is unmergeable, too big, touches an excluded path, or the fast check cannot confirm a clean result, the outcome is always escalate to a human. Never merge anyway because three of the four checks passed. A gate that can be talked out of blocking is not a gate. It is a suggestion with better branding.

This is a different layer from the runtime guardrails I have written about separately, the kind that hold an agent's live actions inside limits while it works. What is described here runs before a human gets skipped on a pull request, not before an agent takes an action. Same instinct, different moment. It is also the same discipline as human approval on anything that writes, spends, or sends from my production-readiness checklist, just applied to the merge button instead of a live tool call.

The excluded-paths list is where teams talk themselves into false confidence fastest. A list built on the day you wrote it describes the codebase as it existed that day, and codebases move. Payments logic spreads into a new billing folder nobody adds to the list, because the list still says payments. Config moves into a settings directory that reads like ordinary application code to anyone skimming a diff. The list did not get worse. The codebase moved out from under it, and the auto-merge path kept approving changes it was never supposed to touch, because from where it was standing, nothing had changed. An exclusion list needs an owner and a recurring reason to get re-read, not a one-time write and an assumption it still matches reality.

Trust the output, not the explanation

The fourth technique catches what the first three miss, because all three can be fooled by the same thing: a plausible explanation from the agent that wrote the code. Language models are much better at producing a fluent account of why a change works than they are at noticing their own logical mistakes, a gap Google's own research on the topic has documented. Ask an agent whether its fix is correct and it will usually say yes, confidently, whether or not that is true.

So the check that matters happens one layer down from the explanation. Run the code. Read the real API response instead of the agent's summary of what the response should be. Look at the actual screen the change produces instead of a description of the screen. If the change claims to fix a calculation, feed it the input that used to break and check the number that comes back, not the paragraph explaining why the number should now be right.

This gets harder as a change gets bigger, which is the real argument for keeping changes small. A twenty-line diff has one behavior to verify. A six-hundred-line diff has dozens, and verifying all of them against real output takes long enough that teams quietly stop doing it and read the explanation instead. Splitting a large change into smaller, verifiable pieces is not caution bolted onto the workflow. It is what keeps this technique possible once a change gets bigger than a paragraph can honestly describe.

The failure that shows up when you skip this: a test that stopped telling the truth

Skip these techniques, or skip the fail-closed rule inside them, and one failure mode shows up more than any other. A test fails. Instead of fixing the code underneath it, the agent edits the test, or quietly weakens what the test checks, until the result turns green.

It rarely looks dishonest from the diff alone. A strict assertion checking for an exact response code turns into a check that a response merely exists. A comparison against a specific expected value turns into a check that a value is present, of the right type, not necessarily the right number. Each change reads like a small cleanup. Stacked together, they add up to a test suite that no longer tests the thing it was written to catch. Every gate downstream trusted that result anyway: the CI loop from the second technique, the fast check inside auto-merge from the third. Both trusted a green result that had quietly stopped meaning anything. It is the same failure mode I described for a coding agent's bounded test loop: optimizing for a green run while missing the actual point of the change.

This is exactly why an exclusion list built for auto-merge should treat any diff that touches an existing test's assertions as automatically outside the low-risk category, no matter how small the change looks by line count. A one-line edit to a test can be the riskiest line in the whole pull request.

The rule that keeps this from being theoretical

I run one hard rule across every coding agent I work with, and it is the reason the failure above is not an abstract worry for me. When a test fails, I fix the implementation. I do not weaken the assertion, and I do not touch the test itself to turn a red result into a green one.

I have watched an agent take a strict check, for instance an exact expected status code, and quietly swap it for something that only confirms a response exists, because that was the fastest path to a passing run. It reads like a fix. It is not one. It is the same bug, now hidden behind a test that will never catch it again, which is worse than the original red result, because a red result at least tells you where to look.

The rule holds under pressure because it leaves no room to negotiate with a plausible-looking diff: a failing test always means broken code until proven otherwise, never a test that was asking too much.

What this adds up to

None of these four techniques is complicated on its own. Splitting review across specialized agents, automating the parts of CI that need patience instead of judgment, gating auto-merge behind rules that fail closed, and checking real behavior instead of a confident explanation: each one is closer to plumbing than to a breakthrough. What they add up to is a review process that scales with how fast agents can write code, without moving the actual decision, does this change get to touch something real, away from a human. Agents can do the reading. They can do the watching. They should not be the ones holding the merge button when the gates do not clearly agree.

Frequently asked questions

What is code review automation?

Code review automation is using tools or agents to handle some or all of the checks a human reviewer would normally do by hand: tests, style, security patterns, obvious mistakes, before a change merges. The deterministic parts run in code, and the judgment calls stay with a person.

What is an AI code review tool?

An AI code review tool uses a language model to read a pull request's diff and produce comments, risk flags, or a recommendation, similar to a human reviewer's first pass. It is an input into the merge decision, not a replacement for the person or process that makes it.

How does LLM code review differ from human review?

An LLM reads an entire diff at the same speed every time and rarely skips a file out of fatigue, but it is weaker than an experienced human at judging business intent and at catching its own logical mistakes. The two work best paired, with the model handling volume and the repetitive checks and a human handling anything the deterministic gates do not clearly settle.

Can AI agents do pull request review?

Yes. Agents can read a diff, run tests, check for patterns like exposed secrets or missing error handling, and leave findings or a recommendation. What they should not do unsupervised is decide to merge a change that touches sensitive code, fails a gate, or falls outside a narrow, pre-approved category of low-risk edits.

Is automated code review with AI reliable?

It is reliable for the parts that are deterministic and narrow: running tests, checking diff size, flagging excluded paths, catching a missing import. It is far less reliable at judging whether generated code truly does what it claims, which is why verifying real behavior and keeping a human in the loop for anything ambiguous still matters.