AI Code Review: What It Catches, What It Misses

The pitch for agent code review is that it never gets tired, never skips a file, and never rubber-stamps a 40-file diff at 5 p.m. on a Friday. All three are true. None of them are the reason agent review fails when it fails.

The direct answer: AI code review effectiveness splits cleanly into two halves. Agents are genuinely strong at mechanical, checklist-shaped problems: missing error handling, inconsistent naming, unclosed resources, a changed branch with no matching test. They are weak, reliably and predictably, at architecture, at judging intent against a specification they never saw, and at the question that matters most on any pull request: whether the change should have been written at all.

AI Code Review Effectiveness: Where It Beats a Tired Human

Consistency is the real advantage, not intelligence. A human reviewer's attention degrades across a long diff: the first ten files get a careful read, the last ten get a skim, and a Friday-afternoon review of a 40-file pull request is a worse review than the same reviewer would give on Tuesday morning. An agent applies the same checklist to file 40 that it applied to file 1. It doesn't decide a hunk is "probably fine" because it's tired of reading. That breadth is where the honest wins live: catching a missing null check on line 380 of a file nobody read past line 60, flagging an inconsistent error-handling pattern that drifted in three commits ago, or noticing that a changed function has no corresponding test update anywhere in the diff.

The Defect Classes Worth Delegating

The failures agent review catches well share a shape: they're locatable by pattern, verifiable without broader context, and don't require knowing why the code exists. Unhandled exceptions, resource leaks, naming drift from the surrounding file's conventions, an off-by-one in a loop bound, a copy-pasted block that diverged from its original in one place it shouldn't have. Testing AI-generated code covers the sibling problem on the test side: a passing suite doesn't prove the fix works, and the same mechanical, pattern-shaped checking that works for tests works for the code around them.

This class of check matters more than it used to, because the code showing up in review has changed. Independent analysis of open-source pull requests by CodeRabbit, reported by The Register in December 2025, found AI-authored pull requests averaged 10.83 flagged issues against 6.45 for human-written ones in the same repositories, with roughly 1.4 times more critical-severity findings. That's an argument for treating an AI-authored diff as a higher-scrutiny review, not a lower one, and it's exactly the mechanical class an agent reviewer is built to catch at volume.

AI Reviewer vs Human Reviewer, By Task

Review task	Agent reviewer	Human reviewer
Missing error handling, unclosed resources	Catches reliably, every file	Catches when attention holds
Naming and convention drift	Catches reliably	Often skipped as "style nitpick"
Whether a changed branch has a test	Catches reliably	Frequently missed on large diffs
Whether the abstraction is the right one	Rarely surfaces the question	Where this judgment lives
Whether the change matches an implicit requirement	Cannot verify against context it never saw	Where this judgment lives
Whether the feature should exist at all	Never asks	The only place this gets asked
Consistency across a 40-file diff	Same depth on file 1 and file 40	Degrades with reviewer fatigue

The diagram below shows the same split as a boundary: one side is what pattern-matching review reaches, the other is what stays a human judgment call regardless of how good the tooling gets.

The Failure Worth Naming: A Confident Review of Code That Shouldn't Exist

The dangerous outcome isn't a missed null check. It's an agent reviewer approving a pull request, cleanly, with every mechanical check green, for a feature that never should have shipped: an abstraction that duplicates one three files away, a workaround for a problem better solved upstream, scope creep nobody scoped. A clean approval reads as a stamp of quality, and a stamp of quality is exactly the wrong signal for a question the review never asked. AI agents in software development makes the same point from the delivery side: agents are strong at mechanical fixes and weak at judging whether a requirement is right, and code review inherits that split unchanged.

Getting the Split Right

1. Route mechanical checks to the agent first. Error handling, naming, unclosed resources, and missing tests are pattern-shaped problems; let the reviewer that never skips a file catch them before a human's time gets spent on them.
2. Keep architecture and "should this exist" with a human, every time. No amount of pattern-matching depth substitutes for the judgment call these questions require.
3. Scope the reviewer to the diff, not the narrative around it. A PR description or comment shouldn't be able to talk an agent reviewer into approving something the code itself doesn't support.
4. Treat an approval as evidence, not a decision. How to review AI-generated work before you ship it covers the ordering that applies here too: check presence and correctness before quality, and don't let a clean-looking approval skip that order.
5. Give AI-authored diffs more scrutiny, not less, given what the data on issue density in AI-generated code actually shows.

Getting this split right is mostly a matter of routing, not tooling: send the checklist-shaped work to whatever catches it fastest and keep the judgment calls where judgment lives. Onplana's issue tracker keeps both classes of finding, mechanical and architectural, in the same log against the same task, so neither gets lost in a review thread that scrolls away. The rest of the Onplana blog covers where else agent judgment holds up and where it doesn't across the delivery pipeline.