Every AI review tool now opens with the same gift: a tidy paragraph explaining what the pull request does. It reads well. It is usually mostly right. And the moment you approve based on it, you have quietly changed jobs: you are no longer reviewing the code, you are reviewing a description of the code, written by a system that does not ship the consequences. Read the code, not the AI summary.
The verification asymmetry
The pitch for summaries is speed: read 200 words instead of 800 lines. The catch is that the 200 words are only trustworthy if someone checks them against the 800 lines, and that someone is you. When the stakes are real, you end up reading both, which is slower than reading the code alone. When the stakes feel low, you read only the summary, which is exactly when the substitution happens. The category’s own track record says this is not hypothetical: Cotera’s evaluation found 7 of 47 AI review suggestions flatly wrong, and false positives remain the #1 complaint across every tool. A summary is those same generations, stitched into confident prose.
Reading is not overhead. Reading is the review.
The research has said this for a decade: Microsoft’s study of modern code review found the reviewer’s hardest problem is understanding the change, and review’s most durable outputs are shared understanding and knowledge transfer, not just defects. That understanding is produced by the act of reading. Outsource the reading and the team still gets a green checkmark, but nobody’s mental model of the system updated. You have review theatre with better production values. As Addy Osmani put it about the agent era: reading code has always been the skill, and it is about to be the skill.
Where summaries genuinely help
- Upstream, on the author’s side. Generate the description draft, then the author verifies it line by line before publishing. Context the author vouches for is context (see capturing intent for AI changes); context pushed straight at the reviewer is homework.
- Orientation before the read, in unfamiliar territory. A map of an unknown codebase area can tell you where to start. It earns minutes only if you then spend them on the diff itself, files that matter first (triage is the real accelerant).
- Never as the basis for approval. The question to ask of any generated text in your review flow: does this need to be verified by the person it is shown to? A summary always does. Approval based on unverified text is delegation, not review.
Make reading cheap instead of replacing it
The honest fix for slow review was never less reading; it was cheaper reading. Small changes (the size discipline), triage that puts the three files that matter first, commit-scoped re-reads so round two costs only the delta, a surface that keeps your place. That is the direction we build Pyor in (disclosure: ours): the AI groups review-worthy changes apart from the noise, labels each group in one line, and points; it does not narrate, because the narration would be one more thing you have to verify against the code. Whatever tool you use, the test is the same: does it get you into the diff faster, or does it stand between you and the diff, speaking on its behalf?


The habit that survives every model upgrade
Models will get better, summaries will get more accurate, and the temptation to skip the read will grow in proportion. The habit worth keeping is older than any of this: the reviewer’s signature means I read it, I understood it, I would co-own it in production. That sentence has no summary-shaped substitute. Read the code. It is the only version of the change that is true.
Frequently asked questions
Are AI summaries of pull requests ever useful?
Yes, in two places: helping the author write an honest description they verify before publishing, and orienting a reviewer in unfamiliar territory before the real read. The failure mode is substitution: when the summary replaces the reading, the review inherits every mistake the summary made, and approval becomes a signature on someone else's claim.
Doesn't skipping summaries make review slower?
Only if reading stays as slow as the tools currently make it. The time sink in review is rarely raw reading speed: it is finding the files that matter, keeping your place, and re-reading on every round. Fix those and reading the code directly is competitive with reading a summary plus verifying it, which is the honest comparison.
What should I do when the summary and the code disagree?
Trust the code, always: it is the thing that ships. Then treat the disagreement as a signal about the summary generator, not a coin flip. A summarizer that got this diff wrong got other diffs wrong more quietly. Recalibrate how much orientation weight you give it, and say what you found so teammates recalibrate too.