@@ · our position @@

What AI should (and shouldn’t) do in code review

Othman Shareef · July 14, 2026 · 6 min read · AI and Code Review

This series looked at what AI reviewers actually catch, what their noise does to reviewer attention, and what the generation flood is doing to the people who review for free. Time to put our own position on the record, including the part where we tell you what our product deliberately doesn’t do.

Upstream: where AI earns its keep

  • At authoring time: the agent that wrote the code drafting its own tests, the author using a model to interrogate their diff before anyone else sees it. This is self-review with a power tool, and every defect caught here costs nobody’s attention but the author’s.
  • As a pre-filter in CI: after the deterministic layer (linters, types) has taken everything decidable, a tuned-quiet AI pass on the risky paths. The Cotera numbers cut both ways: most suggestions duplicated a linter, but nine real catches in thirty PRs, before any human looked, is real money, provided the seven wrong ones didn’t cost the bot its credibility.
  • As the author’s explainer, not the reviewer’s: generating the “why” and the review map for the description, where the author can verify it before publishing. Generated context the author vouches for is context; generated context pushed straight at the reviewer is homework.

Downstream: what we refuse to automate

The approval is a human act (“I understand this well enough to co-own it”), and the research has been clear for a decade that review’s deepest value is shared understanding, not defect yield. That value is produced by the reading itself. An AI summary that replaces the reading doesn’t speed the review up; it hollows it out. And when the summary is wrong (the category’s signature failure), it hollows it out convincingly.

Why Pyor ships no AI summaries

So when people ask where the AI features are in Pyor, this is the answer: the product is the downstream half of the division. File triage that puts the three files that matter first; commit-scoped diffs so re-review costs only the delta; threads that survive force-pushes; focus mode; one window from read to merge. No generated prose between you and the diff, because our read of the evidence is that the prose is a tax, and because everything upstream can be done by tools that already exist, including the model you already pay for. We’d rather be the best place the human part happens than the fifth bot leaving comments.

The test for any AI review feature

One question sorts the catalog: does this need to be verified by the person it’s shown to? If yes, it spends reviewer attention and must out-earn its verification cost, a bar most generated commentary misses today. If no (because the author verified it upstream, or because it’s deterministic), it’s a free win. That test is vendor-neutral, model-agnostic, and it will still be the right question when the models are better than they are this year.

Frequently asked questions

Will AI eventually do the whole review?

It will keep absorbing the mechanical share. But an approval is a human taking ownership of a change on behalf of a team. That’s an accountability act, not a defect scan, and delegating it entirely means nobody on the team understands the system they ship. We’d bet on the mechanical share shrinking toward zero while the judgment share becomes the whole job.

Isn’t refusing AI summaries just contrarian positioning?

It’s a falsifiable bet: that reviewers are net-faster reading well-presented code than reading AI prose about the code plus the code (since the prose must be verified against the code anyway, when it matters). If models become reliable enough that their summaries don’t need verification, the bet loses and we’ll revisit. The evidence today (false positives as the category’s #1 complaint) says it holds.

← All posts