Human on the loop code review vs human in the loop: pick one honestly

Two phrases are doing heavy lifting in every discussion of AI and code review, and they are routinely used as if they were synonyms: human in the loop and human on the loop. They name different postures with different guarantees, and conflating them is how teams end up with the worst of both: the ceremony of full review with the assurance of none. This piece defines both terms crisply, then argues that honest human on the loop code review beats dishonest human in the loop review every single time.

Human in the loop: a person gates every change

In the loop is the classic pull request model. A human reads the diff, asks questions, requests changes, and only then approves. The guarantee is strong: no change reaches the main branch without at least one person having understood it, or at least having had the chance to. The cost is equally strong: review capacity caps merge throughput. Microsoft researchers Bacchelli and Bird found that understanding the change is the top challenge reviewers face, and understanding does not get faster just because the queue got longer. When authoring speed multiplies and review speed does not, an in-the-loop process becomes the bottleneck, and bottlenecks under pressure do not hold; they leak.

Human on the loop code review: sample, audit, intervene

On the loop comes from automation doctrine: the system acts on its own, and a human supervises with the power to intervene. Translated to code review, human on the loop code review means low-risk changes merge through automated gates (tests, static analysis, AI review) without per-change human approval, while a person watches the stream, samples merged changes for audit, and can stop the line at any point. Addy Osmani describes this shift in his agentic code review essay: as agents write more of the code, the reviewer moves from inspecting every line to supervising the process that inspects every line. The guarantee changes shape. You lose the promise that every diff was read; you keep, and must actively maintain, the promise that someone is watching and accountable.

Choosing by blast radius and volume

Neither posture is right in general; they are right for different changes. Two variables do most of the deciding:

• Blast radius. What breaks if this change is wrong, and how fast can you undo it? Auth, payments, data migrations, and public APIs justify a human gate. A copy tweak behind a feature flag does not.
• Volume. If agents open ten times the pull requests your reviewers can honestly read, insisting on in-the-loop for all of them just decides where the dishonesty happens. Better to gate the risky tenth properly and supervise the rest.

This is the same tiering logic we covered in our piece on agentic code review: rigor should follow risk, not habit. In-the-loop review is a scarce, expensive resource. Spending it uniformly across a firehose of low-stakes changes is how it stops being spent at all.

The failure mode: in the loop on paper, absent in practice

Here is the pattern worth being blunt about. A team requires approval on every PR, so it is officially in the loop. But volume has tripled, reviewers are exhausted, and approval has quietly become a formality measured in seconds. Osmani cites a Faros analysis of 22,000 developers in which zero-review merges rose 31.3% and defect rates jumped from 9% to 54%; the process did not announce its failure, it just stopped working. This is LGTM culture wearing an in-the-loop costume. It is strictly worse than honest on-the-loop review, because nobody is sampling, nobody is auditing, and everybody believes inspection happened. A rubber stamp provides the liability of review with none of the protection.

Making the posture explicit

The fix is not choosing one posture for everything; it is declaring which posture applies where, and then doing that posture properly. A workable split looks like this: define the paths and change types that always get a human gate, and hold that line without exception. For everything else, invest in the supervision that on-the-loop actually requires: real automated gates, a sampling routine (say, a fixed percentage of merged changes audited weekly), and dashboards that surface anomalies like test deletions or permission changes. Sampling is not a euphemism for nothing; an audit that finds a bad merge should trigger a revert and a widened sample. If your team cannot name its sampling rate, it is not on the loop, it is out of the loop.

A human still owns the merge

The deepest difference between the postures is not who reads the diff; it is that in both, a person answers for the outcome. On-the-loop done honestly keeps that chain of accountability intact: someone chose the gates, someone reviews the samples, someone can stop the line, and that someone has a name. What breaks accountability is the unacknowledged middle state, where the org chart says every change is reviewed and the commit log says otherwise. Pick the posture per tier, write it down, and make the supervision real. Pretending to be in the loop is the only option that fails silently.