AI review bots are useful. On my team one reads every pull request, and it catches real things: a missing null check, a query inside a loop, a typo in an error message. I'd keep it.
What I've stopped expecting is that it helps anyone understand the change. Those are two different jobs, and on big AI-written pull requests the second one is the one going wrong.
What a bot reviewer is answering
A bot reviewer reads the diff and asks, line by line or file by file, whether something here looks like a problem. When it finds one, it leaves a comment on that line.
That's a good question to have answered. It's a different question from the one the person pressing Approve has to answer, which is closer to this: what does this pull request do, does it do what was asked, and am I happy for it to go to production with my name on it?
You can't get to that from comments. A comment is about one spot. Understanding a change means knowing how the spots connect: what the migration stores, which service writes it, which endpoint calls the service, what the page does with the response. Twenty comments on twenty lines don't add up to that, any more than twenty sticky notes on a map tell you the route.
More comments means more reading
On a 60-file pull request, a bot might leave 20 comments. Each one is reasonable on its own. Together they're another document to read, on top of the diff, and they arrive without an order.
They also don't always agree. On our team we've seen one comment ask for a check to move into a shared helper, and a later run ask for the same check to be inlined. Somebody has to decide, and deciding needs the context the comments don't give.
The loop where nobody reads
Here's a sequence that happens a lot now:
- An agent writes the pull request.
- A bot reviewer leaves comments.
- An agent addresses the comments and pushes fixes.
- The threads get resolved.
- A person sees a green check and resolved conversations, and approves.
Every step looks responsible. The pull request was reviewed, the comments were addressed, a human signed off. At no point did a person read the change and build a picture of how it works.
That's fine until it breaks. Then the person on call opens the file and has to learn the code from scratch, under pressure, because nobody who touched it actually knows it. A resolved review thread is no help at that point.
Understanding is a reading problem
If the goal is for a person to understand the change, the help has to be aimed at reading. In my experience three things make the biggest difference:
A summary first. Two sentences on what the pull request does, before any code, so you know what you're looking at.
A sensible order. Data, then logic, then the edges that call it, then the interface, then tests. GitHub's alphabetical list puts the page before the migration it depends on, and you spend your energy rebuilding the order yourself.
A reason next to each change. What changed, and why it's there. The why is what lets you spot a change nobody asked for.
None of this replaces reading. It makes reading the code faster and less tiring, so people actually do it.
What I built for this
I built Hyperpowers, a free VS Code extension, to do that part. You paste a pull request link, your own Claude Code explains it, and you go through it one change at a time in the order the code runs, with What changed and Why next to the real diff. When the code was written with Claude Code, the why comes from the session logs, with secrets removed, so you're comparing the code with what was actually asked. You comment on lines, and nothing posts until you press Submit review.
It has a bot-style pass too, called Review with Claude, because sometimes you want a second pair of eyes. It's optional and scoped to a file, an area or the whole thing. Findings come one at a time and only become comments if you press Add as comment. I didn't want to build a tool that adds to the pile of unread comments.
Explanations can be wrong too
A model's explanation of a change can be wrong, the same way a bot's comment can be wrong. The difference is where it sits. An explanation is shown next to the code it describes, while you're reading that code, so a wrong one tends to stand out. A wrong bot comment that an agent then "fixes" can change the code without anyone checking whether the comment was right.
So I'd still say: read the code. Use the explanation to know where you are and what to look for.
Use both
Running a bot reviewer on every pull request is a reasonable default, and it catches things people miss when they're tired. Just don't count its comments as someone having understood the change. For the big pull requests, and the ones where your name is on the approval, a person still has to read them.
For a step-by-step method, see how to review a pull request that AI wrote. For how bot reviewers and guided review compare side by side, see AI code review tools compared.
If you want to see what reading with a guide feels like, try Hyperpowers in your browser on an example pull request. There's nothing to install. To use it on your own pull requests, install the extension. It runs on your own Claude Code plan.
Common questions
Should I turn off my AI review bot?
No need. A bot reviewer catches real problems on single lines, like a missing null check or a query inside a loop. Keep it, and don't count its comments as someone having understood the change.
Why don't AI review comments help me understand a pull request?
A comment is about one spot in the diff. Understanding a change means knowing how the spots connect: what the data stores, which code writes it, and what calls that code. Comments arrive without that order.
Can an explanation from a model be wrong?
Yes. In Hyperpowers each explanation sits next to the code it describes, while you read that code, so a wrong one tends to stand out. Read the code and use the explanation to know where you are.