Every team that tries an AI code reviewer goes through the same stages. In week one it catches bugs and impresses the team. By week three somebody asks, "Why did it leave three comments on a README change?" Soon everyone scrolls past the bot, including its useful comments.
AI review earns its place by catching bugs, not producing comments. Most AI reviewers are optimized for the latter. The noise comes from several parts of the review pipeline, not only model hallucinations, and each source needs a different fix.
What counts as noise
Three different things get lumped together as "false positives" and they need different fixes:
- Wrong findings. The bot says there is a bug and there is not. It missed that some check happens one layer up or invented a race condition that cannot occur.
- True but useless findings. Technically correct, practically irrelevant: a variable could be renamed, a function could theoretically be split. Nobody's incident report ever started with the variable name being slightly vague. These comments only help when the team asks the tool to find them.
- Restating the diff. "This change adds a parameter to the function." Yes, it does. I wrote it.
The false-positive rate also misses the damage to trust. A reviewer that is wrong one time in three does not keep two-thirds of your attention.
Where the noise comes from
1. The model only sees the diff
Missing context causes many wrong findings. A diff shows what changed, not what the change means. Whether a call is safe often depends on a type definition, caller or config default in a file the model never saw. Ask a strong model to review a fragment without context and it will guess, like a reviewer given page 212 of a novel.
So no prompt fixes this. If the contract between the code and the model was not in the input, the model usually cannot recover it. We walked through concrete cases in why diff-only code review misses bugs: the same missing context that hides real bugs also regularly manufactures fake ones.
2. Nobody checks the model's first draft
What separates these tools is mostly not the model: several of them call the same frontier models. The difference is what happens between the model's raw output and your pull request.
The cheapest architecture takes the model's first pass and posts it. That first pass at "list the problems with this code" includes guesses. A focused check such as "does updateRepository still invalidate the cache?" gives the model a concrete claim to verify. A second pass can reject unsupported findings before they reach the pull request.
Ask any vendor, us included: what happens to a finding between the model and my PR? If the answer is close to "nothing," you have found a likely noise source. Scopy independently checks findings against repository context before posting them. That step takes more time and model usage, but it filters out unsupported comments.
3. The bot has no taste
The "true but useless" category is a threshold problem here. A model asked to review code will always try to find something. It has no natural sense of which findings are worth a human's time. Left uncalibrated, it applies staff-engineer scrutiny to a typo in a comment.
The fix is more about discipline: findings should be filtered or labeled by whether they affect correctness, security or maintainability of the project, and pure style commentary should be dropped or held to an explicit opt-in from the maintainers.
4. It does not know your rules
Some noise is a genuine disagreement: the bot flags a pattern your team decided is fine or misses one your team specifically banned. A generic model knows general best practices and not the fact that your team allows raw SQL where an ORM call could be used. Every rule the tool does not know about becomes either a false positive or a missed catch, and both are bad.
What you can do about it, starting today
These apply to any AI reviewer:
Count before you tune. For a couple of weeks, track two numbers per PR: findings that led to a code change and findings dismissed. If the team dismisses most comments, tune the tool or replace it. The diff noise counter shows how much of your average diff is lockfiles and generated code that no reviewer should be reading anyway.
Raise the reporting threshold before turning the tool off. Restrict comments to correctness and security. A quiet bot that people read beats an annoying one.
Write down your standards as rules. If the tool supports custom rules, encode the recurring disagreements between your team's standards and its understanding. Every rule you add converts a repeat false positive into silence, or a repeat miss into a catch. This is what natural-language review rules are for.
Dismiss with a reason. When you reject a finding from the bot, a one-line comment helps in two ways: teammates stop re-checking it, and tools that learn from feedback have something to learn from.
Give it better PRs. Small PRs with a real description give human and AI reviewers the reasoning behind the change. Our pull request description template covers the useful details.
What to demand from the tool
When choosing or re-evaluating a reviewer, check four things:
- Context: does it gather repository context beyond the diff before judging?
- Verification: are candidate findings checked against the code before posting or is the first draft the final answer?
- Thresholds: can you control severity and how much style commentary is posted?
- Teachability: can you add team-specific rules without a config language, and does dismissal feedback go anywhere?
A tool can miss one of these and still help. Missing all four is how you get three comments on a README.
The payoff is trust
The goal is a bot whose comments are worth reading. When the signal is strong, developers can act on findings quickly instead of sorting through filler.
If you want to see what a verification-first reviewer feels like on your own pull requests, Scopy's quickstart takes a few minutes, and if the code staying on your infrastructure matters, it is open source and self-hostable.
Frequently asked questions
What is a false positive in AI code review? A finding that claims a problem which does not exist, or would never matter in practice. The subtler kind, technically-true-but-irrelevant nitpicks, is just as damaging, because it trains developers to ignore the reviewer.
What false-positive rate is acceptable? There is no universal number. Track action rate: how often does a comment lead to a code change or useful discussion? A low action rate means the tool needs tuning, better context or a higher reporting threshold.
Why does my AI reviewer flag things that are clearly fine? Usually one of three causes: it only saw the diff and missed the context that makes the code safe, nothing verified the model's first draft or it does not know a team-specific convention that you know from experience. All three are fixable: the first two by tool choice, the third by writing rules.
Can better prompts fix a noisy AI reviewer? Only partially. Prompts can tune tone and severity but they cannot give repository context the tool never collected, and they cannot substitute for a verification pass from a different model.
If you want to compare reviewers instead of tuning one by feel, use the AI code review quality scorecard. It separates useful findings, missed issues, investigation time and cost.