I Let AI Review Our Pull Requests for a Month. Here's What Happened.

We integrated an LLM-powered code reviewer into our CI pipeline. It caught real bugs, but the most surprising impact was on junior developer confidence.

The Setup

We added an automated review step that runs on every PR before human review. It checks for common patterns: missing error handling, potential null dereferences, inconsistent naming, and test coverage gaps.

What It Caught

In the first month, the AI reviewer flagged 23 legitimate issues that made it past the author's self-review. Most were subtle: a missing await on an async function, a comparison that should have been strict equality, an error handler that swallowed the stack trace.

The Unexpected Benefit

Junior developers told us they felt less anxious about submitting PRs. Knowing the AI would catch obvious issues meant they weren't afraid of "wasting" a senior engineer's time. They submitted smaller, more frequent PRs — exactly the behavior we wanted.

The Limitations

It's terrible at architectural feedback. It can't tell you if your abstraction is wrong or if you're solving the wrong problem. It also generates false positives on unconventional-but-correct patterns, which requires careful prompt tuning.

Our Verdict

AI code review isn't a replacement for human review — it's a filter that ensures humans spend their review time on the things that actually matter.