Code Review for Claude Code

https://gist.github.com/rlueder/a3e7b1eb40d90c29f587a4a8cb7c...

An average of $0.04/review (200+ PRs with two rounds each approx.) total of $19.50 using Opus 4.6 over February.

It fills in a gap of working on a solo project and not having another set of eyes to look at changes.

So the take would be that 84% heavily Claude driven PRs are riddled with ~7.5 issues worthy bugs.

Not a great ad of agent based development quality.

Product A: "AI looks at your diff and leaves comments." This is getting commoditized fast. GitHub Copilot does it, Anthropic just launched it, and every startup in the space does some version of it. The value ceiling is low because it's fundamentally a linting++ problem with better language models.

Product B: "AI-native governance for when AI agents write production code." This is a different problem. When 41% of commits are AI-assisted and Dependabot, Renovate, Copilot, Cursor, Claude Code, and Devin are all opening PRs on the same repo, the question isn't "are there bugs in this diff." It's: what's the risk profile of this change? Which agent wrote it and do we trust that agent? Should this auto-merge or need human review? What's the approval workflow?

The bubble is in Product A. Product B barely exists yet, and it's where the actual value is for engineering teams trying to ship at AI speed without losing governance. The Jane Street example in this thread is interesting - their model works because they have an extremely homogeneous engineering culture. Most teams don't. They need policy-driven automation that encodes their standards.

If you're a big shop pushing, say, 2,000 PRs a week and reviews average $15–25, that’s on the order of $30k–$50k a week in AI review spend, or $1.5-2.5M a year. That is quite a line item to justify.

"It's $20 cheaper than a senior engineer’s hourly rate,"... so what are you actually doing with your human reviewers once you add this on?

If you keep your existing review culture and just bolt this on, then you've effectively said "we’re willing to add $1–2M+ a year to the budget." That might be fine, but then you should be able to point to fewer incidents, shorter lead times, higher coverage, something like that.

Either this is a replacement story (fewer humans, different risk profile) or it's an augmentation story (same humans, bigger bill, hopefully better outcomes). "It’s cheaper than a great engineer" by itself skips over the fact that, at scale, you’re stacking this cost on top of the engineers you already have in the org.

* https://github.com/anthropics/claude-plugins-official/tree/m...

* https://github.com/anthropics/claude-plugins-official/tree/m...

It's totally worth it.

If it takes 17 rounds of review from 5 different models/harnesses – I don't care. Just spit out the right code the first time. Otherwise I'm wasting my time clicking "review this" over and over until the PR is worth actually having a human look at.

So part of the workflow becomes filtering signal vs noise.

You've got to be completely insane to use AI coding tools at this point.

This is the subsidised cost to get users to use it, it could trivially end up ten times this amount. Plus, you've got the ultimate perverse incentive where the company that is selling you the model time to create the PRs is also selling you the review of the same PR.