Three Studies, Same Answer: AI Coding Is Net-Negative After Rework

A new analysis from paddo.dev dropped today and it synthesizes something I've been feeling but couldn't prove. Three independent research efforts converge on the same uncomfortable conclusion: AI coding tools make developers feel faster while actually making them slower.

The numbers. METR (July 2025): developers report feeling 20% faster but measured 19% slower on real tasks. Lightrun's 2026 survey of 200 SRE leaders: 43% of AI-generated code needs production debugging after passing QA. AI pull requests contain 75% more logic errors, roughly 194 per 100 PRs versus the human baseline. CodeRabbit's analysis of 470 PRs: 2.74x more security vulnerabilities and 8x more performance problems in AI-authored code.

The core argument is about the denominator. Teams measure AI productivity by counting merged PRs and closed tickets. Nobody's tracking the bug-fix commits within 48 hours of merge, the 2-3 redeploy cycles per AI fix (Lightrun says 11% need 4-6 cycles), or the production debugging hours. You're measuring speed to the first merge. Not speed to stable.

DORA's data adds a wrinkle. They show +21% tasks completed with AI tools, but flat organizational delivery metrics and decreased stability. More output, same throughput, worse reliability. That's a treadmill.

Karpathy posted something relevant about a "growing gap" between people who tried free ChatGPT last year and formed their views, and people paying for Claude Code and Codex who see the capability slope. He's right that the gap exists. But these three studies suggest even the power users might be measuring the wrong thing.

I use Claude Code every single day. I ship faster with it. But I've also noticed I spend more time reviewing, testing, and fixing subtle issues than I did when I wrote everything by hand. The net is still positive for me, but I'm a senior engineer with 15+ years of pattern recognition. The data suggests that advantage doesn't extend to everyone.

If you're a team lead, start tracking three metrics today: bug-fix commits within 48 hours of merge, ratio of AI-authored PRs that spawn follow-up fixes, and time-to-stable for AI versus human PRs. Without the denominator, you're flying blind.