Fetching from the wire…
01
02
03
04
05
06
07
08
Public story · 2026-03-23 · source-backed
The most-cited AI productivity study is now scientifically dubious. METR's landmark 2025 study found AI tools caused a 19-20% slowdown for experienced open-source developers. Their February 2026 follow-up reverses the headline: developers are now estimated 18% faster (CI: -38% to +9%). But the methodology broke down entirely.
The problem: 30-50% of participants now refuse to submit study tasks without AI access, even when paid $50/hour to do so. They're self-selecting which tasks to work on, choosing not to submit tasks they "did not want to do without AI." This invalidates the clean random assignment the study depended on. METR acknowledged the data "gives an unreliable signal of the productivity effect" and announced a complete study redesign.
On r/programming, the community reaction (140+ points) focused on the structural implication: AI dependency has become so entrenched in professional dev workflows that researchers literally cannot get a clean control group anymore. The selection bias likely masks significant AI speedups on tasks developers actually choose to use it for.
For any team evaluating AI coding tool ROI based on the original METR study: stop citing it. The reversal isn't just a correction — it reveals that the measurement framework itself is broken because the intervention has already changed the population. You can't run an RCT on breathing.
Each link below shares sources, entities, or timing with this story.
Shared entity: METR / Same source domain / Shared topic / Earlier coverage
Both cover METR; reported by the same outlet (metr.org); overlapping topics (developer, metr, productivity, study, tool).
OpenAI acquired Roi / Shared entities / What happened next
Linked by a graph relationship (OpenAI acquired Roi); both cover February, Their; picks up the February thread on 2026-07-29.
Linked by a graph relationship (OpenAI acquired Roi); both cover ROI, Their; picks up the ROI thread on 2026-07-27.
Linked by a graph relationship (OpenAI acquired Roi); both cover February, Their; picks up the February thread on 2026-07-25.
OpenAI acquired Roi / Shared entity: METR / Same source domain / What happened next
Linked by a graph relationship (OpenAI acquired Roi); both cover METR; reported by the same outlet (metr.org).
Linked by a graph relationship (OpenAI acquired Roi); both cover METR; reported by the same outlet (metr.org).
OpenAI acquired Roi / Shared entity: February / Shared topic / Earlier coverage
Linked by a graph relationship (OpenAI acquired Roi); both cover February; overlapping topics (access, developer).
OpenAI acquired Roi / Shared entity: February / What happened next / Tension
Linked by a graph relationship (OpenAI acquired Roi); both cover February; picks up the February thread on 2026-08-04.