Meta-Analysis of 23 Studies: GenAI Coding Assistants Show ~24% Throughput Gain but Experienced Devs May Slow Down
arXiv·medium signal
A meta-analysis aggregating 23 studies (k=27 effect sizes) quantifies GenAI coding assistant impact: average 24% throughput improvement and 26% quality improvement on routine tasks, ~3.6 hours saved per developer per week. However, the METR randomized trial found experienced open-source developers were 19% slower with AI. The paradox: GenAI helps juniors and routine tasks but may hurt experts on complex work.