SWE-CI Benchmark: 75% of AI Coding Agents Break Working Code Over Time — Only Claude Opus Stays Above 50% Zero-Regression Across 233-Day Maintenance Simulations

arXiv·high signal

Alibaba and Sun Yat-sen researchers released SWE-CI, the first benchmark evaluating AI agents on long-horizon codebase maintenance via continuous integration. 100 tasks spanning 233 days and 71 consecutive commits per repo on average. Introduces EvoScore to measure whether agent changes support future code advancement. 18 models tested; most accumulate technical debt and break previously working code. Claude Opus and GLM-5 are the only strong performers.