Skills
A third of agent patches that pass every functional test still violate the review constraints a human reviewer would have named
arXiv 2609.04167 introduces SWE-Gate, 303 repository-level repair instances across 75 open-source Python repos, where review constraints are mined from real pull request review comments and shipped as a second test lane separate from functional tests. Across four LLM backends under a common coding-agent scaffold, 221 of the 644 repairs that passed functional tests failed the review constraints. The actionable move is to encode your own recurring review comments as executable constraint tests alongside the functional suite, because a green test run is measurably not the acceptance signal it is treated as.
↳ Follow the thread