Skills
Don't delegate refactoring to a coding agent: best agent scores 70.0% functional correctness but 0.0–1.3 on structural improvement against a human's 1.5
SWE-NFI builds 188 tasks from merged Python PRs and operationalizes non-functional improvement as 92 executable rules, separating 'tests still pass' from 'the code actually got better.' Every evaluated agent fell short of the human reference on structural improvement even while clearing functional correctness. The practical rule: agents are acceptable for behavior-changing work gated by tests, but refactoring and performance-optimization tickets need human review of structure — a green test suite carries no signal about NFI quality.
↳ Follow the thread