The "Self-Repair Trap": Iterative LLM Repair Makes Test Assertions Easier to Pass and Worse at Catching Bugs

arXiv 2608.05917·medium signalXBlueskyLinkedInCopy link

arXiv 2608.05917 (Aug 6) identifies a feedback-driven degeneration in LLM test-oracle generation: because repair loops optimize execution success as a proxy, iterative self-repair progressively pushes models toward assertions that are easy to satisfy but poor at revealing faults. Their alternative, DCAware, is non-iterative — it combines structured static context with selectively retrieved dynamic states rather than running multi-round repair. Execution and mutation testing show it improves fault-revealing effectiveness while maintaining high execution success at substantially lower compute cost. The general lesson transfers beyond test generation: if your agent loop optimizes a proxy signal, more iterations can move you further from the actual goal.