Successful LLM Bug Fixes Show Diffuse Attention Across Stacktraces and Tests; Failures Over-Focus on Version Metadata
The first empirical study of attention patterns in LLM-based automated program repair (arXiv 2607.25873, 2026-07-28) analyzes 319 real Python and Java bugs from SWE-bench Verified and Multi-SWE-bench to ask why a model patches one bug correctly and fails a closely related one given identical context. Successful repairs are characterized by attention diffused across multiple diagnostic components — bug descriptions, stacktraces, test cases — while failures exhibit over-localized attention on metadata such as version information. Stronger alignment between model attention and the sections and phrases developers identify as important correlates with higher repair success, giving the first empirical evidence that attention misallocation, not just capability, drives APR failures.
↳ Follow the thread