Repo2Skill-Evo: Every One of 105 Release Transitions Invalidated Part of an Agent's Skill Set, and Frontier Agents Fix Only 29.9-69.7% of It

arXiv 2608.21964·high signalXBlueskyLinkedInCopy link

Agent skills capture repository-specific procedural knowledge (which APIs to call, which scripts to run), but the version specificity that makes them useful also makes them decay silently after a release. Across 57 real repositories and 105 selected V1-to-V2 release transitions, every single transition invalidated part of the V1 skill set, yet six frontier agents given the official patch reached only 29.9-69.7% avg@3 macro F1 on a patch-grounded removal metric balancing stale-content recall against over-editing. Externalizing knowledge into a skill file makes its decay invisible, which is a direct warning for anyone maintaining long-lived skill or rules files.