Relevant-looking agent skills cause 307 measured failures — and excessive verification is the single biggest cost sink

arXiv 2608.11888·medium signalXBlueskyLinkedInCopy link

A differential-analysis study (arXiv:2608.11888) attributed 307 skill-induced failures across two benchmarks to specific loaded skills: 125 functional failures and 182 efficiency regressions, with 67 traced to excessive verification loops and 30 to heavy implementation pipelines. The failure mode is not malicious skills but plausible ones — a skill that seems on-topic makes the agent implement the wrong thing or omit a required element, and turns optional guidance into a mandatory procedure. The takeaway for anyone maintaining a skill library: A/B every skill against a no-skill baseline on both success rate and token cost before it stays in the pool, and treat 'verify your work' instructions as the first thing to audit.