A weaker agent recovered 80% of a stronger proprietary agent's capability gap from black-box execution differences alone

arXiv 2609.07131·medium signalXBlueskyLinkedInCopy link

AgentLeak shows that stealing a proprietary agent's skill files does not transfer its capability, because the missing piece is procedural behavior the stronger agent realizes implicitly at execution time. The attack treats the skill execution gap itself as the leakage surface, diffing successful victim executions against failed attacker executions to identify capability-critical behaviors and folding them into attacker-side skills, with the attacker's model, harness and tools unchanged. Across 20 task scenarios and 600 instances it improved pass rates over 40% versus direct skill reuse and recovered more than 80% of the victim-attacker gap, which means locking down skill artifacts does not protect an agent whose executions are observable.