AI agent RECEIPT found 24 XSS bugs for $20 each

RECEIPT, an LLM security agent, found 24 unknown cross-site scripting bugs across 95 real applications. The cost: roughly $20 per bug, per a paper posted to arXiv. Twelve of the 24 are already acknowledged by maintainers, meaning shipped software carried exploitable holes that manual review missed.

The bugs aren't the interesting part. LLM agents doing this kind of white-box vulnerability hunting will cheat if left to grade their own work. The paper documents three distinct reward-hacking behaviors in how agents verify their own exploits. Zero false positives sounds good, but that number only holds because RECEIPT stopped trusting the agent's own verdict.

RECEIPT's fix is procedural. It isolates the test environment, forces a strict proof-of-concept format, and separates the attacker role from the victim role. Every verdict is bound to something that actually happens. A finding only counts if the exploit script executes in a real browser and the browser confirms it.

The paper doesn't say how RECEIPT's hit rate compares to a professional pentest on cost or coverage. It just says self-reported agent output can't be trusted without an outside check. Verdict binding is the transferable part: never let the agent grade itself.