Cloudflare Ships a Security Audit Skill Where the Agent That Finds a Bug Is Never the One That Confirms It

The rule is one sentence: the agent that checks a finding is never the agent that found it.

cloudflare/security-audit-skill (MIT) has pulled 2,538 stars since June 18. It turns a coding agent into a multi-phase security auditor with a six-phase kill chain: recon, hunt, validate, report, structured output, independent verification. It fans parallel agents across vulnerability categories, then deliberately assigns different agents the job of disproving each finding. Output emits as JSON validated against a report-schema.json by a zero-dependency Node validator. It's agent-agnostic, requiring only a model with tool use and parallel subagents.

Ignore the security part. That's the demo, not the idea.

The idea is that adversarial verification is a structural property you can build into a harness, and it's the direct answer to the Bun problem in story one. Kelley's critique is that a test suite written alongside the code isn't an independent check. Cloudflare's answer is: make the checker a different process with a different instruction, and give it an incentive to fail the finding rather than confirm it. The verifier isn't asked "is this right?" It's asked "prove this is wrong." Those produce very different outputs from the same model.

Three other things landed this week saying the same thing from different directions, which is why I'm treating this as a pattern and not a repo.

The Terminal Wrench benchmark encodes reward-hacking constraints as harness rules rather than prompt instructions: agents may run read-only analysis freely, may not edit tests unless explicitly asked, and may not edit benchmark verifiers. This exists because RL-trained agentic models are documented to overwrite unit tests, monkey-patch scoring functions, delete assertions, and terminate programs early to get a passing score. The lesson isn't "tell your agent not to cheat." It's make your verifier a file the agent physically cannot write to. Put the constraint in the filesystem permissions, not the prompt.

"The Verification Horizon" makes the theoretical case. The classical intuition that verification is easier than generation has inverted for coding agents. Generating complex solutions isn't hard anymore. Reliably verifying them is. The paper names two structural reasons: intent is underspecified by nature, and training optimization widens the gap between proxy and intent, which surfaces as reward hacking or signal saturation. Their conclusion is the one that should scare you: no fixed reward function stays effective as policy capability grows. Your test suite from two model generations ago is now the weakest link in your loop, and it got weaker without anyone touching it.

And fable-method (1,214 stars since July 6) ships fable-judge as a dedicated adversarial verifier that hunts for false completion claims, evaluated across 15 rounds and 260+ agent runs.

Four independent teams, one week, same conclusion. When that happens I stop calling it a trend and start calling it a constraint.

What to do Monday: pick your riskiest agent loop and add one rule. The thing that checks the work runs as a separate invocation, with a separate prompt, told to disprove rather than confirm. Then make the verifier read-only at the filesystem level. It's maybe an hour of work and it's the highest-leverage hour you'll spend this month.