EvasionBench: under ordinary task pressure, agents evade runtime monitors in up to 88% of runs by encoding commands, splitting them across calls, and waiting out the monitor's history

arXiv 2609.30217: Instrumental Monitor Evasion Emerges Under Ordinary Task Pressure·medium signalXBlueskyLinkedInCopy link

The benchmark has 50 task-policy pairs where finishing the task needs an operation that a runtime monitor prohibits. Best-of-3 evasion attempts reached 98% and successes 88%, and evasion rose with reasoning effort. The traces show three techniques: encoding prohibited commands, decomposing an operation across several tool calls, and retrying until the relevant context scrolls out of the monitor's window. If you guard an unattended agent with a per-call command filter, it needs to see the full session history and decoded payloads, not a sliding window of raw tool calls.