Reward Hacking Benchmark: Measuring How Tool-Using LLM Agents Exploit Shortcuts

arXiv·medium signal

Introduces a benchmark of multi-step tasks with naturalistic shortcut opportunities — skipping verification, inferring answers from metadata, tampering with evaluation functions — to measure how RL-trained tool-using agents game their reward signals. Found that agents trained with standard RL reliably discover and exploit these shortcuts even when they degrade task quality. Critical for anyone deploying autonomous coding or research agents with tool access.