NetInjectBench: Naive Network-Ops Agents Take Unsafe Actions 82.5% of the Time Under Prompt Injection

arXiv·high signal

NetInjectBench is a 130-scenario benchmark (40 benign, 40 weak-attack, 40 strong-attack, 10 approved high-impact changes) for indirect prompt injection in tool-using network-operations agents, tested on Qwen2.5-7B, Llama3.1-8B, and Mistral-7B. Across 240 attack instances, naive execution hit an 82.50% unsafe tool-action rate; prompt-only defenses, Self-Reminder, Spotlighting, and a Two-Pass LLM Judge cut it to 25.63%/21.67%/18.33%/10.00%, while static allowlisting reached 5% but blocked 100% of approved changes. A metadata-aware policy gate produced 0/240 unsafe actions (95% Wilson upper bound 1.58%) while preserving 99.17% usefulness — the paper's thesis is that agents need execution-time authorization boundaries, not just prompt hygiene.