AI agents flag fake commands, then run them anyway

Agents identify forged tool calls, then execute them anyway, a fleet test of 46 endpoints from six vendors found, per Recognition Without Enforcement.

Average execution across 14,294 spoofed trials was only 1.21%. But the failures aren't spread evenly. The gap between the best and worst configuration in the same deployment window reached 47 percentage points.

The authors call it a recognition-enforcement gap. A model's activations linearly encode who a request claims to be from. The model will even say aloud that an instruction looks forged when asked directly. Some configurations execute the forged tool call anyway. The failures cluster in specific, repeatable setups rather than spreading evenly across trials.

Prompt-layer defenses didn't generalize across the six vendors tested. What worked sits outside the model. An external reference monitor combines authenticated source routing with capability-gated tool execution. Tested against forged, tampered, replayed and unsigned requests, it rejected all of them.

A related report on an offline evidence-bundle verifier for agent messaging reaches the same conclusion from a different angle. It checks the source outside the model, not inside it. Verification that lives in the model's judgment is a capability. It isn't a security boundary.