AI agents hallucinate tool calls at any model size

Agents built on tool calling have a blind spot: the model can invoke a tool that was never registered, or fill in arguments that violate the tool's own schema, and most safety layers never check for it. A new paper, arXiv 2609.19425, catalogs 322 of these hallucinations across ten hosted models using a five-class taxonomy. The headline result is blunt. Scale doesn't fix it, and a 675B model fabricates calls at roughly the same rate as a 7B one.

That matters because the standard defenses, tool-selection filters and policy layers, both assume the call already names something real. They judge whether a chosen tool is the right one, not whether it exists. A model that invents a tool sails past both checks.

The fabrications aren't spread evenly. The paper finds 34 hallucinations on unconstrained JSON surfaces against 3 on more rigid interfaces. Loose schemas give the model room to invent fields and tool names that were never defined. Extend the setup to MCP, where multiple servers get merged into one namespace, and the paper surfaces 154 more hallucinations specific to that merge, on top of the base 322.

The fix the paper proposes needs no retraining. Check registry membership and validate the call's signature before dispatch, and you catch the fabrication before it runs. That's a cheap, deterministic filter sitting in front of models that get no more reliable as they get bigger.

For anyone wiring tool calls into an agent, the takeaway is to stop trusting the model to only ever name real tools. Whatever it emits, verify it against the actual registry first, especially on JSON-typed arguments where the paper shows the failures cluster.