Agents hallucinate tools that do not exist, a 675B model does it as often as a 7B one, and merging MCP servers adds new failure surfaces

arXiv 2609.19425·medium signalXBlueskyLinkedInCopy link

A Sep 16 arXiv paper isolates a failure mode most defenses skip entirely: the agent invokes a nonexistent tool or supplies arguments violating the declared schema, while tool-selection and policy defenses both assume the emitted call names a real tool. Across ten hosted models and two invocation surfaces the authors catalog 322 genuine hallucinations under a five-class taxonomy, find fabricated calls concentrate on unconstrained JSON surfaces (34 vs 3), and report that scale gives no protection, with a 675B model matching 7-8B models. Extending to MCP surfaces a further 154 hallucinations specific to merging multiple servers into one namespace, and the fix is a training-free 'Resolution Rung' that checks registry membership and signature validity before dispatch, released with a Hallucinated-Tools Benchmark.