ToolMaze Shows Agent Fault-Tolerance Scales 3.66x Slower Than Task Execution When Tools Fail

HuggingFace Daily Papers·medium signal

Most tool-use benchmarks test idealized 'happy paths'; ToolMaze (arXiv 2606.05806) instead injects a 2x2 taxonomy of tool perturbations (explicit/implicit, transient/permanent) across DAG-structured tasks. Perturbation Recovery Rate plummets ~37% under implicit semantic failures because agents over-trust corrupted outputs, and complex topologies trap them in trial-and-error loops. Critically, dynamic-replanning ability improves 3.66x slower with model scale than basic task execution — meaning bigger models alone won't fix recovery, a key signal for anyone shipping production agents.