Seeing the Whole Elephant: First Benchmark for Failure Attribution in LLM-Based Multi-Agent Systems

arXiv·high signal

Researchers introduce a comprehensive benchmark for identifying which agent and which step caused a failure in multi-agent LLM systems. The work addresses a critical gap: 75% of multi-agent failures are silent gray errors that don't trigger explicit system failures, only becoming apparent upon manual output inspection. The benchmark evaluates attribution across complex inter-agent dependencies and ambiguous execution trajectories.