ComplexMCP: First Benchmark for LLM Agents in Dynamic, Interdependent Tool Environments

arXiv·high signal

ComplexMCP introduces a benchmark built on the Model Context Protocol with 300+ tools from 7 stateful sandboxes to evaluate agents in realistic conditions where tools are atomic, interdependent, and noisy. Current SOTA models achieve only 47.3% on interdependent tool chains, revealing a critical gap between isolated API calling and real commercial automation. This is the first MCP-native benchmark and directly relevant to anyone building tool-using agents.