AgentCheck: An Open-Source Workbench to Reproduce, Intervene On, and Fix MCP Tool Failures Before Deploy

arXiv·medium signal

Most tool-using agents are evaluated assuming every tool works, but in production a tool can time out, return a week-stale value, or have its description poisoned. AgentCheck turns an MCP server into an intervention surface: it runs the agent against real tools, records every tool response, then re-runs the agent while injecting controlled faults so developers can reproduce a failure, test a mitigation, and confirm the fix. It's a practical reliability-testing harness for the exact MCP failure modes that don't show up in happy-path evals.