An Agent Can Name the Right Broken Microservice and Still Have No Idea Why, Across 3,500 Diagnostic Trajectories

arXiv (2608.21310, submitted 2026-08-21)·medium signalXBlueskyLinkedInCopy link

Existing root cause analysis benchmarks score only endpoint correctness, whether the agent localized the responsible service, which tells an on-call SRE nothing about the evidence behind the call or the fault-propagation route. This trajectory-level framework evaluates 3,500 agent executions on a public microservice RCA benchmark against manually curated service-level propagation paths and finds a real disconnect: an agent can localize the fault source yet fail to reconstruct how it propagated. Successful investigations stayed on the fault-impact surface, acted on retrieved evidence, and broadened their query repertoire as the search deepened, while failures came from omitted decisive evidence, misread telemetry, or unsupported inference filling in for missing evidence; the taxonomy is packaged as DiagGuard.