Neuro-symbolic behavioral abstractions localize agent failures better than reading trajectories with an LLM judge

arXiv 2609.02371·medium signalXBlueskyLinkedInCopy link

AgentScope (arXiv 2609.02371, 2026-09-02) abstracts an agent's trajectory into structured representations and introduces neural invariants, declarative properties the behavior is supposed to satisfy, then uses LLM-guided reasoning over the structured form rather than the raw trace to pinpoint both the failing step and its type. The authors argue traditional software debugging techniques do not carry over to agent failures and that pure LLM-as-judge diagnosis is unreliable. It outperforms the current state of the art on fault localization and attribution accuracy on the public Who&When dataset and on AgentErrata, a more comprehensive dataset the authors built.