TraceSafe: Systematic Assessment of LLM Guardrails on Multi-Step Tool-Calling Trajectories

arXiv·high signal

TraceSafe introduces a benchmark that shifts safety evaluation from final LLM outputs to intermediate execution traces in agentic tool-calling workflows. The paper reveals that existing guardrails, designed for chatbot-style interactions, fail to detect unsafe actions embedded in multi-step agent trajectories — a critical blind spot as LLMs become autonomous agents with real-world tool access. Covers attack surfaces across cs.CR, cs.AI, cs.CL, cs.LG, and cs.SE.