Evaluate agent trajectories, not final outputs — output-only evals miss 20–40% of failures

Confident AI·high signal

2026 agent-eval guidance shows agents judged only on final-output quality pass 20–40% more test cases than full trajectory evaluation reveals, because a model update can corrupt an early planning or tool-calling step that only surfaces downstream. The recommended loop: a golden dataset drawn from real failures, graders/judges calibrated against human reviewers, and a replay regression suite (~30 golden cases, under 5 minutes) wired as a CI gate that blocks merges when any metric drops below threshold. Builder move: add planning and tool-call assertions to your agent's CI, and never ship a change that doesn't clear the replay suite.