Upgrade LLM-as-judge to 'agent-as-judge' — score the trajectory, not just the final answer

Confident AI·medium signal

A 2026 evolution in agent evaluation is the agent-as-judge: the evaluator is itself an agent that inspects intermediate steps and tool calls, not only the final text, catching agents that reach the right answer through wrong or unsafe paths. Pair it with the cost-calibration split — run the LLM judge on 100% of production traffic cheaply, then verify a 1-2% human sample for agreement. For builders shipping tool-using agents, trace-based judging surfaces failure modes that output-only scoring misses entirely.