Dual-Layer Agent Evaluation: Trajectory Metrics for Process Debugging + Outcome Metrics for Business Validation

Galileo·high signal

Production agent evaluation requires two separate measurement layers: outcome metrics (task success rate, correctness, cost) that validate business goals, and trajectory metrics (step precision/recall, exact step matching, in-order matching) that capture every reasoning step and tool call to explain why an agent succeeded or failed. Relying solely on outcome metrics is the primary cause of low prototype-to-production translation — passing outcomes can mask flawed trajectories that will break on slight input variation. Google Vertex AI provides trajectory_exact_match, trajectory_precision, and trajectory_recall primitives that pair directly with task success rate for production agent monitoring.