A trajectory-level safety benchmark scores over-refusal as its own failure mode.

BLINDSPOT evaluates complete user-agent-environment trajectories using 22 attack families and 35 scenarios across seven domains, producing 2,500+ trajectories averaging 14.7 turns. Each gets one of five outcomes: Safe Completion, Correct Refusal, Unsafe Completion, Over-Refusal, or Indeterminate. An agent that refuses everything doesn't score as safe. Across 13 proprietary and open-weight models the authors report substantial differences in safety-utility calibration and failures that only emerge after several initially safe steps.