OR-Clarify scores agents on asking before they guess

OR-Clarify scores agents on whether they ask for missing details or guess, per the OR-Clarify paper. The benchmark runs each task through a simulated user. It grades slot recovery, when the agent stops asking, whether it made a silent assumption instead, and the cost of the back-and-forth. Most agent benchmarks only check the final answer, so a wrong guess and a right question score the same. This one splits them apart.

The paper's own method, InterOPT, works out which of the withheld details would change the answer, then decides whether to ask again or stop. It beat every baseline on exact slot recovery in the choice-based setting the researchers tested.

The paper doesn't say how InterOPT or the silent-assumptions score hold up outside that choice-based setting, or outside operations-research tasks. Whether it generalizes to a coding agent or a support bot turning a vague request into a spec is untested, not proven.