ORAgentBench: Can LLM Agents Solve Challenging Operations Research Tasks End to End?

arXiv·medium signal

An execution-grounded benchmark of 107 human-reviewed operations-research tasks scoring agents on code execution, schema validity, constraint feasibility, and solution quality. The best agent passes only 35.51% of tasks and 20.59% of hard tasks, with failures driven by missed operational rules, brittle formulations, and weak refinement. A sober reality check on autonomous agents for structured optimization work.