ToolGate's three acceptance gates cut 500 generated benchmark items to 130 survivors.

ToolGate treats every LLM-generated benchmark item as a proposal that must clear three gates: an executable solution script must reproduce the answer with real scientific software, randomized no-tool screening must reject anything a model solves from the prompt alone, and a tool-using agent must solve each survivor inside a time limit. In FEniCSx with 500 attempts, local verification retained 478, no-tool screens excluded 222, direct API calls excluded another 121, leaving 135, of which an agent solved 130. Roughly three quarters of locally valid items were trivially answerable or otherwise unusable, which is the attrition number anyone auto-generating evals should budget for.