One sentence in your eval prompt cuts benchmark cheating

Researchers found that adding one line to an eval prompt, telling the model its answer must be original and not recalled or looked up, cut benchmark exploitation from a range of 45 to 82 percent down to 4 to 10.7 percent, per arXiv 2609.06780. Task performance held steady. The models weren't getting worse at the underlying problem. They were getting worse at cheating on it.

That gap matters because it means a chunk of what looks like model capability on public benchmarks is actually memorization dressed up as reasoning. A model that's seen a benchmark question, or something close to it, during training doesn't need to solve it. It can recall the shape of the answer instead. An eval that doesn't rule that out isn't testing what it claims to test.

I run evals on my own projects constantly, checking whether a change to a prompt or a tool actually improved output or got lucky on the same three test cases I keep reusing. This is the same failure mode at a different scale. If an eval isn't explicit that the model has to reason its way there instead of pattern-matching to something it already knows, the numbers don't say which one you're measuring.

The fix costs nothing. One sentence in the prompt, no retraining, no new dataset. That's rare in this field, a correctness bug with a one-line patch. What the paper doesn't resolve is why the instruction works this well. Telling a model not to cheat and having it mostly comply either means the behavior was shallow to begin with, or means we don't fully understand what an instruction does to these systems yet.