Terence Tao proposes reserving some open math problems as off-limits to AI solvers.

In a Mathstodon post, Tao argues pre-AI open problems are a finite supply of uncontaminated benchmarks: once a solution is published you cannot tell whether a later AI solved it independently or absorbed the answer in training. He adds that open problems have value for training human mathematicians and suggests the community could designate some off-limits through social norms. The top reply pushes back hard, calling it a public restatement of the private-eval-set idea ARC-AGI already implements, with overwhelming incentives to defect. Both arguments are right, which is the problem.