Test-Time Budget Allocation Gains in Neural Combinatorial Optimization Are Mostly Sampling Luck Measured In-Sample

arXiv 2608.13087·low signalXBlueskyLinkedInCopy link

arXiv 2608.13087 (2026-08-13) audits its own measurement and finds that on in-distribution TSP-100, an oracle budget allocation computed and evaluated on the same stored samples reports a 2.2-2.6% gain with intervals excluding zero across POMO, AM and SymNCO — but measured out of sample the gain is indistinguishable from zero (0.457%, 0.015%, -0.512%). The bias does not shrink with more samples or more instances. Applying the same correction under distribution shift preserves a real 11.5% (AM) and 12.0% (SymNCO) improvement in a pre-registered confirmatory experiment, with a pre-registered negative control on the shift-robust POMO showing -0.3%. Data, code and the pre-registration record are released; the reporting checklist generalizes well beyond NCO.