MNIST-PRO strips agentic perception back to MNIST and finds models stop exploring before they have seen everything

arXiv 2608.31022·low signalXBlueskyLinkedInCopy link

arXiv 2608.31022 (submitted 2026-08-31) converts MNIST digit recognition into a sequential glimpse-based search task with lookback constraints, deliberately removing the physical and control complexity that confounds embodied benchmarks. Ten multimodal models were evaluated across four memory representations (raw visual history, textual state, structured metric grid maps, a consolidated visual canvas); all do fine under full observability and degrade sharply under partial observability. The three named bottlenecks are integrating fragmented glimpses into a state, halting exploration early, and refusing to revise an early wrong belief even against contrary evidence.