Tabular Foundation Models Beat Every Baseline on 316 Physical Equations but Cannot Represent Units

arXiv 2609.02766·low signalXBlueskyLinkedInCopy link

Tabular foundation models fill in tables the way language models fill in text, and tables are the format most physical measurement arrives in, so the authors ask what these models' priors actually contain — a well-posed question because TFMs are Bayesian by construction. Probing TabPFN-3, TabICLv2, TabDPT and Real-TabPFN-2.5 against six baselines on datasets sampled from 316 physical equations, in and out of domain, the TFMs dominate both out of the box and after tuning. The limitation is structural rather than a matter of more data: their prior can represent neither a noiseless mechanism nor physical units, so they interpolate physics without being usable as physical models.