LimiX-2 pretrains on synthetic structural causal models and recovers causal skeletons from attention

arXiv / HuggingFace Daily Papers·medium signalXBlueskyLinkedInCopy link

arXiv 2609.17488, submitted 2026-09-15 by Xingxuan Zhang and 59 co-authors, proposes Contextual Mechanism Networks that learn the joint p(x, y | D_context) rather than only predicting y from x. Pretraining uses Context-Conditional Masked Modeling over synthetic datasets generated from structural causal models with varying graph structures and functional mechanisms. It reports beating both dataset-specific models and existing tabular foundation models on TabArena, TALENT and BCCO, and the notable side effect is that feature attention encodes direct causal relationships well enough to recover causal skeletons.