Factor-Wise Expert Composition Improves How Discrete Diffusion Models Combine Pre-Trained Experts

arXiv·low signal

Discrete diffusion models can solve complex reasoning tasks via compositional generation — combining multiple pre-trained experts to generalize beyond any single expert's training data — and recent work added time-dependent mixing weights to better align the composed dynamics. But those methods work per-sample, treating each generated state monolithically. This paper moves from global to factor-wise expert composition, mixing experts at the level of individual factors rather than whole samples, a more granular approach to steering compositional generation.