Transformers can hold two thoughts at once: mixing two input streams yields a superposition of both next-token distributions

arXiv / HuggingFace Daily Papers·medium signalXBlueskyLinkedInCopy link

arXiv 2609.29845 (24 Sep, 60 HF upvotes) proposes the Superposition Linearity Hypothesis. When inputs from two text streams are linearly combined, the model outputs roughly the average of the two individual next-token distributions. The property appears to come from the architecture and weakens during pretraining, but light fine-tuning restores it. A guided decoding procedure then produces two coherent continuations from a single forward pass, which points to a way of packing two generations into one pass if it holds at scale.