Soft Latent Thinking Drops the Vocabulary Head During Reasoning and Improves Pass@k While Cutting Per-Step Compute

arXiv 2608.31069·medium signalXBlueskyLinkedInCopy link

Standard decoding projects hidden states through a large vocabulary head every step, which is expensive and forces all reasoning into discrete tokens; Soft Latent Thinking swaps that head for a lightweight projector during reasoning, doing autoregressive rollout in embedding space so intermediate steps stay continuous. On DeepSeek-Qwen-1.5B and LLaMA-3.2-3B it improves pass@k at every k tested while reducing per-step compute in the chain of thought, and it posts the highest pass@32 among soft-thinking approaches. The claim worth noting for builders is that the token bottleneck, not the reasoning itself, was costing accuracy as well as latency.