Free Pause Tokens Add Thinking Compute Without Adding a Token, KV Cache or Latency

arXiv 2609.03807·medium signalXBlueskyLinkedInCopy link

A pause token normally buys a model extra compute by occupying an extra sequence position, which costs context length and KV cache. A free pause token carries that compute in a parallel prediction stream over a weight-shared backbone while riding an existing position, improving next-token prediction by 2-3 centinats on a 1B parameter model. Because it adds no position, inference costs no context length, no KV cache and essentially no latency, since the added flops are not the throughput bottleneck; the only real cost is training, held to as low as 1.14x an optimized pretraining pipeline while preserving most of the benefit, making this an isoflop, isoparameter and isotoken improvement over standard next-token training.