KV Cache, Not Token Identity, Carries Reasoning Divergence: BF16 Replay Differs on 166 of 200 Suffixes, FP32 on Zero

arXiv·medium signalXBlueskyLinkedInCopy link

This audit (arXiv 2607.28495, July 30) tests the common assumption that replaying identical token prefixes reconstructs the decoder state that originally produced them. On a Qwen2.5-derived system with a matched 200-item experiment, retained live cache versus one-shot prefill of identical integer tokens diverge on 166 suffixes and 20 correctness labels in BF16, though the accuracy difference is only one point. A fixed-prefix 2x2 crossing construction and precision reproduces the BF16 disagreements while FP32 produces none (95% Wilson upper bound 1.88%). Bidirectional transplantation of all 48 key/value layers makes every tested divergent continuation follow its cache donor — 24/24 at the primary checkpoint and 43/43 in an outcome-blind replication — establishing boundary K/V cache as the causally sufficient carrier.