Fetching from the wire…
01
02
03
04
05
06
07
08
Public story · 2026-07-31 · high
The gap traces to floating-point precision: BF16 caching flips answers, FP32 shows zero mismatches across 200 tests.
Why now: The paper posted to arXiv as 2607.28495 in July 2026.
Researchers got different answers 166 times out of 200 when they replayed identical tokens through a Qwen2.5-derived model, per arXiv paper 2607.28495.
For anyone building prompt caching or session replay, that's the gap between what you saved and what the model does next.
The researchers ran a matched 200-item experiment. One session kept its key/value cache alive, a second rebuilt it by re-feeding identical integer tokens. At 16-bit floating point (BF16), the two paths diverged on 166 of 200 suffixes.
At 32-bit floating point (FP32), the divergence disappears. Zero mismatches, with a 95% Wilson upper bound of 1.88%. The paper doesn't say whether the effect holds outside this Qwen2.5-derived setup or scales to other model families.
To prove the cache itself was steering the output, the researchers ran a bidirectional transplant. They swapped all 48 key/value layers between a live session and its rebuilt twin. Every divergent continuation followed its cache donor, 24 of 24 at the primary checkpoint and 43 of 43 in a second, outcome-blind run.
The fix in this test is precision, not prompt design. Switching cached inference to FP32 removes the divergence, though the paper doesn't say what that costs in memory or latency at scale.
Each link below shares sources, entities, or timing with this story.
Shared entities / Same source domain / Earlier coverage
Both cover BF16, FP32; reported by the same outlet (arxiv.org); earlier BF16 coverage from 2026-07-26.
Shared entity: BF16 / Shared topic / Earlier coverage / Tension
Both cover BF16; overlapping topics (bf16, checkpoint); earlier BF16 coverage from 2026-05-10.
Shared entity: Qwen2 / Same source domain / Earlier coverage
Both cover Qwen2; reported by the same outlet (arxiv.org); earlier Qwen2 coverage from 2026-07-30.
Shared entity: BF16 / Shared topic / Earlier coverage
Both cover BF16; overlapping topics (bf16, token); earlier BF16 coverage from 2026-07-28.
Shared entity: Qwen2 / Same source domain / Earlier coverage
Both cover Qwen2; reported by the same outlet (arxiv.org); earlier Qwen2 coverage from 2026-07-26.
Both cover Qwen2; reported by the same outlet (arxiv.org); earlier Qwen2 coverage from 2026-07-25.
Both cover Qwen2; reported by the same outlet (arxiv.org); earlier Qwen2 coverage from 2026-07-20.
Both cover Qwen2; reported by the same outlet (arxiv.org); earlier Qwen2 coverage from 2026-06-27.