Default prefix caching changes an agent's trajectory on 36% of episodes at 16-bit, 75% at 4-bit

Every flaky quantized agent I've debugged, I blamed the quantization. A paper published September 4 says I've probably been blaming the wrong layer (arXiv 2609.04748).

The setup is clean. Model, decoding parameters, seed, request order and batch size all fixed. Requests issued serially, not batched. An 80-episode multi-turn tool-use workload run with prefix caching on and off, across two serving engines and four weight formats. Enabling the cache changed the agent's trajectory on 36.2% of episodes at 16-bit and 75.0% at 4-bit. With caching disabled, repeated execution was bit-identical in 800 of 800 episodes.

One server-level setting moved run-to-run divergence by 37.5 percentage points. That setting is on by default in most serving stacks, absent from the request payload, and never reset between runs. You cannot see it in your logs. It isn't in your trace. Your replay harness doesn't capture it because there's nothing in the request to capture. The cache state is deterministic in the sense that the server knows exactly what it's doing; it's invisible in the sense that nothing on your side of the wire records it.

The quantization interaction is the part I hadn't considered. At 4-bit the numerical margins are tight enough that a cached prefix versus a recomputed one lands on different tokens more than twice as often as at 16-bit. So the popular folk explanation, "4-bit quants are unreliable for agentic work," is partly a measurement of cache behavior wearing quantization's name. That does not mean 4-bit is fine. It means the experiment separating the two effects hadn't been run, and now it has.

What I'd change tomorrow: before you file a bug about nondeterministic agent behavior, before you swap models, before you turn reasoning effort up, disable prefix caching and rerun. If the flakiness vanishes, you have a serving-configuration problem and a reproducibility story to write down. If it doesn't, now you're debugging the model with a control in place.

This connects to something else in today's material. A hands-on comparison on r/LocalLLaMA found Qwen3.8-Flash-Next on "xhigh" reasoning effort failed to complete in about three hours a task it finished in 25 minutes on "medium" (r/LocalLLaMA). Same weights, same task, one knob. We are surrounded by settings that reorder outcomes and almost none of them are in the trace.