Default prefix caching changes an agent's trajectory on 36% of episodes at 16-bit and 75% at 4-bit, with everything else held fixed

arXiv 2609.04748·high signalXBlueskyLinkedInCopy link

With model, decoding parameters, seed, request order and batch size all fixed and requests issued serially, an 80-episode multi-turn tool-use workload was run with prefix caching on and off across two engines and four weight formats. Enabling the cache changed the agent's trajectory on 36.2% of episodes at 16-bit and 75.0% at 4-bit; with caching disabled, repeated execution was bit-identical in 800 of 800 episodes. The cause is that cache state is deterministic but invisible: it is absent from the request and never reset by default, and one server-level prompt-cache setting moved run-to-run divergence by 37.5 percentage points, so anyone debugging a flaky quantized agent should disable prefix caching before blaming the model.