Video World Models Lose Their Memory Past the Training Horizon Because RoPE Offsets Fall Out of Distribution — WorldTrace Fixes It Without Retraining

arXiv 2608.07408·medium signalXBlueskyLinkedInCopy link

Interactive video world models use a growing KV cache as visual memory, but once rollouts extend beyond the training horizon the temporal RoPE offsets land outside the trained range and attention can no longer address stored content — and naive cache compression makes it worse by averaging incompatible positional phases. WorldTrace keeps compressed memory addressable by assigning each summary slot a distinct in-distribution virtual position, in two variants: Field for temporal coherence, Landmark for verbatim episodic scene traces at detected transitions. On the new LoopBench benchmark (can a compressed cache reconstruct a scene after a long detour?) Field improved temporal consistency +15.5% and Landmark episodic recall +19.5%, all training-free.