Marionette Fixes Game World-Model Drift by Repairing an Explicit 276-Dimensional State Instead of Retraining the Generator

arXiv 2608.14530·medium signalXBlueskyLinkedInCopy link

Rather than autoregressing pixels or latents, Marionette predicts an interpretable 276-dimensional 3D world state (articulated skeletons, metric root trajectories, rotations), hands geometry and occlusion to a zero-parameter renderer, and lets a video-diffusion model synthesize only appearance. The payoff is that long-horizon failures become editable: left free, two generated characters drift to 21.2 m apart against ~5 m in recorded sessions with a third of frames showing ground penetration, and two rules imposed on the state — a terrain collider and a separation cap — cut penetration 66% with no change to the observation model. Routing appearance through the predicted state costs almost nothing in fidelity (FVD 831 vs 799 for recorded pose).