Addressable Memory for Video World Models

We study visual persistence in autoregressive video world models: the Key-Value (KV) cache accumulates a growing visual memory, but once rollouts extend beyond the training horizon, the model can no longer reliably address stored content. A standard fix compresses the cache into a fixed-size memory that retains recent context while summarizing the distant past. However, we show compression alone cannot restore recall: past this horizon, temporal positional encodings go out-of-distribution, so the model cannot reliably retrieve stored history regardless of content. On the evaluated architecture, content compression with out-of-distribution positions yields results identical to a fixed-size sliding window over recent KV entries. We propose WorldTrace, a training-free framework that keeps compressed memory addressable by assigning each slot a fixed, in-distribution position relative to the current frame. With addressable memory, we explore two retention approaches: WorldTrace-Field (coherence-oriented) aggregates history in a rotation-invariant space, improving TempSSIM by +15.5% while reducing scene drift; WorldTrace-LandMark (recall-oriented) stores verbatim scene traces at detected boundaries. The recall-oriented variant sustains scene reconstruction over long rollouts, with stronger long-range recall and smooth temporal coherence.