Interpreting Physics in Video World Models

A long-standing question in physical reasoning is whether video models rely on factorized physical state variables, or on task-specific distributed representations. We present the first mechanistic interpretability study of physical variables inside large-scale video encoders, combining layerwise probing, subspace geometry, patch-level decoding, and targeted attention ablations to characterize where and how physical information is orga- nized. Across architectures, we identify a sharp intermediate-depth transition, the Physics Emergence Zone, at which physical variables become linearly accessible. Scalar speed and acceleration are available from early layers, whereas motion direction emerges only at the Physics Emergence Zone, mirroring the V1 to MT motion hierarchy in primate visual cortex. Direction is encoded as a circular high-dimensional population code: dozens of orthogonal probe dimensions must be steered jointly to change the decoded direction, orders of magnitude more than the low-dimensional steering interventions seen in language models. These findings argue against compact physics- engine state variables and support distributed, hierarchically-organized, “brain-like” representations that are nonetheless sufficient for making physical predictions.

We investigated how AI video models understand physics. Rather than storing simple variables like a traditional physics engine, they appear to build distributed, brain-like representations spread across many neurons. We found a specific “Physics Emergence Zone” in the middle of the network where physical understanding suddenly becomes accessible, with more complex concepts like motion direction emerging later than simpler concepts like speed. These findings suggest that video models reason about the physical world using hierarchical representations that resemble aspects of biological vision.