Structure Abstraction and Generalization in a Hippocampal-Entorhinal Inspired World Model

Humans abstract experiences into structured representations to facilitate pattern inference and knowledge transfer. While the hippocampal-entorhinal (HPC-MEC) circuit is known to represent both spatial and conceptual spaces, the mechanisms for concurrently extracting abstract structures from continuous, high-dimensional dynamics remain poorly understood. We propose a brain-inspired hierarchical model that simultaneously infers latent transitions and constructs a predictive visual world model. Our architecture employs an inverse model for structural extraction alongside an HPC-MEC coupling model that dissociates relational structures (MEC) from integrated episodic scenes (HPC). Using rotation dynamics as a benchmark, we demonstrate the model's capacity for structural abstraction. By leveraging velocity-driven path integration, the framework enables robust prediction and structural reuse across diverse contexts, thereby achieving structural generalization. This work provides a novel computational framework for understanding how brain-inspired, self-supervised learning of world models facilitates the acquisition of reusable abstract knowledge.

Our paper asks how a learning system can discover the reusable “rules” behind what it sees, rather than memorizing each scene separately. Humans are good at this: after seeing one object rotate or one hand movement happen, we can often understand the same kind of change in a new situation. Inspired by brain regions involved in memory and navigation, especially the hippocampus and entorhinal cortex, we design a model that separates what is in a scene from how the scene is changing. The model learns from videos without needing action labels. It builds a memory-like representation of visual details, while also learning a more general representation of movement and change. This lets it predict future frames and reuse learned patterns in new settings, such as applying a motion learned from one object or scene to another. Our results suggest that ideas from brain circuits for memory and mental maps can help machine learning systems acquire more flexible, reusable knowledge from visual experience.