Temporal Context Reinstatement Drives Episodic-Like Order Memory in Long-Context Language Models

Human episodic memory supports the retrieval of experiences that unfold over extended timescales, yet the computational mechanisms underlying this ability remain debated due to the difficulty of mechanistic accessibility in long-term memory experiments in humans. Long-context LLMs may offer promising ways to reveal plausible computational mechanisms that drive this type of retrieval. Here, we investigate whether and, if so, how LLMs capture core behavioral signatures of humans of a central aspect of episodic memory via a temporal order memory task. Using a new dataset of human behavior based on a full-length novel, we show substantial similarities between the human and model performances on the temporal order memory task. We next perform long-context mechanistic interpretability analyses to reveal the underlying mechanisms in the model, and find that model performance relies on a one-dimensional temporal code that is reinstated during retrieval by a single time-reinstatement attention head. These findings support temporal context reinstatement as an important mechanism for episodic-like temporal-order memory in LLMs, offering new insights into how temporal aspects of long-term episodic memory may be instantiated in both artificial and biological systems.