Symmetries in language statistics shape the geometry of model representations

Although learned representations underlie neural networks' success, their fundamental properties remain poorly understood. A striking example is the emergence of simple geometric structures in LLM representations: for example, calendar months organize into a circle, years form a one-dimensional manifold, and the latitude and longitude of cities can be decoded by low-dimensional linear probes. We show that the statistics of language exhibit a translation symmetry---e.g,. the co-occurrence probability of two months depends only on the time interval between them---and we prove that the latter governs the aforementioned geometric structures in high-dimensional word embedding models. Moreover, we find that these structures persist even when the co-occurrence statistics are strongly perturbed (for example, by removing all sentences in which two months appear together) and at moderate embedding dimension. We show that this robustness naturally emerges if the co-occurrence statistics are collectively controlled by an underlying continuous latent variable. We empirically validate this theoretical framework in word embedding models, text embedding models, and large language models.

How do language models "think about" continuous concepts like calendar time and geographic space? Previous research has shown that these models automatically learn to internally represent timelines and maps as wiggly lines and sheets, though we did not understand why. Furthermore, this geometry appears important for reasoning about the ordering of events ("What day comes 4 days after Thursday?") and the relative locations of places ("Is San Francisco to the north of Los Angeles?"), though we did not understand exactly how. In this paper, we give a theory that mathematically predicts the wiggly geometry from first principles, showing that it follows from simple properties of the training data. We also explain how models likely exploit this wiggly geometry for computations. Our work is therefore a useful step towards understanding the origin and characteristics of intelligent behavior in language models.