Skip to yearly menu bar Skip to main content


Poster

Position Paper: The Platonic Representation Hypothesis

Minyoung Huh · Brian Cheung · Tongzhou Wang · Phillip Isola


Abstract:

We argue that representations in AI models, particularly deep networks, are converging. First, we survey many examples of convergence in the literature: over time, in multiple domains, the ways different neural networks represent data are becoming more aligned. Next we demonstrate convergence across data modalities: as vision models and language models get larger, they measure distance between datapoints in a more and more alike way. We hypothesize that this convergence is driving toward a shared statistical model of reality, akin to Plato's concept of an ideal reality. We term such a representation the \textit{Platonic representation}, and discuss several of the possible selective pressures toward it. Finally, we discuss the implications of these trends and limitations and counterexamples to our analysis.

Live content is unavailable. Log in and register to view live content