Multi-Way Representation Alignment

The Platonic Representation Hypothesis suggests that independently trained neural networks converge to increasingly similar latent spaces. However, current strategies for mapping these representations are inherently pairwise, scaling quadratically with the number of models and failing to yield a consistent global reference. In this paper, we study the alignment of $M \ge 3$ models. We first adapt Generalized Procrustes Analysis (GPA) to construct a shared orthogonal universe that preserves the internal geometry essential for tasks like model stitching. We then show that strict isometric alignment is suboptimal for retrieval, where agreement-maximizing methods like Canonical Correlation Analysis (CCA) typically prevail. To bridge this gap, we finally propose Geometry-Corrected Procrustes Alignment (GCPA), which establishes a robust GPA-based universe followed by a post-hoc correction for directional mismatch. Extensive experiments demonstrate that GCPA consistently improves any-to-any retrieval while retaining a practical shared reference space.

Many AI models learn useful internal descriptions of the data they see. For example, one model may process images, another may process text, and another may process speech. Even when these models capture related information, their internal descriptions are not directly comparable. A common way to align models is to learn a map from one model’s space to another. With many models, this requires a separate map for each pair, which quickly becomes inefficient. It also gives no single shared reference for comparing all models together. We propose a way to align many models through one shared reference space. Each model is mapped once into this space, so different models can be compared through the same reference. We first build the shared space so that each model’s internal structure is preserved, and then add a small correction that improves agreement between matching examples. Across a range of tasks, we find that the shared space improves alignment and performance. Because each model only needs one map into the shared space, new models can also be added without rebuilding all pairwise maps.