Multi-Way Representation Alignment
Preprint
Abstract
The Platonic Representation Hypothesis suggests that independently trained neural networks converge to increasingly similar latent spaces. However, current strategies for mapping these representations are inherently pairwise, scaling quadratically with the number of models and failing to yield a consistent global reference. In this paper, we study the alignment of $M \ge 3$ models. We first adapt Generalized Procrustes Analysis (GPA) to construct a shared orthogonal universe that preserves the internal geometry essential for tasks like model stitching. We then show that strict isometric alignment is suboptimal for retrieval, where agreement-maximizing methods like Canonical Correlation Analysis (CCA) typically prevail. To bridge this gap, we finally propose Geometry-Corrected Procrustes Alignment (GCPA), which establishes a robust GPA-based universe followed by a post-hoc correction for directional mismatch. Extensive experiments demonstrate that GCPA consistently improves any-to-any retrieval while retaining a practical shared reference space.
Keywords
All Related Versions
This publication has 11 references indexed in Scilit:
- Similarity of Neural Network Models: A Survey of Functional and Representational MeasuresACM Computing Surveys, 2025
- Latent Functional Maps: a spectral framework for representation alignment2024
- ASIF: Coupled Data Turns Unimodal Models to Multimodal without Training2023
- MASSIVE: A 1M-Example Multilingual Natural Language Understanding Dataset with 51 Typologically-Diverse Languages2023
- Towards Universal Backward-Compatible Representation Learning2022
- Ranking the information content of distance measuresPNAS Nexus, 2022
- Generalized Canonical Correlation Analysis: A Subspace Intersection ApproachIEEE Transactions on Signal Processing, 2021
- A Survey of Multi-View Representation LearningIEEE Transactions on Knowledge and Data Engineering, 2018
- Scalable and Flexible Multiview MAX-VAR Canonical Correlation AnalysisIEEE Transactions on Signal Processing, 2017
- Generalized canonical correlations and their applications to experimental dataJournal of Clinical Psychology, 1961