FiGuRO: Intrinsic Dimension Estimation for Multi-Modal Data

Determining the complexity, or Intrinsic Dimension (ID), of data is fundamental to efficient and interpretable representation learning. This is particularly challenging in multi-modal settings when trying to learn disentangled representations for shared and private information. Existing techniques leave a critical gap: they are often static, uni-modal, or in the case of contrastive methods, adapt only to the shared ID implicitly. We introduce Fidelity-Guided Rank Optimization (FiGuRO), a framework for approximating the ID of uni- and multi-modal data under constraints of model capacity and hyperparameters. FiGuRO learns the dimensions of low-rank projections using truncated singular value decomposition and an algorithm that determines when to reduce or increase dimension and in which latent space. Disentanglement of shared and private information arises as an emergent property of this optimization, eliminating the need for complex auxiliary loss functions. We demonstrate that FiGuRO outperforms existing ID estimation techniques and is more robust to hyperparameter changes. Across simulations and real-world data, FiGuRO captures distinct ID scales and varying subspace ratios, and decomposes shared and private information successfully. Furthermore, we show that FiGuRO can be applied to modern uni-modal pretrained models, enabling efficient, post-hoc disentanglement of multi-modal representations.

Modern AI models often learn from multiple sources (modalities) simultaneously, such as combining a patient's medical scans with their genetic data. To make these models efficient and trustworthy, scientists must find their true underlying complexity (the absolute minimum number of factors needed to describe the data). However, existing tools struggle to differentiate between information shared across sources and details unique to just one. To solve this, we developed a framework called FiGuRO that automatically estimates the complexity of both shared and unique information streams in a single step. The central algorithm expands or shrinks FiGuRO's representations based on how perfectly it can reconstruct the original data. As it balances this process, FiGuRO untangles shared concepts from modality-specific details. We demonstrate FiGuRO's success on diverse datasets, including paired audio-image digits and complex biological measurements. By revealing precisely how much new information each source contributes, FiGuRO helps researchers determine if collecting difficult or expensive data is truly worthwhile. Ultimately, this makes multi-source AI systems more lightweight and transparent.