Learnability and Competition in High-Dimensional Multi-Component ICA

Independent Component Analysis (ICA) is a foundational tool for unsupervised representation learning, yet its high-dimensional theory remains largely limited to single-component recovery. We develop an asymptotically exact mean-field theory for multi-component online ICA, capturing the coupling from simultaneous learning and orthogonalization. In the high-dimensional limit, the overlap matrix between estimates and true components obeys a closed ODE system. This reveals an initialization-driven phase structure: a decoupled regime, where estimates align with distinct components and evolve nearly independently, and a competition regime, where overlapping initializations induce orthogonality-driven conflicts and delayed convergence. Explicit learnability and competition boundaries show larger higher-order moments and competition shrink the stable learning-rate window and induce a staircase in component recovery. Experiments on synthetic data and hyperspectral remote sensing data validate the predicted trajectories and phase behavior.