Understanding Feature Learning Dynamics in Isotropic Regularizers via BHEP Statistics

Self-supervised training of LeJEPA is governed by the SIGReg regularizer to ensure an isotropic Gaussian embedding distribution and prevent dimensional collapse. However, directly analyzing its learning dynamics remains mathematically challenging, due to its random projection and lack of closed-form expressions. In this work, we bridge this gap by reformulating the SIGReg objective into a tractable BHEP-type statistic that shares the identical target distribution. Leveraging this theoretical proxy, we derive exact closed-form ODEs governing the macroscopic eigenvalue learning dynamics. Based on this formulation, we characterize the feature learning dynamics through two distinct phase transitions: a rapid initial $\textit{Explosion Phase}$ followed by a self-stabilizing $\textit{Shrinking Phase}$. Crucially, we decouple the global driving force into a collapsing $\textit{engine}$ and a stiffening $\textit{brake}$. Our exact analytic analysis reveals an extreme asymmetric decay between these forces, mathematically proving how the early saturation of top features profoundly depletes the driving force for subsequent features. This mechanism explains the severe, non-linear stepwise delay observed during training, while ultimately guaranteeing stable convergence toward isotropy.