Factor Imbalance and Plasticity Loss in Low-Rank Factorized Networks

Loss of plasticity is a critical challenge in continual and non-stationary learning, where neural networks gradually lose their ability to adapt to new data. We study this phenomenon in low-rank factorized neural networks. Across task-streaming, sample-incremental, and class-incremental learning, low-rank factorized MLPs adapt worse to later tasks or distributions than full-matrix counterparts. This degradation is not explained by the rank constraint alone: frozen-factor controls preserve the low-rank constraint while substantially changing the degradation pattern, implicating the coupled optimization of the two factors. We identify factor imbalance, measured by the Gram mismatch between the factors, as a factor-level diagnostic associated with plasticity loss. Finally, we show that directly regularizing this mismatch reduces imbalance and partially recovers plasticity.