Plasticity Activation via Polar Operator: A Plug-in Method for Balancing Stability and Plasticity

Continual learning (CL) seeks models that acquire new knowledge while avoiding catastrophic forgetting. However, many methods that mitigate forgetting constrain parameter updates and thereby reduce model plasticity. We revisit the singular value spectrum of gradients in representative CL methods and show that they commonly exhibit singular value collapse, where only a small subset of gradient directions drive parameter updates. Motivated by this observation, we propose **P**lasticity **A**ctivation via **P**olar **O**perator (PAPO), a plug-in that preserves the dominant directions that mitigate forgetting while activating previously suppressed directions to enhance plasticity. Concretely, PAPO modifies the gradient $\mathbf{G}$ as $\mathbf{G}\leftarrow \mathbf{G}+\lambda \cdot \operatorname{polar}(\mathbf{G})$, which uniformly increases near-zero singular values without changing the singular vectors. To avoid the cost of explicit singular value decomposition, we approximate the polar factor using the iteration-dependent Polar Express scheme, which relies only on matrix multiplications and additions. In our empirical evaluation on both vision and language benchmarks, incorporating PAPO yields consistent improvements. In particular, on MiniImageNet, integrating PAPO into ER, MAS, GPM and TRGP produces substantial accuracy gains of $9.01\%$, $4.76\%$, $8.90\%$ and $9.19\%$, respectively.

Artificial intelligence models that learn continuously tend to forget earlier skills when picking up new ones, known as catastrophic forgetting. Many existing solutions prevent forgetting by locking down most of the model’s knowledge, but this comes at a cost: the model becomes rigid and struggles to learn anything new. We discovered that these protective methods typically update the model using only a handful of internal directions, leaving many other directions completely idle. To fix this, we developed a simple add-on tool called PAPO that gently wakes up those sleeping directions. It does so by applying a smart mathematical adjustment to the learning signal, spreading the model’s ability to change more evenly without damaging the important directions that safeguard old memories. The adjustment can be computed quickly and does not require heavy, expensive calculations. When we plugged PAPO into several popular continual learning algorithms, it consistently boosted their ability to absorb new information.