Mitigating Plasticity Loss through Architectural Design in Continual Learning

Neural networks for continual reinforcement learning (CRL) often suffer from plasticity loss, i.e., a progressive decline in their ability to learn new tasks arising from increased representational drift (churn) and Neural Tangent Kernel (NTK) rank collapse. Current methods mitigating this problem involve algorithmic interventions such as regularization, resets, and optimization schedules. Here, we propose InterpLayers, a lightweight architectural solution that combines a fixed, parameter-free reference pathway with a learnable projection pathway using input-dependent interpolation weights. This structure makes InterpLayers orthogonal to existing algorithmic solutions. We show through theoretical analysis that InterpLayers upper-bound the output variability, bound churn, and prevent a collapse of the NTK rank through continual non-zero rank contribution from the interpolation mechanism. Across different distributional shifts, including permutation, windowing, and expansion, InterpLayers outperform similar gated architectures and achieve similar performance as current state-of-the-art methods without the need for optimization-level intervention or the introduction of sensitive hyperparameters. Ablation studies highlight that these improvements are sustained when InterpLayers are combined with existing algorithmic methods for preventing plasticity loss. These results position InterpLayers as a simple, complementary solution for maintaining plasticity in CRL.

In continual learning, neural networks have to learn many tasks one after another. A common problem is plasticity loss: over time, the network gets worse at learning new tasks, although these tasks are not harder than the earlier ones. Most existing solutions try to solve this by changing the training process, for example by resetting parts of the network or adding extra constraints during learning. In this work, we introduce Interpolation Layers, a change to the network itself. Each Interpolation Layer has one stable pathway and one learning pathway, and the network learns how much it should rely on each of them. This makes it possible for the network to stay stable while still being able to adapt to new tasks. Additionally, these new layer types can be added to existing neural networks as replacements for their current layers. Across different continual reinforcement learning benchmarks, Interpolation Layers show strong performance over long sequences of tasks and perform similarly to leading existing solutions. Lastly, they can also be combined with current training-based solutions, showcasing that better network design and better training methods do not have to be competitive but can complement each other.