MoCL: Metabolic Optimization for Curvature-Aware Continual Learning

Continual learning requires models to mitigate catastrophic forgetting of prior knowledge while learning a sequence of tasks. Although existing methods based on orthogonal projection prevent interference by constraining parameter updates, they tend to limit plasticity as the task sequence progresses. The reliance on the linear approximation further causes the projected gradients to deviate from the nonlinear manifold. To address these issues, we propose Metabolic Optimization for Continual Learning (MoCL), a rehearsal-free framework that strikes a balance between stability and plasticity. To capture the geometric manifold of prior knowledge, MoCL introduces a factorized subspace approximation that avoids expensive explicit matrix inversion. Given the heavy-tailed distribution of the Fisher Information Matrix, we employ a metabolic gating based on Tsallis entropy to suppress updates that conflict with historical knowledge. Theoretical and empirical analyses show that MoCL suppresses interference while supporting shared low-loss behavior across sequential tasks. Extensive experimental results across multiple benchmarks demonstrate that MoCL outperforms state-of-the-art methods in both classification performance and efficiency.

Modern AI systems are often trained to solve one task at a time. However, when they learn a new task, they may forget what they learned before. This problem is known as catastrophic forgetting and is a major challenge for building AI systems that can keep learning over time. Our work proposes MoCL, a new training method that helps a model protect important old knowledge while still allowing it to learn new information. Instead of strictly blocking many model updates, MoCL softly controls the learning process by identifying which directions are important for previous tasks and reducing harmful changes in those directions. This makes learning more flexible and efficient. Experiments on several image classification benchmarks show that MoCL achieves better accuracy than existing continual learning methods while using less memory and computation. These results suggest that MoCL can help make continual learning models more practical for real-world applications where data and tasks change over time.