Energy-Structured Low-Rank Adaptation for Continual Learning

While orthogonal subspace methods try to mitigate task interference in Continual Learning (CL), they often suffer from energy diffusion across the basis, hindering knowledge compaction and exhausting capacity for future tasks. We observe that output feature drift induced by parameter updates is inherently low-rank, and theoretically prove that preserving parameters along the principal directions of this drift minimizes the output reconstruction error. Motivated by this, we propose **E**nergy-Concentrated and **E**nergy-Ordered **Lo**w-**R**ank **A**daptation (E$^2$-LoRA). By explicitly ordering and concentrating knowledge into leading ranks, E$^2$-LoRA frees capacity for subsequent tasks. Furthermore, we design a dynamic rank allocation strategy to balance stability and plasticity by jointly optimizing energy retention and model plasticity. Extensive experiments across multiple benchmarks demonstrate that E$^2$-LoRA achieves state-of-the-art performance.

Problem. When AI models learn new tasks one after another, they often overwrite previously learned knowledge—a phenomenon known as catastrophic forgetting. Solution. We designed E$^2$-LoRA, a method that automatically reorganizes what the model has learned into a compact, prioritized structure: the most important knowledge stays at the front, while less critical information is set aside, freeing up space for future tasks without interfering with earlier ones. Impact. This enables AI systems to keep learning continuously across hundreds of new tasks without forgetting old ones, making them more suitable for real‑world applications that need to adapt over time.