Towards Understanding Catastrophic Forgetting in Two-layer Convolutional Neural Networks

Continual learning (CL) focuses on the ability of models to learn sequentially from a stream of tasks. A major challenge in CL is catastrophic forgetting (CF). CF is a phenomenon where the model experiences significant performance degradation on previously learned tasks after training on new tasks. Although CF is commonly observed in convolutional neural networks (CNNs), the theoretical understanding about CF within CNNs remains limited. To fill the gap, we present a theoretical analysis of CF in a two-layer CNN. By employing a multi-view data model, we analyze the learning dynamics of different features throughout CL and derive theoretical insights. The findings are supported by empirical results from both simulated and real-world datasets.

When AI systems learn many tasks one after another, they often "forget" what they learned before — a problem known as catastrophic forgetting. While this is common in convolutional neural networks (CNNs), the underlying causes remain poorly understood.We develop a theoretical framework to study this phenomenon using a simplified CNN trained on multi-view data. By analyzing how the model updates its internal features over time, we show that gradient descent tends to learn features with stronger signals, even if they are less robust. As a result, more stable but weaker features may fail to be learned.We also show that features useful for earlier tasks can be inherently unrelated to the labels of later tasks. As the model adapts to new data, these task-specific features are naturally discarded — leading to forgetting. Our theoretical findings are supported by experiments on both synthetic and real-world datasets.