Self-Distillation Enables Continual Learning

Continual learning, enabling models to acquire new skills and knowledge without degrading existing capabilities, remains a fundamental challenge for foundation models. While on-policy reinforcement learning can reduce forgetting, it requires explicit reward functions that are often unavailable. Learning from expert demonstrations, the primary alternative, is dominated by supervised fine-tuning (SFT), which is inherently off-policy. We introduce Self-Distillation Fine-Tuning (SDFT), a simple method that enables on-policy learning directly from demonstrations. SDFT leverages in-context learning by using a demonstration-conditioned model as its own teacher, generating on-policy training signals that preserve prior capabilities while acquiring new skills. Across skill learning and knowledge acquisition tasks, SDFT consistently outperforms SFT, achieving higher new-task accuracy while substantially reducing catastrophic forgetting. In sequential learning experiments, SDFT enables a single model to accumulate multiple skills over time without performance regression, establishing on-policy distillation as a practical path to continual learning from demonstrations.

AI models need to learn new skills and knowledge over time without losing what they already know. This remains a major challenge. While some learning methods can reduce forgetting, they often require clear reward signals, which are difficult or impossible to provide in many real-world settings. The most common alternative is to train models directly on expert examples, but this can cause the model to forget previous abilities. We introduce Self-Distillation Fine-Tuning, a simple method that lets models learn from examples while better preserving what they already know. Our method uses the model’s ability to learn from examples shown in context: the model first looks at an expert demonstration, and this example-guided version of the model then acts as a teacher for the original model. This creates training signals that help the model acquire the new skill without drifting too far from its previous behavior.