Lightweight Federated Incremental Learning via Decoupled Replay

Federated Incremental Learning (FIL) aims to learn streaming tasks across distributed clients without catastrophic forgetting while preserving privacy. Most existing methods mitigate forgetting by replaying historical samples, but it can pose privacy risks and incur high resource overhead, limiting deployment on resource-constrained edge devices. To address this challenge, we propose a novel Lightweight Federated Incremental Learning framework called Li-FIL that leverages dense features synthesized by a server-side secure generator to enable efficient feature-based decoupled replay. More specifically, each client extracts high-confidence features from new tasks, enrich them via mixup, and privatize them before uploading to the server, which reduces both storage and communication overhead. A generator is deployed on the server to learn the distributions of clients and generate global features for replay. Moreover, to enable clients to better learn from these dense features, we decouple local training into classifier stabilization and encoder regularization. This design allows feature replay and alignment between new and previous features to be conducted separately and more effectively. Extensive experiments demonstrate that Li-FIL outperforms other state-of-the-art methods by up to 10.14% in terms of accuracy on both old and new tasks with superior resource efficiency.

Federated learning allows organizations to train AI models together without sharing sensitive data. However, in many real-world applications, new types of data arrive over time, and AI models often forget what they learned before when adapting to these new tasks. Existing solutions usually rely on storing or recreating past data to refresh the model’s memory, but this can create privacy concerns and require substantial storage and computing resources. We explored whether models could remember previous knowledge without ever keeping old data. Instead of storing data samples, our approach allows devices to share only compact, privacy-protected summaries of what they have learned. These summaries are used to help the model recall previous tasks while continuing to learn new ones. Our experiments show that this strategy helps models retain more knowledge from the past while using fewer resources than existing methods. This makes continual federated learning more practical for privacy-sensitive applications such as healthcare, finance, and mobile devices, where both data protection and efficiency are critical.