Personalized Additive Modeling for Multi-level Federated Learning

Contemporary AI faces the challenge of balancing generality with user-specific personalization. In federated learning (FL), this challenge is amplified by highly heterogeneous client data with complex non-IID patterns beyond standard modeling assumptions. Many existing FL methods are designed for relatively restricted heterogeneity settings (e.g., a fixed number of clusters or a fixed form of personalization), limiting their robustness under complex structures. In this work, we study FL from a \emph{multi-level non-IID} perspective, where client similarity is approximated by multiple granularities of shared knowledge: global, subgroup, and client-specific components. This view captures coarse-to-fine relationships while requiring less prior knowledge of task boundaries. Building on this insight, we propose \emph{Federated Multi-level Additive Modeling} (FeMAM), which learns multiple levels of shareable models and constructs personalized predictors via additive composition across levels. To move beyond a fixed structure, FeMAM allows models to grow and be pruned dynamically during training, adapting to diverse federated scenarios. Despite employing multiple models, FeMAM remains cost-friendly by activating only a small subset (one level) of models for training at a time. Extensive experiments show that FeMAM effectively approximates complex non-IID structures and consistently outperforms representative clustered and personalized FL baselines.

AI systems often need to work for many different people, but people’s data and needs are not the same. A single shared model is like giving every student the same textbook: useful, but not enough for everyone. Training a fully separate model for each person can work better, but it wastes data and computing power. We propose FeMAM, a way for many devices or organizations to learn together while building a model that fits each one better. FeMAM learns several layers of knowledge: some knowledge is shared by everyone, some is shared by smaller groups, and some is specific to one user. Each user’s final model is made by adding together the useful pieces, like building a custom LEGO model from common blocks and personal blocks. The system can also stop adding pieces when they no longer help, so it does not make the model unnecessarily large. In experiments, this approach worked better than several existing methods when users’ data followed different and complex patterns. This could help future AI services become more accurate for different users without forcing everyone into one fixed model.