LEGO-FL: Learning Heterogeneous Federated Models as a LEGO Assembly Games

Just as LEGO pieces can be assembled into an unlimited variety of structures, heterogeneous federated learning (HFL) can be viewed as the assembly of diverse model components. Inspired by this analogy, we reformulate HFL as a LEGO-like assembly game. The central challenge in HFL lies in learning across heterogeneous model architectures, which hinders direct parameter sharing. To address this challenge, we propose to decompose models into a set of modular components—analogous to LEGO pieces and collaboratively learn these components across clients under predefined composition rules. Based on this perspective, we develop a novel federated learning framework, termed LEGO-FL, which enables flexible model construction while preserving collaborative learning. Extensive experiments validate the effectiveness of LEGO-FL under different heterogeneous settings and system scales.

In Heterogeneous Federated Learning (HFL), devices often use different model architectures, making it highly challenging to aggregate their knowledge directly due to mismatched structures. Existing methods either depend on shallow knowledge exchange or restrict the global architecture, which underutilizes architectural diversity. To tackle this, we introduce LEGO-FL, a flexible training paradigm. We decompose diverse models into modular components—analogous to LEGO pieces—and dynamically assemble them under predefined rules. By exploiting latent correlations across different designs, we customize local heterogeneous models beyond predefined templates, effectively integrating both global representations and personalized features. This work drives a profound paradigm shift from restrictive neuron-level pruning to dynamic block-level reassembly. It not only delivers significant advantages in collaborative performance but also drastically reduces computational overhead, offering a groundbreaking solution to the time-consuming bottlenecks of previous reassembly methods.