Scaling Unsupervised Multi-Source Federated Domain Adaptation through Group-Wise Discrepancy Minimization

Unsupervised multi-source domain adaptation (UMDA) leverages labeled data from multiple source domains to generalize to an unlabeled target. While federated UMDA addresses privacy by avoiding raw data sharing, existing methods scale poorly as the number of sources increases, often suffering from high computational overhead or training instability. We propose GALA, a scalable and robust federated UMDA framework designed for high-diversity settings. GALA achieves scalability by coupling a novel inter-group discrepancy minimization objective that approximates pairwise alignment with linear complexity alongside a temperature-controlled, centroid-based weighting strategy for dynamic source prioritization. These components enable stable, parallelizable training across many heterogeneous sources, addressing a critical scalability bottleneck that remains largely unaddressed in current literature. To evaluate performance in high-diversity scenarios, we introduce Digit-18, a new benchmark comprising 18 datasets with varied synthetic and real-world domain shifts. Extensive experiments demonstrate that GALA achieves state-of-the-art results on standard benchmarks and significantly outperforms prior methods in large-scale settings where others either fail to converge or become computationally infeasible.

Training AI models usually requires large amounts of correctly labeled data, which is often not available in the real world. One way to address this problem is to use additional datasets from related areas where labels already exist. However, current methods struggle when a large number of diverse helper datasets are involved, especially when the data is distributed and cannot be shared. While well-aligned datasets help the model learn, those that are too different can distort the training process and harm overall performance. We propose GALA, a method that robustly learns from distributed helper datasets by encouraging their models to agree on predictions for the target data. GALA also uses a similarity score to estimate which datasets are most relevant and prioritizes them during training. Since existing benchmarks don't adequately capture the large-scale scenario, we introduce Digit-18. It is a new benchmark that extends existing digit recognition datasets with additional datasets and variations. Our experiments show that GALA consistently performs well and significantly outperforms existing methods when many diverse datasets are available. Our work highlights the importance of developing methods for these realistic challenges, provides a benchmark for evaluation, and introduces an effective solution.