FedGain: Toward Negative-Gain-Free Client Collaboration in Federated Learning

Data heterogeneity is a fundamental challenge in Federated Learning (FL), where induced model drift often results in "negative gains" for global models on data-abundant clients, with performance falling below that of local training. To address this issue, we propose FedGain, a novel framework that optimizes collaborative client clustering to mitigate the negative gain. We are the first to develop a modified Scaling Law (SL) to quantify the reduction in data utility caused by heterogeneity and define Effective Federated Capacity to align clients with the highest potential collaboration gains. Extensive experiments demonstrate that our modified SL strictly adheres to the power-law learning discipline in non-IID scenarios. FedGain effectively suppresses negative gains to a negligible level across various FL algorithms and outperforms other Clustered FL methods.

Federated learning is a technique that enables multiple institutions (such as different hospitals or banks) to collaboratively train a powerful AI model without exchanging private data. However, in practice, there is a problematic phenomenon: not all participants benefit from collaboration. Sometimes, due to significant differences in the data held by different participants, forced collaboration can actually be counterproductive, resulting in a model that performs worse for a given institution than if it had trained on its own in isolation. Our research addresses this issue by proposing a new solution called FedGain. Our goal is to compute a model that ensures all participants benefit from collaboration, by quantifying the differences between different clients and their varying data scales.