FedEBA+: Towards Fair and Effective Federated Learning via Entropy-Based Model

Federated Learning (FL) often suffers from sacrificing global model accuracy when improving client-level fairness due to data heterogeneity, which often leads to inconsistent performance of the globally trained models, resulting in unfair outcomes among users. Existing fair FL algorithms face a bottleneck: they either sacrifice global model accuracy to promote fairness or fall short of achieving optimal fairness. In this paper, we propose a novel framework that effectively improves fairness while preserving global accuracy by integrating information-theoretic principles with model alignment. Specifically, we leverage the Maximum Entropy Principle to derive an analytic, closed-form solution for fair aggregation weights, ensuring significant fairness enhancements. We further employ a step-wise model alignment strategy that synchronizes gradient directions across heterogeneous clients, effectively mitigating the drift induced by local updates. Theoretical analysis proves that our method guarantees convergence even in non-convex settings. Importantly, we push the theoretical frontier of federated fairness by extending performance variance analysis to generalized regression, providing broader guarantees. Extensive experiments on five datasets demonstrate that our approach consistently outperforms state-of-the-art methods, achieving superior fairness without sacrificing global accuracy. Our code is available at \href{https://github.com/T-Lab-CUHKSZ/FedEBA-Plus}{https://github.com/T-Lab-CUHKSZ/FedEBA-Plus}.

Federated learning allows many users or organizations to jointly train an AI model without sharing their private data. However, when different participants have very different data, the final model may work well for some participants but poorly for others. This creates an unfair outcome: some users benefit much less from the shared model, even though they contributed to training it. This paper proposes FedEBA+, a new approach for making federated learning both fairer and effective. The key idea is simple: during training, the system should pay more attention to participants for whom the current model performs poorly, while still preserving the overall quality of the model. FedEBA+ does this through a principled weighting strategy inspired by the idea of choosing the least biased distribution of attention across participants. It also guides local training updates so that different participants do not drift too far away from a shared fair direction. Our theoretical analysis shows that FedEBA+ can reliably converge during training and can reduce differences in performance across participants. Experiments on multiple image and text datasets show that FedEBA+ consistently improves fairness across clients while maintaining, and sometimes improving, the overall model accuracy. These results suggest that federated learning systems can be designed to serve participants more evenly without sacrificing the usefulness of the final model.