Timezone: »

Personalized Federated Learning under Mixture of Distributions
Yue Wu · Shuaicheng Zhang · Wenchao Yu · Yanchi Liu · Quanquan Gu · Dawei Zhou · Haifeng Chen · Wei Cheng

Wed Jul 26 05:00 PM -- 06:30 PM (PDT) @ Exhibit Hall 1 #514

The recent trend towards Personalized Federated Learning (PFL) has garnered significant attention as it allows for the training of models that are tailored to each client while maintaining data privacy. However, current PFL techniques primarily focus on modeling the conditional distribution heterogeneity (i.e. concept shift), which can result in suboptimal performance when the distribution of input data across clients diverges (i.e. covariate shift). Additionally, these techniques often lack the ability to adapt to unseen data, further limiting their effectiveness in real-world scenarios. To address these limitations, we propose a novel approach, FedGMM, which utilizes Gaussian mixture models (GMM) to effectively fit the input data distributions across diverse clients. The model parameters are estimated by maximum likelihood estimation utilizing a federated Expectation-Maximization algorithm, which is solved in closed form and does not assume gradient similarity. Furthermore, FedGMM possesses an additional advantage of adapting to new clients with minimal overhead, and it also enables uncertainty quantification. Empirical evaluations on synthetic and benchmark datasets demonstrate the superior performance of our method in both PFL classification and novel sample detection.

Author Information

Yue Wu (UCLA)
Shuaicheng Zhang (Virginia Tech)
Wenchao Yu (NEC Labs)
Yanchi Liu (NEC-Labs)
Quanquan Gu (University of California, Los Angeles)
Dawei Zhou (Virginia Tech)
Haifeng Chen (NEC Labs)
Wei Cheng (NEC Laboratories America)
Wei Cheng

Wei Cheng is a Senior Researcher at NEC Labs America. He received his Ph.D. from the Department of Computer Science, UNC at Chapel Hill in 2015, advised by Prof. Wei Wang. His research interests include data science, machine learning and bioinformatics. He has filed more than sixty patents, and has published more than 100 research papers in top-tier conferences such as NeurIPS, ICML, SIGKDD, ICLR, WWW, EMNLP, ISMB and journals such as Nature, Science, TNNLS, TKDE, Bioinformatics, etc. His research results received Best Research Paper Runner-Up Award at SIGKDD 2016 and were nominated for the Best Paper Award at ICDM 2018, ICDM 2017, ICDM 2015 and SDM 2012. He has also served as area chair, senior program committee member for several top-tier conferences including SIGKDD, IJCAI, SDM, AAAI, WSDM, etc.

More from the Same Authors