Timezone: »

 
Poster
Virtual Homogeneity Learning: Defending against Data Heterogeneity in Federated Learning
Zhenheng Tang · Yonggang Zhang · Shaohuai Shi · Xin He · Bo Han · Xiaowen Chu

Thu Jul 21 03:00 PM -- 05:00 PM (PDT) @ Hall E #709
in Poster Session 3 »

In federated learning (FL), model performance typically suffers from client drift induced by data heterogeneity, and mainstream works focus on correcting client drift. We propose a different approach named virtual homogeneity learning (VHL) to directly ``rectify'' the data heterogeneity. In particular, VHL conducts FL with a virtual homogeneous dataset crafted to satisfy two conditions: containing \emph{no} private information and being separable. The virtual dataset can be generated from pure noise shared across clients, aiming to calibrate the features from the heterogeneous clients. Theoretically, we prove that VHL can achieve provable generalization performance on the natural distribution. Empirically, we demonstrate that VHL endows FL with drastically improved convergence speed and generalization performance. VHL is the first attempt towards using a virtual dataset to address data heterogeneity, offering new and effective means to FL.

Keywords: DL: Algorithms · OPT: Large Scale, Parallel and Distributed

Author Information

Zhenheng Tang (Hong Kong Baptist University)
Yonggang Zhang (Hong Kong Baptist University)
Shaohuai Shi (The Hong Kong University of Science and Technology)
Xin He (Hong Kong Baptist University)
Bo Han (HKBU / RIKEN)
Xiaowen Chu (Hong Kong University of Science and Technology (Guangzhou))

Dr. Chu received his B.Eng. degree in Computer Science from Tsinghua University, Beijing, P. R. China, in 1999, and the Ph.D. degree in Computer Science from The Hong Kong University of Science and Technology in 2003. He is currently a Professor at the Data Science and Analytics Thrust, Information Hub of HKUST(GZ), and an Affiliate Professor in the Department of Computer Science and Engineering, HKUST. He has been working at the Department of Computer Science, Hong Kong Baptist University during 2003-2021. He is a senior member of IEEE and a member of ACM. He is a vice-chairman of the Blockchain Technical Committee of China Institute of Communications. His current research interests include GPU Computing, Distributed Machine Learning, Cloud Computing, and Wireless Networks. He is especially interested in the modelling, parallel algorithm design, application optimization, and energy efficiency of GPU computing.

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors