Timezone: »
Personalized federated learning is proposed to handle the data heterogeneity problem amongst clients by learning dedicated tailored local models for each user. However, existing works are often built in a centralized way, leading to high communication pressure and high vulnerability when a failure or an attack on the central server occurs. In this work, we propose a novel personalized federated learning framework in a decentralized (peer-to-peer) communication protocol named DisPFL, which employs personalized sparse masks to customize sparse local models on the edge. To further save the communication and computation cost, we propose a decentralized sparse training technique, which means that each local model in DisPFL only maintains a fixed number of active parameters throughout the whole local training and peer-to-peer communication process. Comprehensive experiments demonstrate that DisPFL significantly saves the communication bottleneck for the busiest node among all clients and, at the same time, achieves higher model accuracy with less computation cost and communication rounds. Furthermore, we demonstrate that our method can easily adapt to heterogeneous local clients with varying computation complexities and achieves better personalized performances.
Author Information
Rong Dai (University of Science and Technology of China)
Li Shen (JD Explore Academy)
Fengxiang He (JD.com Inc)

Fengxiang He received his BSc in statistics from University of Science and Technology of China and MPhil and PhD in computer science from the University of Sydney. He is currently an algorithm scientist at JD Explore Academy, JD.com Inc, leading its trustworthy AI team. His research interest is theory and practice of trustworthy AI, including deep learning theory, privacy preservation, and fairness. He has published in top conferences and journals, including ICML, NeurIPS, ICLR, CVPR, ICCV, UAI, AAAI, IJCAI, TNNLS, TCSVT, TMM, and Neural Computation.
Xinmei Tian (University of Science and Technology of China)
Dacheng Tao
Related Events (a corresponding poster, oral, or spotlight)
-
2022 Poster: DisPFL: Towards Communication-Efficient Personalized Federated Learning via Decentralized Sparse Training »
Wed. Jul 20th through Thu the 21st Room Hall E #326
More from the Same Authors
-
2023 : Learning Better with Less: Effective Augmentation for Sample-Efficient Visual Reinforcement Learning »
Guozheng Ma · · Haoyu Wang · Lu Li · Zilin Wang · Zhen Wang · Li Shen · Xueqian Wang · Dacheng Tao -
2023 Oral: Dynamic Regularized Sharpness Aware Minimization in Federated Learning: Approaching Global Consistency and Smooth Landscape »
Yan Sun · Li Shen · Shixiang Chen · Liang Ding · Dacheng Tao -
2023 Oral: Tilted Sparse Additive Models »
Yingjie Wang · Hong Chen · Weifeng Liu · Fengxiang He · Tieliang Gong · YouCheng Fu · Dacheng Tao -
2023 Poster: Structured Cooperative Learning with Graphical Model Priors »
Shuangtong Li · Tianyi Zhou · Xinmei Tian · Dacheng Tao -
2023 Poster: Tilted Sparse Additive Models »
Yingjie Wang · Hong Chen · Weifeng Liu · Fengxiang He · Tieliang Gong · YouCheng Fu · Dacheng Tao -
2023 Poster: Moderately Distributional Exploration for Domain Generalization »
Rui Dai · Yonggang Zhang · zhen fang · Bo Han · Xinmei Tian -
2023 Poster: Are Large Kernels Better Teachers than Transformers for ConvNets? »
Tianjin Huang · Lu Yin · Zhenyu Zhang · Li Shen · Meng Fang · Mykola Pechenizkiy · Zhangyang “Atlas” Wang · Shiwei Liu -
2023 Poster: Decentralized SGD and Average-direction SAM are Asymptotically Equivalent »
Tongtian Zhu · Fengxiang He · Kaixuan Chen · Mingli Song · Dacheng Tao -
2023 Poster: Improving the Model Consistency of Decentralized Federated Learning »
Yifan Shi · Li Shen · Kang Wei · Yan Sun · Bo Yuan · Xueqian Wang · Dacheng Tao -
2023 Poster: Dynamic Regularized Sharpness Aware Minimization in Federated Learning: Approaching Global Consistency and Smooth Landscape »
Yan Sun · Li Shen · Shixiang Chen · Liang Ding · Dacheng Tao -
2023 Poster: Learning to Learn from APIs: Black-Box Data-Free Meta-Learning »
Zixuan Hu · Li Shen · Zhenyi Wang · Baoyuan Wu · Chun Yuan · Dacheng Tao -
2023 Poster: CoCo: A Coupled Contrastive Framework for Unsupervised Domain Adaptive Graph Classification »
Nan Yin · Li Shen · Mengzhu Wang · Long Lan · Zeyu Ma · Chong Chen · Xian-Sheng Hua · Xiao Luo -
2022 : Paper 12: SafeRL-Kit: Evaluating Efficient Reinforcement Learning Methods for Safe Autonomous Driving »
· Li Shen · Bo Yuan · Xueqian Wang -
2022 Poster: Identity-Disentangled Adversarial Augmentation for Self-supervised Learning »
Kaiwen Yang · Tianyi Zhou · Xinmei Tian · Dacheng Tao -
2022 Spotlight: Identity-Disentangled Adversarial Augmentation for Self-supervised Learning »
Kaiwen Yang · Tianyi Zhou · Xinmei Tian · Dacheng Tao -
2022 Poster: Understanding Robust Overfitting of Adversarial Training and Beyond »
Chaojian Yu · Bo Han · Li Shen · Jun Yu · Chen Gong · Mingming Gong · Tongliang Liu -
2022 Spotlight: Understanding Robust Overfitting of Adversarial Training and Beyond »
Chaojian Yu · Bo Han · Li Shen · Jun Yu · Chen Gong · Mingming Gong · Tongliang Liu -
2022 Poster: Improving Task-free Continual Learning by Distributionally Robust Memory Evolution »
Zhenyi Wang · Li Shen · Le Fang · Qiuling Suo · Tiehang Duan · Mingchen Gao -
2022 Poster: Deep Neural Network Fusion via Graph Matching with Applications to Model Ensemble and Federated Learning »
Chang Liu · Chenfei Lou · Runzhong Wang · Alan Yuhan Xi · Li Shen · Junchi Yan -
2022 Poster: Topology-aware Generalization of Decentralized SGD »
Tongtian Zhu · Fengxiang He · Lan Zhang · Zhengyang Niu · Mingli Song · Dacheng Tao -
2022 Spotlight: Topology-aware Generalization of Decentralized SGD »
Tongtian Zhu · Fengxiang He · Lan Zhang · Zhengyang Niu · Mingli Song · Dacheng Tao -
2022 Spotlight: Deep Neural Network Fusion via Graph Matching with Applications to Model Ensemble and Federated Learning »
Chang Liu · Chenfei Lou · Runzhong Wang · Alan Yuhan Xi · Li Shen · Junchi Yan -
2022 Spotlight: Improving Task-free Continual Learning by Distributionally Robust Memory Evolution »
Zhenyi Wang · Li Shen · Le Fang · Qiuling Suo · Tiehang Duan · Mingchen Gao -
2020 Poster: Dual-Path Distillation: A Unified Framework to Improve Black-Box Attacks »
Yonggang Zhang · Ya Li · Tongliang Liu · Xinmei Tian -
2020 Poster: Communication-Efficient Distributed Stochastic AUC Maximization with Deep Neural Networks »
Zhishuai Guo · Mingrui Liu · Zhuoning Yuan · Li Shen · Wei Liu · Tianbao Yang -
2018 Poster: An Algorithmic Framework of Variable Metric Over-Relaxed Hybrid Proximal Extra-Gradient Method »
Li Shen · Peng Sun · Yitong Wang · Wei Liu · Tong Zhang -
2018 Oral: An Algorithmic Framework of Variable Metric Over-Relaxed Hybrid Proximal Extra-Gradient Method »
Li Shen · Peng Sun · Yitong Wang · Wei Liu · Tong Zhang -
2017 Poster: Beyond Filters: Compact Feature Map for Portable Deep Model »
Yunhe Wang · Chang Xu · Chao Xu · Dacheng Tao -
2017 Talk: Beyond Filters: Compact Feature Map for Portable Deep Model »
Yunhe Wang · Chang Xu · Chao Xu · Dacheng Tao -
2017 Poster: Algorithmic Stability and Hypothesis Complexity »
Tongliang Liu · Gábor Lugosi · Gergely Neu · Dacheng Tao -
2017 Talk: Algorithmic Stability and Hypothesis Complexity »
Tongliang Liu · Gábor Lugosi · Gergely Neu · Dacheng Tao -
2017 Poster: GSOS: Gauss-Seidel Operator Splitting Algorithm for Multi-Term Nonsmooth Convex Composite Optimization »
Li Shen · Wei Liu · Ganzhao Yuan · Shiqian Ma -
2017 Talk: GSOS: Gauss-Seidel Operator Splitting Algorithm for Multi-Term Nonsmooth Convex Composite Optimization »
Li Shen · Wei Liu · Ganzhao Yuan · Shiqian Ma