Timezone: »

 
Spotlight
Federated Learning with Partial Model Personalization
Krishna Pillutla · Kshitiz Malik · Abdel-rahman Mohamed · Michael Rabbat · Maziar Sanjabi · Lin Xiao

Thu Jul 21 01:40 PM -- 01:45 PM (PDT) @ Hall G
in Applications/Optimization »

We consider two federated learning algorithms for training partially personalized models, where the shared and personal parameters are updated either simultaneously or alternately on the devices. Both algorithms have been proposed in the literature, but their convergence properties are not fully understood, especially for the alternating variant. We provide convergence analyses of both algorithms in the general nonconvex setting with partial participation and delineate the regime where one dominates the other. Our experiments on real-world image, text, and speech datasets demonstrate that (a) partial personalization can obtain most of the benefits of full model personalization with a small fraction of personal parameters, and, (b) the alternating update algorithm outperforms the simultaneous update algorithm by a small but consistent margin.

Keywords: OPT: Non-Convex · OPT: Large Scale, Parallel and Distributed

Author Information

Krishna Pillutla (University of Washington)

PhD student at the University of Washington. Advisors: Zaid Harchaoui and Sham Kakade Interests: ML/Optimization, structured prediction, federated learning

Kshitiz Malik (Facebook)
Abdel-rahman Mohamed (Facebook AI Research (FAIR))
Michael Rabbat (Facebook)
Maziar Sanjabi (Meta AI)
Lin Xiao (Meta AI Research)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors