Timezone: »

 
Poster
Structured Cooperative Learning with Graphical Model Priors
Shuangtong Li · Tianyi Zhou · Xinmei Tian · Dacheng Tao

Wed Jul 26 05:00 PM -- 06:30 PM (PDT) @ Exhibit Hall 1 #621
in Poster Session 4 »
Event URL: https://github.com/ShuangtongLi/SCooL »

We study how to train personalized models for different tasks on decentralized devices with limited local data. We propose "Structured Cooperative Learning (SCooL)", in which a cooperation graph across devices is generated by a graphical model prior to automatically coordinate mutual learning between devices. By choosing graphical models enforcing different structures, we can derive a rich class of existing and novel decentralized learning algorithms via variational inference. In particular, we show three instantiations of SCooL that adopt Dirac distribution, stochastic block model (SBM), and attention as the prior generating cooperation graphs. These EM-type algorithms alternate between updating the cooperation graph and cooperative learning of local models. They can automatically capture the cross-task correlations among devices by only monitoring their model updating in order to optimize the cooperation graph. We evaluate SCooL and compare it with existing decentralized learning methods on an extensive set of benchmarks, on which SCooL always achieves the highest accuracy of personalized models and significantly outperforms other baselines on communication efficiency. Our code is available at https://github.com/ShuangtongLi/SCooL.

[ Slides]  [ PDF]  [ Poster] 

Author Information

Shuangtong Li (University of Science and Technology of China)
Tianyi Zhou (University of Maryland)
Xinmei Tian (University of Science and Technology of China)
Dacheng Tao

More from the Same Authors