Timezone: »

 
Poster
FedBoost: A Communication-Efficient Algorithm for Federated Learning
Jenny Hamer · Mehryar Mohri · Ananda Theertha Suresh

Tue Jul 14 07:00 AM -- 07:45 AM & Tue Jul 14 06:00 PM -- 06:45 PM (PDT) @
in Poster Session 1 »

Communication cost is often a bottleneck in federated learning and other client-based distributed learning scenarios. To overcome this, several gradient compression and model compression algorithms have been proposed. In this work, we propose an alternative approach whereby an ensemble of pre-trained base predictors is trained via federated learning. This method allows for training a model which may otherwise surpass the communication bandwidth and storage capacity of the clients to be learned with on-device data through federated learning. Motivated by language modeling, we prove the optimality of ensemble methods for density estimation for standard empirical risk minimization and agnostic risk minimization. We provide communication-efficient ensemble algorithms for federated learning, where per-round communication cost is independent of the size of the ensemble. Furthermore, unlike works on gradient compression, our proposed approach reduces the communication cost of both server-to-client and client-to-server communication.

Keywords: Boosting / Ensemble Methods · Other · General Machine Learning Techniques

Author Information

Jenny Hamer (Google Research)
Mehryar Mohri (Google Research and Courant Institute of Mathematical Sciences)
Ananda Theertha Suresh (Google Research)

More from the Same Authors