DAdaQuant: Doubly-adaptive quantization for communication-efficient Federated Learning

Robert Hönig · Yiren Zhao · Robert Mullins

Hall E #406

Keywords: [ OPT: Large Scale, Parallel and Distributed ] [ Deep Learning ]

[ Abstract ]
[ Slides [ Poster [ Paper PDF
Wed 20 Jul 3:30 p.m. PDT — 5:30 p.m. PDT
Spotlight presentation: DL: Robustness
Wed 20 Jul 10:15 a.m. PDT — 11:45 a.m. PDT

Abstract: Federated Learning (FL) is a powerful technique to train a model on a server with data from several clients in a privacy-preserving manner. FL incurs significant communication costs because it repeatedly transmits the model between the server and clients. Recently proposed algorithms quantize the model parameters to efficiently compress FL communication. We find that dynamic adaptations of the quantization level can boost compression without sacrificing model quality. We introduce DAdaQuant as a doubly-adaptive quantization algorithm that dynamically changes the quantization level across time and different clients. Our experiments show that DAdaQuant consistently improves client$\rightarrow$server compression, outperforming the strongest non-adaptive baselines by up to $2.8\times$.

Chat is not available.