Timezone: »

 
Spotlight
The Fundamental Price of Secure Aggregation in Differentially Private Federated Learning
Wei-Ning Chen · Christopher Choquette Choo · Peter Kairouz · Ananda Suresh

Wed Jul 20 02:30 PM -- 02:35 PM (PDT) @ Room 307
in SA: Privacy-preserving Statistics and Machine Learning »
We consider the problem of training a $d$ dimensional model with distributed differential privacy (DP) where secure aggregation (SecAgg) is used to ensure that the server only sees the noisy sum of $n$ model updates in every training round. Taking into account the constraints imposed by SecAgg, we characterize the fundamental communication cost required to obtain the best accuracy achievable under $\varepsilon$ central DP (i.e. under a fully trusted server and no communication constraints). Our results show that $\tilde{O}\lp \min(n^2\varepsilon^2, d) \rp$ bits per client are both sufficient and necessary, and this fundamental limit can be achieved by a linear scheme based on sparse random projections. This provides a significant improvement relative to state-of-the-art SecAgg distributed DP schemes which use $\tilde{O}(d\log(d/\varepsilon^2))$ bits per client. Empirically, we evaluate our proposed scheme on real-world federated learning tasks. We find that our theoretical analysis is well matched in practice. In particular, we show that we can reduce the communication cost to under $1.78$ bits per parameter in realistic privacy settings without decreasing test-time performance. Our work hence theoretically and empirically specifies the fundamental price of using SecAgg.
Keywords: SA: Privacy-preserving Statistics and Machine Learning · SA: Trustworthy Machine Learning

Author Information

Wei-Ning Chen (Stanford University)
Wei-Ning Chen

Wei-Ning Chen is currently a Ph.D. student at Stanford EE under the support of Stanford Graduate Fellowship (SGF). His research interests broadly lie in information-theoretic and algorithmic aspects of data science. He adopt tools mainly from information theory, theoretical machine learning, and statistical inference, with a current focus on distributed inference, federated learning and differential privacy.

Christopher Choquette Choo (Google Brain)
Peter Kairouz (Google)
Ananda Suresh (Google Research)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors