Timezone: »
Poster
Sharp Composition Bounds for Gaussian Differential Privacy via Edgeworth Expansion
Qinqing Zheng · Jinshuo Dong · Qi Long · Weijie Su
Thu Jul 16 06:00 AM -- 06:45 AM & Thu Jul 16 06:00 PM -- 06:45 PM (PDT) @
Datasets containing sensitive information are often sequentially analyzed by many algorithms and, accordingly, a fundamental question in differential privacy is concerned with how the overall privacy bound degrades under composition. To address this question, we introduce a family of analytical and sharp privacy bounds under composition using the Edgeworth expansion in the framework of the recently proposed $f$-differential privacy. In short, whereas the existing composition theorem, for example, relies on the central limit theorem, our new privacy bounds under composition gain improved tightness by leveraging the refined approximation accuracy of the Edgeworth expansion. Our approach is easy to implement and computationally efficient for any number of compositions. The superiority of these new bounds is confirmed by an asymptotic error analysis and an application to quantifying the overall privacy guarantees of noisy stochastic gradient descent used in training private deep neural networks.
Author Information
Qinqing Zheng (University of Pennsylvania)
Jinshuo Dong (University of Pennsylvania)
Qi Long (University of Pennsylvania)
Weijie Su (University of Pennsylvania)
More from the Same Authors
-
2021 : On the Convergence of Deep Learning with Differential Privacy »
Zhiqi Bu · Hua Wang · Qi Long · Weijie Su -
2023 : Reward Collapse in Aligning Large Language Models: A Prompt-Aware Approach to Preference Rankings »
Ziang Song · Tianle Cai · Jason Lee · Weijie Su -
2023 Poster: Semi-Supervised Offline Reinforcement Learning with Action-Free Trajectories »
Qinqing Zheng · Mikael Henaff · Brandon Amos · Aditya Grover -
2023 Poster: The Implicit Regularization of Dynamical Stability in Stochastic Gradient Descent »
Lei Wu · Weijie Su -
2022 Poster: Online Decision Transformer »
Qinqing Zheng · Amy Zhang · Aditya Grover -
2022 Oral: Online Decision Transformer »
Qinqing Zheng · Amy Zhang · Aditya Grover -
2022 Poster: ROCK: Causal Inference Principles for Reasoning about Commonsense Causality »
Jiayao Zhang · Hongming ZHANG · Weijie Su · Dan Roth -
2022 Spotlight: ROCK: Causal Inference Principles for Reasoning about Commonsense Causality »
Jiayao Zhang · Hongming ZHANG · Weijie Su · Dan Roth -
2021 Poster: Oneshot Differentially Private Top-k Selection »
Gang Qiao · Weijie Su · Li Zhang -
2021 Spotlight: Oneshot Differentially Private Top-k Selection »
Gang Qiao · Weijie Su · Li Zhang -
2021 Poster: Toward Better Generalization Bounds with Locally Elastic Stability »
Zhun Deng · Hangfeng He · Weijie Su -
2021 Spotlight: Toward Better Generalization Bounds with Locally Elastic Stability »
Zhun Deng · Hangfeng He · Weijie Su -
2021 Poster: Near-Optimal Confidence Sequences for Bounded Random Variables »
Arun Kuchibhotla · Qinqing Zheng -
2021 Spotlight: Near-Optimal Confidence Sequences for Bounded Random Variables »
Arun Kuchibhotla · Qinqing Zheng -
2020 Poster: Towards Understanding the Dynamics of the First-Order Adversaries »
Zhun Deng · Hangfeng He · Jiaoyang Huang · Weijie Su