Timezone: »

 
Poster
An end-to-end Differentially Private Latent Dirichlet Allocation Using a Spectral Algorithm
Christopher DeCarolis · Mukul A Ram · Seyed Esmaeili · Yu-Xiang Wang · Furong Huang

Tue Jul 14 07:00 AM -- 07:45 AM & Tue Jul 14 07:00 PM -- 07:45 PM (PDT) @
in Poster Session 1 »

We provide an end-to-end differentially private spectral algorithm for learning LDA, based on matrix/tensor decompositions, and establish theoretical guarantees on utility/consistency of the estimated model parameters. We represent the spectral algorithm as a computational graph. Noise can be injected along the edges of this graph to obtain differential privacy. We identify subsets of edges, named ``configurations'', such that adding noise to all edges in such a subset guarantees differential privacy of the end-to-end spectral algorithm. We characterize the sensitivity of the edges with respect to the input and thus estimate the amount of noise to be added to each edge for any required privacy level. We then characterize the utility loss for each configuration as a function of injected noise. Overall, by combining the sensitivity and utility characterization, we obtain an end-to-end differentially private spectral algorithm for LDA and identify which configurations outperform others under specific regimes. We are the first to achieve utility guarantees under a required level of differential privacy for learning in LDA. We additionally show that our method systematically outperforms differentially private variational inference.

Keywords: Privacy-preserving Statistics and Machine Learning · Spectral Methods

Author Information

Christopher DeCarolis (University of Maryland)

Chris is an undergraduate student at the University of Maryland with research interests in machine learning. He does research under Professor Furong Huang, and has done internships at Microsoft and Facebook in the past.

Mukul A Ram (University of Maryland)
Seyed Esmaeili (University of Maryland, College Park)
Yu-Xiang Wang (UC Santa Barbara)
Yu-Xiang Wang

Yu-Xiang Wang is the Eugene Aas Assistant Professor of Computer Science at UCSB. He runs the Statistical Machine Learning lab and co-founded the UCSB Center for Responsible Machine Learning. He is also visiting Amazon Web Services. Yu-Xiang’s research interests include statistical theory and methodology, differential privacy, reinforcement learning, online learning and deep learning.

Furong Huang (University of Maryland College Park)

More from the Same Authors