Skip to yearly menu bar Skip to main content

Workshop: Theory and Practice of Differential Privacy

Wide Network Learning with Differential Privacy

Huanyu Zhang · Ilya Mironov · Meisam Hejazinia


Despite intense interest and considerable effort, the current generation of neural networks suffers a significant loss of accuracy under most practically relevant privacy training regimes. One particularly challenging class of neural networks are the wide ones, such as those deployed for NLP or recommender systems.

Observing that these models share something in common---an embedding layer that reduces the dimensionality of the input---we focus on developing a general approach towards training these models that takes advantage of the sparsity of the gradients. We propose a novel algorithm for privately training neural networks. Furthermore, we provide an empirical study of a DP wide neural network on a real-world dataset, which has been rarely explored in the previous work.

Chat is not available.