Understanding Private Learning From Feature Perspective

Differentially private Stochastic Gradient Descent (DP-SGD) has become integral to privacy-preserving machine learning, ensuring robust privacy guarantees in sensitive domains. Despite notable empirical advances leveraging features from non-private, pre-trained models to enhance DP-SGD training, a theoretical understanding of feature dynamics in private learning remains underexplored. This paper presents the first theoretical framework to analyze private training through a feature learning perspective. Building on the multi-patch data structure from prior work, our analysis distinguishes between label-dependent feature signals and label-independent noise—a critical aspect overlooked by existing analyses in the DP community. Employing a two-layer CNN with polynomial ReLU activation, we theoretically characterize both feature signal learning and data noise memorization in private training via noisy gradient descent. Our findings reveal that (1) Effective private signal learning requires a higher signal-to-noise ratio (SNR) compared to non-private training, and (2) When data noise memorization occurs in non-private learning, it will also occur in private learning, leading to poor generalization despite small training loss. Our findings highlight the challenges of private learning and prove the benefit of feature enhancement to improve SNR. Experiments on synthetic and real-world datasets also validate our theoretical findings.

This paper studies the trade-off between learning from data and protecting the privacy of the people represented in that data. In many applications, such as healthcare, finance, and large-scale AI systems, training data can contain sensitive information. Privacy-preserving training methods aim to prevent models from revealing too much about any individual example, but this protection often comes at the cost of lower accuracy. Our work provides a clearer theoretical understanding of when this trade-off can be improved. We show that, under certain natural structures in the data and model, it is possible to train accurate models while still providing strong privacy protection. These findings help explain how future AI systems can be designed to be both effective and safer for sensitive data.