Optimal Domain-Aware Privacy Mechanisms for Synthetic Data Generation

Differential privacy (DP) imposes fundamental trade-offs between privacy and statistical fidelity in synthetic data generation. While access to public data has been shown to improve these trade-offs empirically, existing approaches use public data only indirectly, through pre-processing (e.g., using pre-trained generative models) or post-processing steps (e.g., matching target statistics estimated from public datasets), while relying on domain-agnostic DP mechanisms. In this work, we lay the theoretical framework to study the principled incorporation of public data into DP mechanisms themselves. We consider normalized histograms as distribution estimators and characterize the asymptotically optimal domain-aware privacy mechanism within a specific class of DP mechanisms. We introduce PubMix, a public-data-aware DP mechanism that can be used in histogram-based data synthesis pipelines. Our experiments demonstrate that PubMix significantly improves synthetic data generation quality compared to domain-agnostic privacy mechanisms.

Private synthetic data generation is widely studied as a way to preserve user privacy when using sensitive datasets for large-scale model training. The key idea is to transform a private dataset into a synthetic one that preserves useful statistical properties of the original data while providing formal differential privacy (DP) guarantees. However, DP introduces an inherent privacy–utility trade-off. To improve this trade-off, certain existing methods leverage public data from the same domain, often through pre-processing steps, such as pre-training generative models, or post-processing steps, such as matching statistics estimated from public datasets. In these approaches, however, the underlying privacy mechanism, namely how noise is injected to ensure DP, remains largely domain-agnostic. In this work, we develop a theoretical framework for incorporating public data directly into the DP mechanism itself. Specifically, we study how to inject domain-aware noise guided by public data, with the goal of improving the privacy–utility trade-off in DP synthetic data generation. For this, we estimate the underlying distribution of the private dataset using a histogram, and then construct a synthetic sampling distribution by linearly mixing this private estimate with a domain-aware noise distribution. Using public data from the same domain, we characterize the asymptotically optimal domain-aware noise distribution and the corresponding optimal mixing parameter. The resulting distribution is then used to sample a pre-specified number of differentially private synthetic samples. From a theoretical stand point, this work serves as a foundational step toward developing domain-aware DP mechanisms for complex distribution estimators used in synthetic data generation. From a practical perspective, our analysis directly informs the design of PubMix, a domain-aware DP mechanism for histogram sampling that can replace standard Gaussian or Laplace mechanisms in existing histogram-based data synthesis pipelines for improved utility.