DataGuard: A Non-intrusive Dataset Auditing Framework via Differential Information Forensics

Concerns over dataset misuse in deep learning have highlighted the need for effective auditing. Unlike existing intrusive methods that require dataset modifications, which risk model performance and security, we present DataGuard, a non-intrusive framework for quantitative dataset auditing. Specifically, DataGuard integrates three key components: 1) a differential comparison between the target dataset and auxiliary non-training datasets, 2) an information-forensic analysis establishing formal inequalities to distinguish training data; and 3) a multivariate statistical test that translates these discrepancies into rigorous auditing scores. Extensive experiments demonstrate that DataGuard can detect both full and partial dataset usage without false positives while remaining robust under diverse training scenarios, offering a principled, information-theoretic solution for transparent AI development.

Data owners often release valuable datasets for research or commercial use, but they have limited ways to check whether a machine learning model was trained on their data without permission. This problem is difficult because most existing auditing methods require data owners to modify their datasets before release, such as by adding hidden watermarks, which can reduce data quality, affect model behavior, and create security risks. We address this problem with DataGuard, a dataset auditing method that does not change the protected dataset. DataGuard queries a suspect model and compares how it responds to the protected dataset versus similar datasets that are known not to have been used for training. If the model was trained on the protected dataset, its responses tend to leave a different information pattern from its responses to unseen data. DataGuard turns this difference into a statistical auditing score that indicates whether the protected dataset was likely used. This matters because it gives data owners a practical way to audit possible unauthorized dataset use, including cases where only part of the dataset was used. By avoiding dataset modification, DataGuard can also help audit datasets that have already been released and reduce the performance and security risks of watermark-based auditing.