Provable Training Data Identification for Large Language Models

Identifying training data of large-scale models is critical for copyright litigation, privacy auditing, and ensuring fair evaluation. However, existing works typically treat this task as an instance-wise identification without controlling the error rate of the identified set, which cannot provide statistically reliable evidence. In this work, we formalize training data identification as a set-level inference problem and propose Provable Training Data Identification (PTDI), a distribution-free approach that enables provable and strict false identification rate control. Specifically, our method computes conformal p-values for each data point using a set of known unseen data and then develops a novel Jackknife-corrected Beta boundary (JKBB) estimator to estimate the training-data proportion of the test set, which allows us to scale these p-values. By applying the Benjamini–Hochberg (BH) procedure to the scaled p-values, we select a subset of data points with provable and strict false identification control. Extensive experiments across various models and datasets demonstrate that PTDI achieves higher power than prior methods while strictly controlling the FIR.

Recent work has argued that membership inference cannot reliably prove that a particular data point was used to train a large AI model, especially when the model’s training process and data are hidden. This paper addresses that limitation by changing the goal: instead of trying to prove membership one example at a time, we identify a set of likely training examples with a guaranteed bound on the expected fraction of false identifications. Our method, PTDI, uses data known to be unseen by the model to calibrate existing detection scores, estimates the number of training examples in the candidate set, and then selects examples using a statistically controlled procedure. Experiments on language and vision-language models show that PTDI can recover more true training examples than prior methods while keeping false identifications under control, making training-data audits more credible for copyright, privacy, and benchmark-contamination cases.