General Synthetic-Powered Inference

The rapid proliferation of high-quality synthetic data---generated by advanced AI models or collected as auxiliary data from related tasks---presents both opportunities and challenges for statistical inference. This paper introduces a GEneral Synthetic-Powered Inference (GESPI) framework that wraps around any statistical inference procedure to safely enhance sample efficiency by combining synthetic and real data. Our framework leverages high-quality synthetic data to boost statistical power, yet adaptively defaults to the standard method using only real data when synthetic data are of low quality. The error rate of our method remains below a user-specified bound without any distributional assumptions on the synthetic data, and decreases as the quality of the synthetic data improves. This flexibility enables seamless integration with conformal prediction, risk control, hypothesis testing, and multiple testing procedures, all without modifying the base inference method. We demonstrate the benefits of our method on challenging tasks with limited labeled data, including AlphaFold protein structure prediction, and comparing large reasoning models on complex math problems.

Making reliable decisions from data is crucial across science, medicine, and technology, yet collecting real-world data is often expensive, difficult, or inherently limited---for example, when studying rare populations. When only a small amount of real data is available, statistical methods can become less informative and more unstable. At the same time, synthetic data---whether generated by modern AI systems or gathered from related tasks---can be abundant and useful, but cannot be blindly trusted because they may not accurately reflect the real world. In this work, we develop a general framework, called General Synthetic-Powered Inference (GESPI), that wraps around a broad class of statistical methods and allows them to safely leverage synthetic data. Our approach automatically adapts to the quality of the synthetic data: it improves the utility of statistical inference when the synthetic data are informative, while maintaining reliable error control regardless of the synthetic data quality. We demonstrate the benefits of our method on challenging tasks with limited real-world data, including AlphaFold protein structure prediction, and large reasoning model evaluation on complex math problems.