Prediction-Powered Adaptive Inference with Pretrained AI Models for Contextual Bandits

In adaptive experiments, statistical inference is essential for reliable decision-making and scientific discovery. Often in these settings, collecting labeled data is expensive, but decision-makers have access to large unlabeled datasets and strong pretrained AI models that can generate outcome predictions. Effectively leveraging these predictions in online experiments poses fundamental challenges for statistical inference: AI models may be misspecified, and data collected under adaptive policies are inherently non-i.i.d., invalidating classical inference techniques. To address these challenges, we propose a Prediction-Powered Adaptive Inference (PPAI) estimator that integrates unlabeled data, predicted labels, and adaptively collected labeled data through a single estimating equation. We establish asymptotic normality of the PPAI estimator under mild conditions on the data-collection policy, enabling valid confidence intervals and hypothesis tests for a broad class of Z-functionals. The method incorporates a data-driven tuning mechanism that adaptively weights AI predictions according to their informativeness, guaranteeing that the resulting asymptotic variance is no worse than that of the labeled-only baseline, and is strictly smaller when predictions are informative. Numerical experiments further support the theory, illustrating efficiency gains with informative AI predictions and robust performance when predictions are inaccurate.

To evaluate new drugs, app features, and educational interventions, researchers often use adaptive experiments, adjusting their decision-making strategy as data come in. To draw trustworthy conclusions, they need lots of labeled data, but collecting real-world outcomes in adaptive experiments is typically slow or expensive. Fortunately, they often have abundant unlabeled data and AI models that can cheaply predict outcomes. Can we use these AI predictions to reach reliable conclusions faster? It's challenging because the AI can be wrong and because adaptive experiments violate the assumptions underlying classical statistics. We develop a statistical framework that uses a small set of real-world outcomes from an adaptive experiment, unlabeled data, and AI predictions on unlabeled data. Our method automatically trusts the AI less when its predictions are inaccurate. We prove mathematically that our method provides substantial improvement when the AI predictions are accurate and always performs at least as well as ignoring the AI predictions. Our framework enables researchers to draw trustworthy conclusions from adaptive experiments with less data. This can accelerate and reduce the cost of adaptive experiments in many domains, such as clinical trials, mobile health, and recommendation systems.