Learning $U$-Statistics with Active Inference

$U$-statistics play a central role in statistical inference. In many modern applications, however, acquiring the labels required for $U$-statistics is costly. Motivated by recent advances in active inference, we develop an active inference framework for $U$-statistics that selectively queries informative labels to improve estimation efficiency under a fixed labeling budget, while preserving valid statistical inference. Our approach is built on the augmented inverse probability weighting $U$-statistic, which is proposed to incorporate the sampling rule and machine learning predictions. We characterize the optimal sampling rule that minimizes its variance and design practical sampling strategies. We further extend the framework to $U$-statistic-based empirical risk minimization. Experiments on real datasets demonstrate substantial gains in estimation efficiency over baseline methods, while maintaining target coverage.

Many statistical studies rely on U-statistics, a broad class of tools used to measure quantities such as income inequality, ranking performance, or dependence between variables. For example, the Gini index is a U-statistic commonly used to measure income inequality. In many modern applications, however, the labels or outcomes needed to compute these quantities are expensive or time-consuming to obtain. This paper studies how to estimate U-statistics accurately under a limited labeling budget. Our method uses machine learning predictions in two complementary ways: it first builds a prediction-based estimate from all available data, and then uses the predictions to select the most informative data points for labeling. By combining predictions with actively collected labels, our method obtains more reliable estimates using fewer labels and produces uncertainty ranges for the final estimate.