Semi-knockoffs: a model-agnostic conditional independence testing method with finite-sample guarantees

Conditional independence testing (CIT) is essential for reliable scientific discovery. It prevents spurious findings and enables controlled feature selection. Recent CIT methods have used machine learning (ML) models as surrogates of the underlying distribution. However, model-agnostic approaches require a train-test split, which reduces statistical power. We introduce Semi-knockoffs, a CIT method that can accommodate any pre-trained model, avoids this split, and provides valid p-values and false discovery rate (FDR) control for high-dimensional settings. Unlike methods that rely on the model-$X$ assumption (known input distribution), Semi-knockoffs only require conditional expectations for continuous variables. This makes the procedure less restrictive and more practical for machine learning integration. To ensure validity when these expectations are estimated, we present two new theoretical results: (i) stability for regularized models trained with a null feature and (ii) the double-robustness property.

Conditional independence consists of testing whether a feature is related to an outcome of interest after accounting for the other features. This is an important problem because it helps avoid false discoveries caused by correlations with truly important features. At the same time, Machine Learning models have demonstrated high predictive accuracy. However, previous methods suffered from several drawbacks, such as relying on overly simple models that cannot capture complex interactions, or being statistically inefficient due to train–test splits. In this work, we propose Semi-Knockoffs, a method that compares predictions made with the information from a feature against predictions made without it, in a way that is natural for Machine Learning models. As a result, Semi-Knockoffs enables scientific discoveries with statistical guarantees, such as control of the type-I error rate and the False Discovery Rate, while remaining compatible with any pretrained black-box Machine Learning model.