Toward Scalable and Valid Conditional Independence Testing with Spectral Representations

Conditional independence (CI) is central to causal inference, feature selection, and graphical modeling, yet it is untestable in many settings without additional assumptions. Existing CI tests often rely on restrictive structural conditions, limiting their validity. Kernel methods using partial covariance operators offer a more principled approach but suffer from limited adaptivity and scalability. In this work, we explore whether representation learning can help address these limitations. Specifically, we focus on representations derived from the singular value decomposition of partial covariance operators and use them to construct a simple test statistic. We also introduce a bi-level contrastive algorithm to learn these representations. Our theory links representation learning error to test performance and establishes asymptotic validity and power guarantees. Experiments on real and synthetic data suggest that this approach offers a principled and statistically grounded path toward scalable CI testing, bridging kernel-based theory with modern representation learning.

Many machine learning and scientific problems ask whether two quantities are still related after accounting for other information, such as whether a medical measurement is useful once age and prior diagnoses are known. Answering this reliably is difficult: existing tests often require strong assumptions or become too slow for large datasets. We propose a new approach that learns compact representations of the remaining relationship between variables after accounting for other factors, then uses these representations to build a simple statistical test. We also develop theory showing when the test is valid and when it can detect real relationships. Our experiments suggest that this approach can make reliable conditional-independence testing more practical for modern machine learning applications.