Measuring Variable Importance in Heterogeneous Treatment Effects with Confidence

Causal machine learning (ML) promises to provide powerful tools for estimating individual treatment effects. While causal methods have placed some emphasis on heterogeneity in treatment response, it is of paramount importance to clarify the nature of this heterogeneity, by highlighting which variables drive it.We propose PermuCATE, an algorithm based on the Conditional Permutation Importance (CPI) method, for statistically rigorous global variable importance assessment in the estimation of the Conditional Average Treatment Effect (CATE).Theoretical analysis of the finite sample regime and empirical studies show that PermuCATE has lower variance than the Leave-One-Covariate-Out (LOCO) method and provides a reliable measure of variable importance.This property increases statistical power, which is crucial for causal inference applications with finite sample sizes.We empirically demonstrate the benefits of PermuCATE in simulated and real datasets, including complex settings with high-dimensional, correlated variables.

The same treatment can have varying effects across different individuals. For example, medicines may be more effective for certain subgroups within a population, while potentially causing adverse events in others. Although precision medicine has advanced in predicting how treatments will work for individuals, understanding why these differences occur remains a challenge.Our work tackles this problem by exploring methods that identify which characteristics of a population contribute to this heterogeneity in treatment effects. We focus on methods that reduce the risk of false discoveries—incorrectly suggesting that a characteristic is important when it is not. Additionally, we consider approaches suitable for situations with limited data, which is often the case in medical research.Our findings offer guidance on selecting the most suitable method for understanding what drives heterogeneity in treatment effects. This could lead to better insights into risk factors for adverse reactions and help in prevention by pinpointing characteristics linked to improved health outcomes for specific groups.