Smooth Multi-Policy Causal Effect Estimation in Longitudinal Settings

Comparative evaluation of multiple dynamic treatment policies is essential for healthcare and policy decisions, yet conventional longitudinal causal inference methods estimate each in isolation, preventing information sharing across counterfactuals. We demonstrate that this separate estimation paradigm induces a structurally uncontrolled second-order bias, inflating finite-sample variance even after standard debiasing with longitudinal targeted maximum likelihood estimation (LTMLE). To address this, we propose a policy-aware reparameterization of Iterative Conditional Expectation (ICE) Q-functions that enables joint estimation through shared representations. We implement this approach in the Policy-Encoded Q Network (PEQ-Net), an architecture centered on a shared policy encoder. The encoder is trained using kernel mean embeddings, ensuring that the learned representation space reflects population-level policy dissimilarities. After applying an LTMLE correction step, we prove this design imposes a structural constraint on the second-order remainder, thereby stabilizing finite-sample variance. Experiments on semi-synthetic datasets demonstrate that PEQ-Net consistently outperforms existing ICE-based methods, achieving substantial reductions in root-mean-square error, particularly when evaluating closely related policies.

In clinical settings, treatment decisions were repeatedly made based on a patient’s evolving condition. Accordingly, causal effect estimation using observed data requires repeatedly adjusting for confounding over time. Iterative G-computation is a widely used statistical framework for this problem, and recent work has combined it with deep learning. However, existing methods typically train a separate model for each treatment policy, ignoring similarities between related policies and introducing unnecessary estimation error. We propose a unified framework that estimates multiple treatment strategies jointly by explicitly treating the policy itself as a model input. Our method measures similarities between treatment policies and maps them into low-dimensional embeddings, which are then fed into the model. By sharing information across related policies, it improves the accuracy of longitudinal causal effect estimation.