Unveiling Prior-Data Fitted Networks on Causal Effect Estimation: Pre-Training or Fine-Tuning?

Amortized causal inference via Prior-data Fitted Networks (PFNs) has emerged as a promising paradigm, enabling zero-shot estimation of causal effects without the need for dataset-specific model tuning. However, the principled effectiveness of unified pre-training across general interventional regimes remains an underexplored question. In this paper, we investigate interventions on subsets of variables within Structural Causal Models (SCMs) and identify a fundamental theoretical limitation of current pre-training approaches. Theoretically, we prove that a single observational SCM induces an exponentially large space of interventional distributions, resulting in a phenomenon we term prior uncoverage. Consequently, this uncoverage yields a mismatch between the learned meta-prior and the true grounding prior, leading to unavoidable posterior inconsistency and estimation bias. To address this, we posit that fine-tuning is a fundamental necessity and propose a target-specific strategy named Point-Wise Interventional Fine-tuning (PWF), enabling the local generalization property. We further scale this approach via Meta-Sampling Fine-tuning (MSF) from a budgeted active learning perspective, thereby achieving uniform generalization on any interventional distribution.

Many decisions ask what would happen if we changed a policy, treatment, or action. Recent AI models try to answer such questions quickly using knowledge learned in advance, but we show that this can fail when the target action was not well covered during training. To address this, we propose two additional training strategies that adapt the model to relevant actions: one focuses on a specific action, while the other covers a small but diverse set of actions. Experiments on simulated and real-world data show that these strategies reduce errors and improve the reliability of reusable causal AI models.