CausalTab: Pretraining Across Causal Environments for Tabular Causal Discovery

Tabular causal discovery seeks directed structure from observational and optionally interventional samples. Causal discovery foundation models amortize this problem by mapping a dataset---values plus optional intervention masks---to graph hypotheses in one forward pass, yet often trail strong classical methods when pretraining coverage is narrow. We introduce CausalTab, a structured-tabular model with axial attention over variables and samples and asymmetric edge scoring, trained under broad causal pretraining over diverse graph priors, mechanisms, noise laws, dimensionalities, sample budgets, and observational versus mixed-interventional layouts, composed dynamically into episodes. We evaluate on seven synthetic benchmark families and a complementary semantic SCM benchmark with authored domains (still simulator-grounded). CausalTab achieves strong aggregate recovery with the largest gains when interventional evidence is available; a qualitative PCA on an administrative-workflow scenario suggests embeddings partially separate interpretable workflow roles, complementing synthetic scores. Together, these results highlight environment-rich pretraining as a practical ingredient for amortized causal discovery.