RAD-TFM: Robust and Domain-Adapted Tabular Foundation Models

Tabular foundation models (TFMs) have shown promise for structured data, but they do not consistently outperform strong traditional ML methods across datasets. Although TFMs can be pretrained entirely on synthetic data, prior work has primarily relied on fixed priors over synthetic data generators. We instead frame generator selection as an adversarial distributional robustness problem: adapt the generator distribution toward datasets with large optimality gaps, defined as the shortfall between TFM performance and the best achievable performance on that dataset. We also align synthetic generation with real-world datasets from a target domain, constraining the adversary toward realistic data. These components form the Robust and Domain-Adapted Tabular Foundation Models (RAD-TFM) pipeline, a model-agnostic adversarial training framework. Applied to TabPFN V2, RAD-TFM improves performance across six tabular benchmarks, yielding up to an 11\% increase in mean normalized AUC over TabPFN V2 and other baselines using only 100k additional synthetic datasets, less than 0.1\% of the original pretraining scale.