TabMGP: Martingale Posterior with TabPFN

Bayesian inference provides principled uncertainty quantification but is often limited by the challenges of prior and likelihood elicitation. The martingale posterior (MGP) (Fong et al., 2023) offers an alternative by replacing these requirements with a predictive rule. In addition, the MGP focuses inference on parameters defined through a loss function. This framework is especially resonant in the era of foundation transformers; practitioners increasingly leverage models like TabPFN for their state-of-the-art capabilities, yet often require epistemic uncertainty for a scientific estimand $\theta$ that need not parameterise the implicit latent model. The MGP provides a mechanism to recover these posterior distributions. We introduce TabMGP, an MGP built on TabPFN for tabular data. TabMGP produces credible sets with near-nominal coverage and often outperforms both handcrafted MGP constructions and standard Bayesian baselines.

Machine-learning tools can make strong predictions, but scientists often also need to know how uncertain they should be about a quantity of interest. Traditional statistical methods can provide this uncertainty, but they often require users to choose a full probability model, a choice that can be difficult, fragile, or expensive. We introduce TabMGP, a way to turn TabPFN, a pretrained transformer model for tabular prediction, into a tool for uncertainty quantification. TabMGP asks TabPFN to predict plausible continuations of the observed dataset, computes the scientific quantity of interest on each completed dataset, and uses the resulting variation as an uncertainty estimate. This gives users a set of plausible values for the quantity of interest without the laborious work of hand-designing a full probability model. Across synthetic and real-world tabular datasets, TabMGP usually produced reliable and meaningful uncertainty estimates. Overall, TabMGP shows how pretrained tabular models can support not only prediction, but also practical scientific uncertainty quantification.