Relational In-Context Learning via Synthetic Pre-training with Structural Prior

Relational Databases (RDBs) are the backbone of modern business, yet they lack foundation models comparable to those in text or vision. A key obstacle is that high-quality RDBs are private, scarce, and structurally heterogeneous, making internet-scale pre-training infeasible. To overcome this data scarcity, we introduce RDB-PFN, the first relational foundation model trained purely via synthetic data. Inspired by Prior-Data Fitted Networks (PFNs), where synthetic data generated from Structural Causal Models (SCMs) enables reasoning on single tables, we design a Relational Prior Generator to create an infinite stream of diverse RDBs from scratch. Pre-training on over 2 million synthetic single-table and relational tasks, RDB-PFN learns to adapt to any new database instantly via genuine in-context learning. Experiments show that RDB-PFN achieves strong few-shot performance on 19 real-world relational prediction tasks, outperforming state-of-the-art tabular foundation models evaluated on the same DFS-linearized inputs, while using a lightweight architecture and fast inference. The code is available at https://github.com/MuLabPKU/RDBPFN.

Many important datasets used by businesses, hospitals, and online services are stored as relational databases: collections of linked tables such as users, orders, products, and clicks. While foundation models have revolutionized text and images, building similar models for relational databases is difficult because real databases are often private, scarce, and organized in very different ways. We introduce RDB-PFN, a model that learns from millions of fully synthetic relational databases rather than relying on large collections of real private data. Our method generates artificial databases with realistic table structures, links between records, and patterns across connected tables, then trains a model to make predictions from a small set of labeled examples. At test time, the model can adapt to a new database in one forward pass, without fine-tuning. Across 19 real-world relational prediction tasks, RDB-PFN performs strongly when only a small number of labeled examples are available, while remaining small and fast. This suggests that carefully designed synthetic databases can help bring foundation model learning to relational data.