GraphPFN: A Prior-Data Fitted Graph Foundation Model

Graph foundation models face several fundamental challenges including transferability across diverse domains and data scarcity, which calls into question the very feasibility of creating such models. However, despite similar challenges, the tabular domain has recently witnessed the emergence of the first successful foundation models such as TabPFN. These models are based on the prior-data fitted networks (PFN) framework, in which models are pretrained on carefully designed synthetic datasets to make predictions in an in-context learning setting. Recently, G2T-FM, a framework that converts graph node-level tasks into tabular tasks, has made the first step towards adopting PFNs for graphs, yet it is limited to hand-crafted features and was never pretrained on graph data. In this work, we make the next step by proposing GraphPFN, a PFN-based model designed and pretrained specifically for graph node-level tasks. Following the PFN framework, we first design a prior distribution of synthetic attributed graphs by using a novel combination of multi-level stochastic block models and a preferential attachment process for structure generation and graph-aware structured causal models for attribute generation. Then, we augment the tabular foundation model LimiX with attention-based graph neighborhood aggregation layers and train it on millions of synthetic graphs sampled from our prior. On diverse real-world graph datasets with node-level tasks, GraphPFN achieves state-of-the-art results in both in-context learning and finetuning regimes, outperforming G2T-FM, prior GFMs, and task-specific GNNs trained from scratch. More broadly, GraphPFN shows the potential of PFN-based models for building graph foundation models. Our code is available at https://github.com/yandex-research/graphpfn.

Graphs are a common way to represent relationships, such as friendships in a social network, links between webpages, or connections between molecules. Building “foundation models” for graphs — models that can work well across many different graph problems — is difficult because graph data varies a lot across domains and there is simply much less data available to learn from compared to text or images. In this work, we introduce GraphPFN, a new foundation model for graph tasks. Our approach is inspired by recent successes in tabular data, where models are trained on many synthetic datasets so they can solve new tasks instantly, without any extra training. To bring this idea to graphs, we create millions of realistic synthetic graphs with both connections and node attributes, using processes that mimic community structure, popular-node effects, and cause-and-effect relationships in features. We then train GraphPFN to solve node-level prediction tasks on these synthetic graphs. Unlike earlier approaches that convert graphs into tables using hand-designed features, GraphPFN directly uses graph structure through neighborhood aggregation layers. On a wide range of real-world graph benchmarks, GraphPFN performs better than previous graph foundation models and even task-specific graph neural networks trained from scratch. These results suggest that synthetic pretraining may be a promising path toward general-purpose graph learning models.