Are Common Substructures Transferable? Riemannian Graph Foundation Model with Neural Vector Bundles

Foundation models have sparked a revolution via a pretraining-adaptation paradigm, with recent efforts extending this success to graphs. Unlike other modalities, graphs contain rich structural patterns, yet their structural transferability remains poorly understood. Prior studies consider common substructures in the discrete realm, and we are motivated by a fundamental question: Are common substructures transferable? The underlying theory is largely underexplored. In this work, we shift toward learning transferable structures through the lens of functional behavior. Theoretically, we connect transferable substructures to intrinsic geometry of the representation space. However, characterizing such intrinsic geometry has rarely been touched. Grounded in Riemannian geometry, we develop a graph intrinsic geometry learning framework—\textbf{Neural Vector Bundle}, which enables parsing intrinsic geometry with local coordinates. Building on this, we design \textbf{\textsc{Gauge}}, a pretrainable neural architecture that constructs the vector bundle, flattening geometrically compatible local coordinates, and a new Dirichlet loss, which also measures the transfer effort. We empirically validate its superior expressiveness in challenging tasks including zero-shot link prediction and graph isomorphism.

(1) Problem: GNNs power many AI applications, but a key question remains: when we pretrain a model on source domain graphs, which learned patterns actually transfer to unseen graphs? Understanding this "transferability" has been surprisingly difficult. (2) Solution: We introduce GAUGE, a new approach that views graphs through the lens of geometry. Instead of searching for common substructures in a discrete way, we ask: what if transferable patterns correspond to "flat" regions in a hidden geometric space? Using concepts from Riemannian geometry, we build a "neural vector bundle" that learns each graph's intrinsic shape. A novel Dirichlet loss then guides the model to focus on behaviorally stable patterns—those requiring minimal adaptation when moving to new graphs. (3) Impact: GAUGE achieves well performance on challenging tasks like zero-shot link prediction and graph isomorphism. More importantly, it offers a principled way to identify which substructures are truly transferable, helping researchers build more efficient, trustworthy graph foundation models that generalize across domains.