A Graph Foundation Model with Cross-Modal Alignment and Modality-Aware Expert Fusion for Multi-Modal Graphs

Graph Foundation Models (GFMs) aim to learn universal patterns through large-scale pretraining on diverse graphs and generalize to open-world scenarios. While GFMs have garnered significant attention, existing works primarily focus on sigle-modal graphs. However, many real-world graphs are multimodal, consisting of structures alongside diverse features derived from modalities such as text and images. To date, exploration into Multimodal Graph Foundation Models (MGFMs) remains limited. Incorporating multimodal data provides a more comprehensive view, allowing models to learn richer semantics, thereby advancing GFMs. We are therefore motivated to explore MGFMs, where the core challenge lies in synergistically encoding structures and multimodal features to achieve effective cross-modal alignment and fusion. To this end, we propose a graph foundation model with Cross-modal Alignment and Modality-aware Expert fusion, CAME. Specifically, CAME first generates graph embeddings for each individual modality. We then introduce a multimodal multi-expert encoding mechanism, which includes a dimension-wise routing strategy to fuse multimodal information. Finally, we employ a cross-modal contrastive loss to train CAME, enabling the adaptive alignment and fusion across different modalities. Extensive experiments demonstrate the effectiveness of CAME across multiple tasks and diverse multimodal graph datasets.

Many real-world networks contain rich information beyond their connections. For example, products may have titles and images, books may have descriptions and covers, and online items may be connected through user behavior. Existing graph learning methods often do not fully use these different sources of information together. We introduce CAME, a method for learning from graphs with both text and images. CAME first processes each type of information separately with the graph structure, then adaptively combines them so that useful text and visual signals can support each other. It also encourages the text, image, and combined representations of the same item to agree with one another. Experiments show that CAME performs well across different graph datasets and tasks, including cases where the model is transferred to new domains. This shows that combining text, images, and graph structure can make graph learning models more general and reliable.