ERAlign: Energy-based Representation Alignment of GNNs and LLMs on Text-attributed Graphs

Text-attributed Graphs (TAGs) incorporate textual node attributes with graph structures to describe rich relational semantics. Recent efforts to integrate Graph Neural Networks (GNNs) and Large Language Models (LLMs) have shown promise for learning on TAGs, yet achieving well-aligned representations remains challenging. Prior studies largely rely on heuristics that perform coarse-grained matching. They lack sufficient constraints and ignore distributional alignment, leading to representation drift and limited generalization. Building on Energy-based Models (EBMs), we propose an Energy-based Representation Alignment (ERAlign) framework that projects GNN-encoded graph structure and LLM-derived text embeddings in a shared latent space to achieve distribution consistency. Concretely, layer-wise alignment is quantified by a distance metric and optimized via an EBM objective. By decreasing energy values, our framework yields well-aligned representations for downstream tasks. During training, we introduce Energy Discrepancy (ED) to avoid high sampling costs associated with intractable normalization. ED also carries theoretical guarantees of higher training efficiency and reduced energy landscape distortion. Empirical evaluations on eight TAG datasets demonstrate that ERAlign obtains state-of-the-art performance across varying levels of supervision and cross-task transfer scenarios.

Many real-world datasets are networks that combine two kinds of information, namely how things connect and what text describes. Papers cite one another and carry titles and abstracts, social media users link to each other and write profiles, and products bought together come with descriptions. To make sense of such data, a computer needs both. But the two common AI tools for the job think differently, with language models excelling at reading text and graph models at mapping connections. Therefore, getting them to agree has proven hard. Existing methods align them only at a coarse level, letting their internal pictures drift apart. We introduce an ERAlign framework that makes these two systems deeply agree. We borrow the idea of energy from physics, where lower energy indicates a more stable and preferred state. Accordingly, we assign low energy when the text meaning and graph structure align, pulling their internal representations together layer by layer. We also design an efficient training scheme that sidesteps the slow and costly calculations such methods usually require. Across eight networks, ERAlign outperforms existing approaches and works well when only a few labeled examples are available. It can even transfer to new tasks without additional training, pointing toward more reliable tools for recommendation, search, and scientific discovery.