Timezone: »

 
Invariance Principle Meets Out-of-Distribution Generalization on Graphs
Yongqiang Chen · Yonggang Zhang · Yatao Bian · Han Yang · Kaili MA · Binghui Xie · Tongliang Liu · Bo Han · James Cheng

@
in Spurious correlations, Invariance, and Stability (SCIS) »
Event URL: https://openreview.net/forum?id=XkuqhYNYIm4 »

Despite recent success in using the invariance principle for out-of-distribution (OOD) generalization on Euclidean data (e.g., images), studies on graph data are still limited. Different from images, the complex nature of graphs poses unique challenges to adopting the invariance principle. In particular, distribution shifts on graphs can appear in a variety of forms such as attributes and structures, making it difficult to identify the invariance. Moreover, domain or environment partitions, which are often required by OOD methods on Euclidean data, could be highly expensive to obtain for graphs. To bridge this gap, we propose a new framework, called Graph Out-Of-Distribution Generalization (GOOD), to capture the invariance of graphs for guaranteed OOD generalization under various distribution shifts. Specifically, we characterize potential distribution shifts on graphs with causal models, concluding that OOD generalization on graphs is achievable when models focus only on subgraphs containing the most information about the causes of labels. Accordingly, we propose an information-theoretic objective to extract the desired subgraphs that maximally preserve the invariant intra-class information. Learning with these subgraphs is immune to distribution shifts. Extensive experiments on both synthetic and real-world datasets, including a challenging setting in AI-aided drug discovery, validate the superior OOD generalization ability of GOOD.

Keywords: Graph Neural Networks · Causal Invariance · Out-of-Distribution Generalization

Author Information

Yongqiang Chen (The Chinese University of Hong Kong)
Yonggang Zhang (Hong Kong Baptist University)
Yatao Bian (Tencent AI Lab)
Han Yang (The Chinese University of Hong Kong)
Kaili MA (CUHK)
Binghui Xie (The Chinese University of Hongkong)
Tongliang Liu (The University of Sydney)
Bo Han (HKBU / RIKEN)
James Cheng (CUHK)

More from the Same Authors