Timezone: »

 
Poster
GOAT: A Global Transformer on Large-scale Graphs
Kezhi Kong · Jiuhai Chen · John Kirchenbauer · Renkun Ni · C. Bayan Bruss · Tom Goldstein

Wed Jul 26 05:00 PM -- 06:30 PM (PDT) @ Exhibit Hall 1 #509
in Poster Session 4 »

Graph transformers have been competitive on graph classification tasks, but they fail to outperform Graph Neural Networks (GNNs) on node classification, which is a common task performed on large-scale graphs for industrial applications. Meanwhile, existing GNN architectures are limited in their ability to perform equally well on both homophilious and heterophilious graphs as their inductive biases are generally tailored to only one setting. To address these issues, we propose GOAT, a scalable global graph transformer. In GOAT, each node conceptually attends to all the nodes in the graph and homophily/heterophily relationships can be learnt adaptively from the data. We provide theoretical justification for our approximate global self-attention scheme, and show it to be scalable to large-scale graphs. We demonstrate the competitiveness of GOAT on both heterophilious and homophilious graphs with millions of nodes.

[ PDF]  [ Poster] 

Author Information

Kezhi Kong (University of Maryland, College Park)
Jiuhai Chen (University of Maryland)
John Kirchenbauer (University of Maryland, College Park)
John Kirchenbauer

Researcher at the Center for Machine Learning (ml.umd.edu, umiacs.umd.edu), advised by Professor Tom Goldstein My current research focuses on understanding various aspects of model behavior including robustness, reliability, and efficiency in the discrete data domains of both natural language and graphs. I am interested in understanding how deep learning models generalize under distribution shift from an empirical and theoretical perspective through dataset curation, architectural changes, and novel analysis techniques. One overarching goal is the development of principled yet practical definitions of what "In" and "Out-of" distribution means in practice for these data domains.

Renkun Ni (University of Maryland)
C. Bayan Bruss (Capital One)
Tom Goldstein (University of Maryland)

More from the Same Authors