Timezone: »

 
Sign and Basis Invariant Networks for Spectral Graph Representation Learning
Derek Lim · Joshua Robinson · Lingxiao Zhao · Tess Smidt · Suvrit Sra · Haggai Maron · Stefanie Jegelka

Fri Jul 22 01:45 PM -- 03:00 PM (PDT) @
in Topology, Algebra, and Geometry in Machine Learning (TAG-ML) »

We introduce SignNet and BasisNet---new neural architectures that are invariant to two key symmetries displayed by eigenvectors: (i) sign flips, since if v is an eigenvector then so is -v; and (ii) more general basis symmetries, which occur in higher dimensional eigenspaces with infinitely many choices of basis eigenvectors. We prove that our networks are universal, i.e., they can approximate any continuous function of eigenvectors with proper invariances. When used with Laplacian eigenvectors, our architectures are also theoretically expressive for graph representation learning, in that they can approximate any spectral graph convolution, can compute spectral invariants that go beyond message passing neural networks, and can provably simulate previously proposed graph positional encodings. Experiments show the strength of our networks for processing geometric data, in tasks including: molecular graph regression, learning expressive graph representations, and learning neural fields on triangle meshes.

Author Information

Derek Lim (MIT)
Joshua Robinson (MIT)

I want to understand how machines can learn useful representations of the world. I am also interested in modeling diversity and its many applications in learning problems. I am Josh Robinson, a PhD student at MIT CSAIL & LIDS advised by Stefanie Jegelka and Suvrit Sra. I am part of the MIT machine learning group. Previously I was an undergraduate at the University of Warwick where I worked with Robert MacKay on probability theory.

Lingxiao Zhao (Carnegie Mellon University)
Tess Smidt (Massachusetts Institute of Technology)
Suvrit Sra (MIT & Macro-Eyes)
Haggai Maron
Stefanie Jegelka (Massachusetts Institute of Technology)

More from the Same Authors