Timezone: »

 
Expressive Sign Equivariant Networks for Spectral Geometric Learning
Derek Lim · Joshua Robinson · Stefanie Jegelka · Haggai Maron

Fri Jul 28 02:10 PM -- 03:00 PM (PDT) @
in 2nd Annual Workshop on Topology, Algebra, and Geometry in Machine Learning (TAG-ML) »
Event URL: https://openreview.net/forum?id=B7RvoCPUi0 »
Recent work has shown the utility of developing machine learning models that respect the structure and symmetries of eigenvectors. These works promote sign invariance, since for any eigenvector $v$ the negation $-v$ is also an eigenvector. However, we show that sign invariance is theoretically limited for tasks such as building orthogonally equivariant models and learning node positional encodings for link prediction in graphs. In this work, we demonstrate the benefits of sign equivariance for these tasks. To obtain these benefits, we develop novel sign equivariant neural network architectures. Our models are based on a new analytic characterization of sign equivariant polynomials and thus inherit provable expressiveness properties. Controlled synthetic experiments show that our networks can achieve the theoretically predicted benefits of sign equivariant models.

Author Information

Derek Lim (MIT)
Joshua Robinson (MIT)

I want to understand how machines can learn useful representations of the world. I am also interested in modeling diversity and its many applications in learning problems. I am Josh Robinson, a PhD student at MIT CSAIL & LIDS advised by Stefanie Jegelka and Suvrit Sra. I am part of the MIT machine learning group. Previously I was an undergraduate at the University of Warwick where I worked with Robert MacKay on probability theory.

Stefanie Jegelka (Massachusetts Institute of Technology)
Haggai Maron (NVIDIA Research)

I am a Research Scientist at NVIDIA Research. My main fields of interest are machine learning, optimization, and shape analysis. More specifically, I am working on applying deep learning to irregular domains (e.g., graphs, point clouds, and surfaces) and graph/shape matching problems. I completed my Ph.D. in 2019 at the Department of Computer Science and Applied Mathematics at the Weizmann Institute of Science under the supervision of Prof. Yaron Lipman.

More from the Same Authors