Timezone: »

 
Riemannian Residual Neural Networks
Isay Katsman · Eric Chen · Sidhanth Holalkere · Aaron Lou · Ser Nam Lim · Christopher De Sa

Fri Jul 22 01:45 PM -- 03:00 PM (PDT) @
in Topology, Algebra, and Geometry in Machine Learning (TAG-ML) »

Recent methods in geometric deep learning have introduced various neural networks to operate over data that lie on Riemannian manifolds. These methods are often inspired by and directly generalize standard Euclidean neural networks. However, extending Euclidean networks is difficult and has only been done for a select few manifolds. In this work, we examine the residual neural network (ResNet) and show how to extend this construction to general Riemannian manifolds in a geometrically principled manner. Originally introduced to help solve the vanishing gradient problem, ResNets have become ubiquitous in machine learning due to their beneficial learning properties, excellent empirical results, and easy-to-incorporate nature when building varied neural networks. We find that our Riemannian ResNets mirror these desirable properties: when compared to existing manifold neural networks designed to learn over hyperbolic space and the manifold of symmetric positive definite matrices, we outperform both kinds of networks in terms of relevant testing metrics and training dynamics.

Author Information

Isay Katsman (Cornell University)
Eric Chen (Cornell University)
Sidhanth Holalkere (Cornell University)
Aaron Lou (Stanford University)
Ser Nam Lim (Meta AI)
Christopher De Sa (Cornell)

More from the Same Authors