Timezone: »

 
Poster
Stochastic Flows and Geometric Optimization on the Orthogonal Group
Krzysztof Choromanski · David Cheikhi · Jared Quincy Davis · Valerii Likhosherstov · Achille Nazaret · Achraf Bahamou · Xingyou Song · Mrugank Akarte · Jack Parker-Holder · Jacob Bergquist · Yuan Gao · Aldo Pacchiano · Tamas Sarlos · Adrian Weller · Vikas Sindhwani

Thu Jul 16 06:00 AM -- 06:45 AM & Thu Jul 16 05:00 PM -- 05:45 PM (PDT) @ Virtual
in Poster Session 41 »

We present a new class of stochastic, geometrically-driven optimization algorithms on the orthogonal group O(d) and naturally reductive homogeneous manifolds obtained from the action of the rotation group SO(d). We theoretically and experimentally demonstrate that our methods can be applied in various fields of machine learning including deep, convolutional and recurrent neural networks, reinforcement learning, normalizing flows and metric learning. We show an intriguing connection between efficient stochastic optimization on the orthogonal group and graph theory (e.g. matching problem, partition functions over graphs, graph-coloring). We leverage the theory of Lie groups and provide theoretical results for the designed class of algorithms. We demonstrate broad applicability of our methods by showing strong performance on the seemingly unrelated tasks of learning world models to obtain stable policies for the most difficult Humanoid agent from OpenAI Gym and improving convolutional neural networks.

Keywords: Approximate Inference · Non-convex Optimization · Optimization - Non-convex
[ Slides] 

Author Information

Krzysztof Choromanski (Google Brain Robotics)
David Cheikhi (Columbia University)
Jared Quincy Davis (DeepMind & Stanford University)
Valerii Likhosherstov (University of Cambridge)
Achille Nazaret (Columbia University)
Achraf Bahamou (Columbia University)
Xingyou Song (Google Brain)
Mrugank Akarte (Columbia University)
Jack Parker-Holder (University of Oxford)
Jacob Bergquist (Columbia University)
Yuan Gao (Columbia University)

Yuan studied Applied Mathematics and Statistics at National University of Singapore. He continued to pursue a PhD in Operations Research at Columbia University. He works on numerical optimization in machine learning and game theory.

Aldo Pacchiano (UC Berkeley)
Tamas Sarlos (Google)
Adrian Weller (University of Cambridge, Alan Turing Institute)
Adrian Weller

Adrian Weller is Programme Director for AI at The Alan Turing Institute, the UK national institute for data science and AI, and is a Turing AI Fellow leading work on trustworthy Machine Learning (ML). He is a Principal Research Fellow in ML at the University of Cambridge, and at the Leverhulme Centre for the Future of Intelligence where he is Programme Director for Trust and Society. His interests span AI, its commercial applications and helping to ensure beneficial outcomes for society. Previously, Adrian held senior roles in finance. He received a PhD in computer science from Columbia University, and an undergraduate degree in mathematics from Trinity College, Cambridge.

Vikas Sindhwani (Google)

More from the Same Authors