Timezone: »

 
Poster
LyaNet: A Lyapunov Framework for Training Neural ODEs
Ivan Dario Jimenez Rodriguez · Aaron Ames · Yisong Yue

Tue Jul 19 03:30 PM -- 05:30 PM (PDT) @ Hall E #310
in Poster Session 1 »

We propose a method for training ordinary differential equations by using a control-theoretic Lyapunov condition for stability. Our approach, called LyaNet, is applicable to function classes that use dynamical systems for inference, such as Neural ODEs. Our method is based on a novel Lyapunov Loss formulation that encourages the inference dynamics to converge quickly to the correct prediction, which has direct implications for robustness, and can be adapted to avoid the cost of backpropagating through a solver or using the adjoint method. Theoretically, we show that minimizing this loss guarantees (exponential) convergence to the correct solution and enables a novel robustness guarantee. Relative to standard Neural ODE training, we empirically find that LyaNet offers improved prediction performance, faster convergence of inference dynamics, and improved adversarial robustness.

Keywords: Deep Learning · DL: Algorithms · DL: Theory · OPT: Control and Optimization

Author Information

Ivan Dario Jimenez Rodriguez (California Institute of Technology)
Aaron Ames (Caltech)
Yisong Yue (Caltech)

Yisong Yue is an assistant professor in the Computing and Mathematical Sciences Department at the California Institute of Technology. He was previously a research scientist at Disney Research. Before that, he was a postdoctoral researcher in the Machine Learning Department and the iLab at Carnegie Mellon University. He received a Ph.D. from Cornell University and a B.S. from the University of Illinois at Urbana-Champaign. Yisong's research interests lie primarily in the theory and application of statistical machine learning. He is particularly interested in developing novel methods for interactive machine learning and structured prediction. In the past, his research has been applied to information retrieval, recommender systems, text classification, learning from rich user interfaces, analyzing implicit human feedback, data-driven animation, behavior analysis, sports analytics, policy learning in robotics, and adaptive planning & allocation problems.

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors