A Dynamical System Perspective for Lipschitz Neural Networks

Laurent Meunier · Blaise Delattre · Alexandre ARAUJO · Alexandre Allauzen

Room 318 - 320
[ Abstract ] [ Livestream: Visit DL: Robustness ]
Wed 20 Jul 10:50 a.m. — 11:10 a.m. PDT
[ Slides [ Paper PDF

The Lipschitz constant of neural networks has been established as a key quantity to enforce the robustness to adversarial examples. In this paper, we tackle the problem of building $1$-Lipschitz Neural Networks. By studying Residual Networks from a continuous time dynamical system perspective, we provide a generic method to build $1$-Lipschitz Neural Networks and show that some previous approaches are special cases of this framework. Then, we extend this reasoning and show that ResNet flows derived from convex potentials define $1$-Lipschitz transformations, that lead us to define the {\em Convex Potential Layer} (CPL). A comprehensive set of experiments on several datasets demonstrates the scalability of our architecture and the benefits as an $\ell_2$-provable defense against adversarial examples. Our code is available at \url{https://github.com/MILES-PSL/Convex-Potential-Layer}

Chat is not available.