Timezone: »

 
Poster
Implicit Euler Skip Connections: Enhancing Adversarial Robustness via Numerical Stability
Mingjie Li · Lingshen He · Zhouchen Lin

Wed Jul 15 09:00 AM -- 09:45 AM & Wed Jul 15 08:00 PM -- 08:45 PM (PDT) @
in Poster Session 23 »

Deep neural networks have achieved great success in various areas, but recent works have found that neural networks are vulnerable to adversarial attacks, which leads to a hot topic nowadays. Although many approaches have been proposed to enhance the robustness of neural networks, few of them explored robust architectures for neural networks. On this account, we try to address such an issue from the perspective of dynamic system in this work. By viewing ResNet as an explicit Euler discretization of an ordinary differential equation~(ODE), for the first time, we find that the adversarial robustness of ResNet is connected to the numerical stability of the corresponding dynamic system, i.e., more stable numerical schemes may correspond to more robust deep networks. Furthermore, inspired by the implicit Euler method for solving numerical ODE problems, we propose Implicit Euler skip connections~(IE-Skips) by modifying the original skip connection in ResNet or its variants. Then we theoretically prove its advantages under the adversarial attack and the experimental results show that our ResNet with IE-Skips can largely improve the robustness and the generalization ability under adversarial attacks when compared with the vanilla ResNet of the same parameter size.

Keywords: Adversarial Examples · Architectures · Supervised Learning · Deep Learning - General

Author Information

Mingjie Li (Peking University)
Lingshen He (Peking University)
Zhouchen Lin (Peking University)

More from the Same Authors