Game Theoretic Neural ODE Optimizer

In this work, we present a novel Game Theoretic Neural Ordinary Differential Equation (Neural ODE) optimizer based on the minimax Differential Dynamic Programming paradigm. As neural networks and neural ODEs tend to be vulnerable to attacks, and their predictions are fragile in the presence of adversarial examples, we aim to design a robust game theoretic optimizer based on principles of Min-Max Optimal Control. By formulating Neural ODE optimization as a Min-Max Optimal Control Problem, our proposed algorithm aims to enhance the robustness of neural networks against adversarial attacks by finding policies that perform well under worst-case scenarios. Leveraging recent advances in the interpretation of Neural ODE training through an Optimal Control Problem perspective, we extend recent second order optimization techniques to a game theoretic setting and adapt them to our proposed method. This allows our optimizer toefficiently handle the increased complexity stemming from the computation of double the amount of learnable parameters. The resulting optimizer, Game Theoretic Second-Order Neural Optimizer (GTSONO), enables more effective exploration of the control policy space, leading to improved robustness against adversarial attacks. Experimental evaluations on benchmark datasets demonstrate the superiority of GTSONO compared to existing state-of-the-art optimizers in terms of both performance and efficiency against state-of-the-artadversarial defense methods.