Timezone: »
Gradient-based methods for two-player games produce rich dynamics that can solve challenging problems, yet can be difficult to stabilize and understand. Part of this complexity originates from the discrete update steps given by simultaneous or alternating gradient descent, which causes each player to drift away from the continuous gradient flow -- a phenomenon we call discretization drift. Using backward error analysis, we derive modified continuous dynamical systems that closely follow the discrete dynamics. These modified dynamics provide an insight into the notorious challenges associated with zero-sum games, including Generative Adversarial Networks. In particular, we identify distinct components of the discretization drift that can alter performance and in some cases destabilize the game. Finally, quantifying discretization drift allows us to identify regularizers that explicitly cancel harmful forms of drift or strengthen beneficial forms of drift, and thus improve performance of GAN training.
Author Information
Mihaela Rosca (DeepMind, UCL)
Yan Wu (DeepMind)
Benoit Dherin (Google)
David GT Barrett (DeepMind)
Related Events (a corresponding poster, oral, or spotlight)
-
2021 Spotlight: Discretization Drift in Two-Player Games »
Fri. Jul 23rd 03:30 -- 03:35 AM Room
More from the Same Authors
-
2022 : Predicting generalization with degrees of freedom in neural networks »
Erin Grant · Yan Wu -
2023 : Investigating the Edge of Stability Phenomenon in Reinforcement Learning »
Rares Iordan · Mihaela Rosca · Marc Deisenroth -
2023 : Implicit regularisation in stochastic gradient descent: from single-objective to two-player games »
Mihaela Rosca · Marc Deisenroth -
2023 : Morse Neural Networks for Uncertainty Quantification »
Benoit Dherin · Huiyi Hu · JIE REN · Michael Dusenberry · Balaji Lakshminarayanan -
2023 : A margin-based multiclass generalization bound via geometric complexity »
Michael Munn · Benoit Dherin · Xavi Gonzalvo -
2022 Workshop: Continuous Time Perspectives in Machine Learning »
Mihaela Rosca · Chongli Qin · Julien Mairal · Marc Deisenroth -
2021 Poster: Spectral Normalisation for Deep Reinforcement Learning: An Optimisation Perspective »
Florin Gogianu · Tudor Berariu · Mihaela Rosca · Claudia Clopath · Lucian Busoniu · Razvan Pascanu -
2021 Spotlight: Spectral Normalisation for Deep Reinforcement Learning: An Optimisation Perspective »
Florin Gogianu · Tudor Berariu · Mihaela Rosca · Claudia Clopath · Lucian Busoniu · Razvan Pascanu -
2020 Poster: An Explicitly Relational Neural Network Architecture »
Murray Shanahan · Kyriacos Nikiforou · Antonia Creswell · Christos Kaplanis · David GT Barrett · Marta Garnelo -
2019 Poster: Deep Compressed Sensing »
Yan Wu · Mihaela Rosca · Timothy Lillicrap -
2019 Oral: Deep Compressed Sensing »
Yan Wu · Mihaela Rosca · Timothy Lillicrap -
2018 Poster: Measuring abstract reasoning in neural networks »
Adam Santoro · Feilx Hill · David GT Barrett · Ari S Morcos · Timothy Lillicrap -
2018 Oral: Measuring abstract reasoning in neural networks »
Adam Santoro · Feilx Hill · David GT Barrett · Ari S Morcos · Timothy Lillicrap -
2018 Poster: Learning Implicit Generative Models with the Method of Learned Moments »
Suman Ravuri · Shakir Mohamed · Mihaela Rosca · Oriol Vinyals -
2018 Oral: Learning Implicit Generative Models with the Method of Learned Moments »
Suman Ravuri · Shakir Mohamed · Mihaela Rosca · Oriol Vinyals -
2017 Poster: Cognitive Psychology for Deep Neural Networks: A Shape Bias Case Study »
Samuel Ritter · David GT Barrett · Adam Santoro · Matthew Botvinick -
2017 Talk: Cognitive Psychology for Deep Neural Networks: A Shape Bias Case Study »
Samuel Ritter · David GT Barrett · Adam Santoro · Matthew Botvinick