Timezone: »
Recent work has shown local convergence of GAN training for absolutely continuous data and generator distributions. In this paper, we show that the requirement of absolute continuity is necessary: we describe a simple yet prototypical counterexample showing that in the more realistic case of distributions that are not absolutely continuous, unregularized GAN training is not always convergent. Furthermore, we discuss regularization strategies that were recently proposed to stabilize GAN training. Our analysis shows that GAN training with instance noise or zero-centered gradient penalties converges. On the other hand, we show that Wasserstein-GANs and WGAN-GP with a finite number of discriminator updates per generator update do not always converge to the equilibrium point. We discuss these results, leading us to a new explanation for the stability problems of GAN training. Based on our analysis, we extend our convergence results to more general GANs and prove local convergence for simplified gradient penalties even if the generator and data distributions lie on lower dimensional manifolds. We find these penalties to work well in practice and use them to learn high-resolution generative image models for a variety of datasets with little hyperparameter tuning.
Author Information
Lars Mescheder (MPI Tübingen)
Andreas Geiger (MPI-IS and University of Tuebingen)
Sebastian Nowozin (Microsoft Research)
Related Events (a corresponding poster, oral, or spotlight)
-
2018 Oral: Which Training Methods for GANs do actually Converge? »
Wed. Jul 11th 02:00 -- 02:20 PM Room A7
More from the Same Authors
-
2017 Poster: Adversarial Variational Bayes: Unifying Variational Autoencoders and Generative Adversarial Networks »
Lars Mescheder · Sebastian Nowozin · Andreas Geiger -
2017 Talk: Adversarial Variational Bayes: Unifying Variational Autoencoders and Generative Adversarial Networks »
Lars Mescheder · Sebastian Nowozin · Andreas Geiger