Timezone: »

 
Poster
Is Generator Conditioning Causally Related to GAN Performance?
Augustus Odena · Jacob Buckman · Catherine Olsson · Tom B Brown · Christopher Olah · Colin Raffel · Ian Goodfellow

Thu Jul 12 09:15 AM -- 12:00 PM (PDT) @ Hall B #95
in Posters Thu »

Recent work suggests that controlling the entire distribution of Jacobian singular values is an important design consideration in deep learning. Motivated by this, we study the distribution of singular values of the Jacobian of the generator in Generative Adversarial Networks. We find that this Jacobian generally becomes ill-conditioned at the beginning of training. Moreover, we find that the average (across the latent space) conditioning of the generator is highly predictive of two other ad-hoc metrics for measuring the “quality” of trained GANs: the Inception Score and the Frechet Inception Distance. We then test the hypothesis that this relationship is causal by proposing a “regularization” technique (called Jacobian Clamping) that softly penalizes the condition number of the generator Jacobian. Jacobian Clamping improves the mean score for nearly all datasets on which we tested it. It also greatly reduces inter-run variance of the aforementioned scores, addressing (at least partially) one of the main criticisms of GANs.

[ PDF] 

Author Information

Augustus Odena (Google Brain)
Jacob Buckman (Google)
Catherine Olsson (Google Brain)
Tom B Brown (Google Brain)
Christopher Olah (Google Brain)

I want to understand things clearly and explain them well. Research scientist on [Google Brain](http://g.co/brain), co-editor of [Distill](http://distill.pub).

Colin Raffel (Google)
Ian Goodfellow (Google Brain)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors