Timezone: »

 
Poster
Small-GAN: Speeding up GAN Training using Core-Sets
Samrath Sinha · Han Zhang · Anirudh Goyal · Yoshua Bengio · Hugo Larochelle · Augustus Odena

Wed Jul 15 10:00 AM -- 10:45 AM & Wed Jul 15 09:00 PM -- 09:45 PM (PDT) @
in Poster Session 24 »

Recent work suggests that Generative Adversarial Networks (GANs) benefit disproportionately from large mini-batch sizes. This finding is interesting but also discouraging -- large batch sizes are slow and expensive to emulate on conventional hardware. Thus, it would be nice if there were some trick by which we could generate batches that were effectively big though small in practice. In this work, we propose such a trick, inspired by the use of Coreset-selection in active learning. When training a GAN, we draw a large batch of samples from the prior and then compress that batch using Coreset-selection. To create effectively large batches of real images, we create a cached dataset of Inception activations of each training image, randomly project them down to a smaller dimension, and then use Coreset-selection on those projected embeddings at training time. We conduct experiments showing that this technique substantially reduces training time and memory usage for modern GAN variants, that it reduces the fraction of dropped modes in a synthetic dataset, and that it helps us use GANs to reach a new state of the art in anomaly detection.

Keywords: Deep Generative Models · Generative Adversarial Networks · Unsupervised Learning · Deep Learning - General

Author Information

Samrath Sinha (University of Toronto)
Han Zhang (Google)
Anirudh Goyal (Université de Montréal)
Yoshua Bengio (Montreal Institute for Learning Algorithms)
Hugo Larochelle (Google Brain)
Augustus Odena (Google Brain)

More from the Same Authors