Timezone: »

VFlow: More Expressive Generative Flows with Variational Data Augmentation
Jianfei Chen · Cheng Lu · Biqi Chenli · Jun Zhu · Tian Tian

Wed Jul 15 05:00 AM -- 05:45 AM & Wed Jul 15 07:00 PM -- 07:45 PM (PDT) @ Virtual #None

Generative flows are promising tractable models for density modeling that define probabilistic distributions with invertible transformations. However, tractability imposes architectural constraints on generative flows. In this work, we study a previously overlooked constraint that all the intermediate representations must have the same dimensionality with the data due to invertibility, limiting the width of the network. We propose VFlow to tackle this constraint on dimensionality. VFlow augments the data with extra dimensions and defines a maximum evidence lower bound (ELBO) objective for estimating the distribution of augmented data jointly with the variational data augmentation distribution. Under mild assumptions, we show that the maximum ELBO solution of VFlow is always better than the original maximum likelihood solution. For image density modeling on the CIFAR-10 dataset, VFlow achieves a new state-of-the-art 2.98 bits per dimension.

Author Information

Jianfei Chen (University of California, Berkeley)
Cheng Lu (Tsinghua University)
Biqi Chenli (Tsinghua University)
Jun Zhu (Tsinghua University)
Tian Tian (RealAI)

More from the Same Authors