Timezone: »

 
Spotlight
Demystifying Inductive Biases for (Beta-)VAE Based Architectures
Dominik Zietlow · Michal Rolinek · Georg Martius

Thu Jul 22 06:20 PM -- 06:25 PM (PDT) @ None
in Representation Learning 5 »

The performance of Beta-Variational-Autoencoders and their variants on learning semantically meaningful, disentangled representations is unparalleled. On the other hand, there are theoretical arguments suggesting the impossibility of unsupervised disentanglement. In this work, we shed light on the inductive bias responsible for the success of VAE-based architectures. We show that in classical datasets the structure of variance, induced by the generating factors, is conveniently aligned with the latent directions fostered by the VAE objective. This builds the pivotal bias on which the disentangling abilities of VAEs rely. By small, elaborate perturbations of existing datasets, we hide the convenient correlation structure that is easily exploited by a variety of architectures. To demonstrate this, we construct modified versions of standard datasets in which (i) the generative factors are perfectly preserved; (ii) each image undergoes a mild transformation causing a small change of variance; (iii) the leading VAE-based disentanglement architectures fail to produce disentangled representations whilst the performance of a non-variational method remains unchanged.

Keywords: Generative Models · Deep Learning · Adversarial Networks · Applications -> Body Pose, Face, and Gesture Analysis; Applications -> Computer Vision; Deep Learning · Embedding and Representation learning
[Paper]

Author Information

Dominik Zietlow (Max Planck Institute for Intelligent Systems)
Michal Rolinek (Max Planck Institute for Intelligent Systems)
Georg Martius (Max Planck Institute for Intelligent Systems)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors