Timezone: »

 
Poster
InfoDiffusion: Representation Learning Using Information Maximizing Diffusion Models
Yingheng Wang · Yair Schiff · Aaron Gokaslan · Weishen Pan · Fei Wang · Chris De Sa · Volodymyr Kuleshov

Wed Jul 26 05:00 PM -- 06:30 PM (PDT) @ Exhibit Hall 1 #110
in Poster Session 4 »

While diffusion models excel at generating high-quality samples, their latent variables typically lack semantic meaning and are not suitable for representation learning. Here, we propose InfoDiffusion, an algorithm that augments diffusion models with low-dimensional latent variables that capture high-level factors of variation in the data. InfoDiffusion relies on a learning objective regularized with the mutual information between observed and hidden variables, which improves latent space quality and prevents the latents from being ignored by expressive diffusion-based decoders. Empirically, we find that InfoDiffusion learns disentangled and human-interpretable latent representations that are competitive with state-of-the-art generative and contrastive methods, while retaining the high sample quality of diffusion models. Our method enables manipulating the attributes of generated images and has the potential to assist tasks that require exploring a learned latent space to generate quality samples, e.g., generative design.

[ PDF]  [ Poster] 

Author Information

Yingheng Wang (Cornell University)
Yair Schiff (Department of Computer Science, Cornell University)
Yair Schiff

I'm a second year PhD student in the Computer Science department at Cornell University. I have also worked as a software engineer at IBM and collaborated with the Trusted AI department in IBM Research. Prior to joining IBM, I completed a MS in Computer Science at Courant Institute at NYU and a BA in Economics at the University of Pennsylvania.

Aaron Gokaslan (Cornell University)
Weishen Pan (Weill Cornell Medicine, Cornell University)
Fei Wang (Cornell University)
Chris De Sa (Cornell)
Volodymyr Kuleshov (Cornell Tech)

More from the Same Authors