Timezone: »

 
Practical and Asymptotically Exact Conditional Sampling in Diffusion Models
Brian Trippe · Luhuan Wu · Christian Naesseth · David Blei · John Cunningham

@
in Structured Probabilistic Inference and Generative Modeling »
Event URL: https://openreview.net/forum?id=r9s3Gbxz7g »

Diffusion models have been successful on a range of conditional generation tasks including molecular design and text-to-image generation.However, these achievements have primarily depended on expensive, task-specific conditional training or error-prone heuristic approximations to them.Ideally, a conditional generation method should provide exact samples for a broad range of conditional distributions without requiring task-specific training.To this end, we introduce the Twisted Diffusion Sampler, or TDS, a sequential Monte Carlo (SMC) algorithm that targets the conditional distributions of diffusion models. The main idea is to use twisting, an SMC technique the enjoys good computational efficiency, to incorporate heuristic approximations without compromising asymptotic exactness. We study the properties of TDS on MNIST image inpainting and class-conditional generation tasks.TDS extends to Riemannian diffusion models, which are crucial for protein modeling.When applied to the motif-scaffolding problem, a core problem in protein design, TDS enables more flexible conditioning criteria than conditionally trained models, and provides state-of-the-art success rates on 9/12 problems in a benchmark set with scaffolds shorter than 100 residues.

Keywords: Protein Design · Sequential Monte Carlo · Diffusion Model · Theoretical Guarantees

Author Information

Brian Trippe (Columbia University)
Luhuan Wu (Columbia University)
Christian Naesseth (University of Amsterdam)
David Blei (Columbia University)

David Blei is a Professor of Statistics and Computer Science at Columbia University, and a member of the Columbia Data Science Institute. His research is in statistical machine learning, involving probabilistic topic models, Bayesian nonparametric methods, and approximate posterior inference algorithms for massive data. He works on a variety of applications, including text, images, music, social networks, user behavior, and scientific data. David has received several awards for his research, including a Sloan Fellowship (2010), Office of Naval Research Young Investigator Award (2011), Presidential Early Career Award for Scientists and Engineers (2011), Blavatnik Faculty Award (2013), and ACM-Infosys Foundation Award (2013). He is a fellow of the ACM.

John Cunningham (Columbia)

More from the Same Authors