Skip to yearly menu bar Skip to main content


Poster

Parallel and Flexible Sampling from Autoregressive Models via Langevin Dynamics

Vivek Jayaram · John Thickstun

Keywords: [ Convex Optimization ] [ Generative Models ] [ Stru ] [ Algorithms -> Large Scale Learning; Algorithms -> Regression; Algorithms -> Sparsity and Compressed Sensing; Algorithms ]


Abstract:

This paper introduces an alternative approach to sampling from autoregressive models. Autoregressive models are typically sampled sequentially, according to the transition dynamics defined by the model. Instead, we propose a sampling procedure that initializes a sequence with white noise and follows a Markov chain defined by Langevin dynamics on the global log-likelihood of the sequence. This approach parallelizes the sampling process and generalizes to conditional sampling. Using an autoregressive model as a Bayesian prior, we can steer the output of a generative model using a conditional likelihood or constraints. We apply these techniques to autoregressive models in the visual and audio domains, with competitive results for audio source separation, super-resolution, and inpainting.

Chat is not available.