Skip to yearly menu bar Skip to main content


Parallel and Flexible Sampling from Autoregressive Models via Langevin Dynamics

Vivek Jayaram · John Thickstun

Keywords: [ Algorithms -> Large Scale Learning; Algorithms -> Regression; Algorithms -> Sparsity and Compressed Sensing; Algorithms ] [ Stru ] [ Generative Models ] [ Convex Optimization ]


This paper introduces an alternative approach to sampling from autoregressive models. Autoregressive models are typically sampled sequentially, according to the transition dynamics defined by the model. Instead, we propose a sampling procedure that initializes a sequence with white noise and follows a Markov chain defined by Langevin dynamics on the global log-likelihood of the sequence. This approach parallelizes the sampling process and generalizes to conditional sampling. Using an autoregressive model as a Bayesian prior, we can steer the output of a generative model using a conditional likelihood or constraints. We apply these techniques to autoregressive models in the visual and audio domains, with competitive results for audio source separation, super-resolution, and inpainting.

Chat is not available.