Structured Inference with Large Language Gibbs

We propose Large Language Gibbs, a scheme for structured probabilistic inference that uses conditional distributions of a large language model (LLM) as transition operators. Rather than sampling structured objects through single-pass autoregressive generation, we iteratively resample individual variables conditioned on others using an LLM's next-token conditionals. This approach avoids order-dependent biases and produces a stationary distribution that reflects a compromise between all local conditionals. We apply this approach to sampling from synthetic distributions, consistent reasoning tasks, and Bayesian structure learning. The results suggest that the use of LLM conditionals in MCMC as a practical alternative to one-pass generation for structured probabilistic inference under a world prior accessible through noisy LLM conditionals.