Timezone: »

 
Spotlight
A Gradient Based Strategy for Hamiltonian Monte Carlo Hyperparameter Optimization
Andrew Campbell · Wenlong Chen · Vincent Stimper · Jose Miguel Hernandez-Lobato · Yichuan Zhang

Thu Jul 22 07:45 AM -- 07:50 AM (PDT) @
in Bayesian Methods »

Hamiltonian Monte Carlo (HMC) is one of the most successful sampling methods in machine learning. However, its performance is significantly affected by the choice of hyperparameter values. Existing approaches for optimizing the HMC hyperparameters either optimize a proxy for mixing speed or consider the HMC chain as an implicit variational distribution and optimize a tractable lower bound that can be very loose in practice. Instead, we propose to optimize an objective that quantifies directly the speed of convergence to the target distribution. Our objective can be easily optimized using stochastic gradient descent. We evaluate our proposed method and compare to baselines on a variety of problems including sampling from synthetic 2D distributions, reconstructing sparse signals, learning deep latent variable models and sampling molecular configurations from the Boltzmann distribution of a 22 atom molecule. We find that our method is competitive with or improves upon alternative baselines in all these experiments.

Keywords: Monte Carlo Methods

Author Information

Andrew Campbell (University of Oxford)
Wenlong Chen (Imperial College London)
Vincent Stimper (University of Cambridge)
Jose Miguel Hernandez-Lobato (University of Cambridge)
Yichuan Zhang (Boltzbit Limited)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors