Timezone: »

 
Poster
Thompson Sampling via Local Uncertainty
Zhendong Wang · Mingyuan Zhou

Tue Jul 14 02:00 PM -- 02:45 PM & Wed Jul 15 02:00 AM -- 02:45 AM (PDT) @
in Poster Session 8 »

Thompson sampling is an efficient algorithm for sequential decision making, which exploits the posterior uncertainty to address the exploration-exploitation dilemma. There has been significant recent interest in integrating Bayesian neural networks into Thompson sampling. Most of these methods rely on global variable uncertainty for exploration. In this paper, we propose a new probabilistic modeling framework for Thompson sampling, where local latent variable uncertainty is used to sample the mean reward. Variational inference is used to approximate the posterior of the local variable, and semi-implicit structure is further introduced to enhance its expressiveness. Our experimental results on eight contextual bandit benchmark datasets show that Thompson sampling guided by local uncertainty achieves state-of-the-art performance while having low computational complexity.

Keywords: Approximate Inference · Bayesian Deep Learning · Bayesian Methods · Probabilistic Inference - Models and Probabilistic Programming

Author Information

Zhendong Wang (University of Texas, Austin)
Mingyuan Zhou (University of Texas at Austin)

More from the Same Authors