Timezone: »
Thompson sampling is an efficient algorithm for sequential decision making, which exploits the posterior uncertainty to address the exploration-exploitation dilemma. There has been significant recent interest in integrating Bayesian neural networks into Thompson sampling. Most of these methods rely on global variable uncertainty for exploration. In this paper, we propose a new probabilistic modeling framework for Thompson sampling, where local latent variable uncertainty is used to sample the mean reward. Variational inference is used to approximate the posterior of the local variable, and semi-implicit structure is further introduced to enhance its expressiveness. Our experimental results on eight contextual bandit benchmark datasets show that Thompson sampling guided by local uncertainty achieves state-of-the-art performance while having low computational complexity.
Author Information
Zhendong Wang (University of Texas, Austin)
Mingyuan Zhou (University of Texas at Austin)
More from the Same Authors
-
2022 Poster: Deep Variational Graph Convolutional Recurrent Network for Multivariate Time Series Anomaly Detection »
Wenchao Chen · Long Tian · Bo Chen · Liang Dai · Zhibin Duan · Mingyuan Zhou -
2022 Spotlight: Deep Variational Graph Convolutional Recurrent Network for Multivariate Time Series Anomaly Detection »
Wenchao Chen · Long Tian · Bo Chen · Liang Dai · Zhibin Duan · Mingyuan Zhou -
2022 Poster: Bayesian Deep Embedding Topic Meta-Learner »
Zhibin Duan · Yishi Xu · Jianqiao Sun · Bo Chen · Wenchao Chen · CHAOJIE WANG · Mingyuan Zhou -
2022 Spotlight: Bayesian Deep Embedding Topic Meta-Learner »
Zhibin Duan · Yishi Xu · Jianqiao Sun · Bo Chen · Wenchao Chen · CHAOJIE WANG · Mingyuan Zhou -
2022 Poster: Regularizing a Model-based Policy Stationary Distribution to Stabilize Offline Reinforcement Learning »
Shentao Yang · Yihao Feng · Shujian Zhang · Mingyuan Zhou -
2022 Spotlight: Regularizing a Model-based Policy Stationary Distribution to Stabilize Offline Reinforcement Learning »
Shentao Yang · Yihao Feng · Shujian Zhang · Mingyuan Zhou -
2021 Poster: Bayesian Attention Belief Networks »
Shujian Zhang · Xinjie Fan · Bo Chen · Mingyuan Zhou -
2021 Spotlight: Bayesian Attention Belief Networks »
Shujian Zhang · Xinjie Fan · Bo Chen · Mingyuan Zhou -
2021 Poster: Sawtooth Factorial Topic Embeddings Guided Gamma Belief Network »
Zhibin Duan · Dongsheng Wang · Bo Chen · CHAOJIE WANG · Wenchao Chen · yewen li · Jie Ren · Mingyuan Zhou -
2021 Poster: ARMS: Antithetic-REINFORCE-Multi-Sample Gradient for Binary Variables »
Alek Dimitriev · Mingyuan Zhou -
2021 Spotlight: ARMS: Antithetic-REINFORCE-Multi-Sample Gradient for Binary Variables »
Alek Dimitriev · Mingyuan Zhou -
2021 Spotlight: Sawtooth Factorial Topic Embeddings Guided Gamma Belief Network »
Zhibin Duan · Dongsheng Wang · Bo Chen · CHAOJIE WANG · Wenchao Chen · yewen li · Jie Ren · Mingyuan Zhou -
2020 Poster: Bayesian Graph Neural Networks with Adaptive Connection Sampling »
Arman Hasanzadeh · Ehsan Hajiramezanali · Shahin Boluki · Mingyuan Zhou · Nick Duffield · Krishna Narayanan · Xiaoning Qian -
2020 Poster: Recurrent Hierarchical Topic-Guided RNN for Language Generation »
Dandan Guo · Bo Chen · Ruiying Lu · Mingyuan Zhou -
2019 Poster: ARSM: Augment-REINFORCE-Swap-Merge Estimator for Gradient Backpropagation Through Categorical Variables »
Mingzhang Yin · Yuguang Yue · Mingyuan Zhou -
2019 Oral: ARSM: Augment-REINFORCE-Swap-Merge Estimator for Gradient Backpropagation Through Categorical Variables »
Mingzhang Yin · Yuguang Yue · Mingyuan Zhou -
2019 Poster: Convolutional Poisson Gamma Belief Network »
CHAOJIE WANG · Bo Chen · SUCHENG XIAO · Mingyuan Zhou -
2019 Poster: Locally Private Bayesian Inference for Count Models »
Aaron Schein · Steven Wu · Alexandra Schofield · Mingyuan Zhou · Hanna Wallach -
2019 Oral: Convolutional Poisson Gamma Belief Network »
CHAOJIE WANG · Bo Chen · SUCHENG XIAO · Mingyuan Zhou -
2019 Oral: Locally Private Bayesian Inference for Count Models »
Aaron Schein · Steven Wu · Alexandra Schofield · Mingyuan Zhou · Hanna Wallach -
2018 Poster: Inter and Intra Topic Structure Learning with Word Embeddings »
He Zhao · Lan Du · Wray Buntine · Mingyuan Zhou -
2018 Oral: Inter and Intra Topic Structure Learning with Word Embeddings »
He Zhao · Lan Du · Wray Buntine · Mingyuan Zhou -
2018 Poster: Semi-Implicit Variational Inference »
Mingzhang Yin · Mingyuan Zhou -
2018 Oral: Semi-Implicit Variational Inference »
Mingzhang Yin · Mingyuan Zhou -
2017 Poster: Deep Latent Dirichlet Allocation with Topic-Layer-Adaptive Stochastic Gradient Riemannian MCMC »
Yulai Cong · Bo Chen · Hongwei Liu · Mingyuan Zhou -
2017 Talk: Deep Latent Dirichlet Allocation with Topic-Layer-Adaptive Stochastic Gradient Riemannian MCMC »
Yulai Cong · Bo Chen · Hongwei Liu · Mingyuan Zhou