Timezone: »

 
Poster
Thompson Sampling Algorithms for Mean-Variance Bandits
Qiuyu Zhu · Vincent Tan

Tue Jul 14 03:00 PM -- 03:45 PM & Wed Jul 15 04:00 AM -- 04:45 AM (PDT) @ Virtual
in Poster Session 9 »

The multi-armed bandit (MAB) problem is a classical learning task that exemplifies the exploration-exploitation tradeoff. However, standard formulations do not take into account risk. In online decision making systems, risk is a primary concern. In this regard, the mean-variance risk measure is one of the most common objective functions. Existing algorithms for mean-variance optimization in the context of MAB problems have unrealistic assumptions on the reward distributions. We develop Thompson Sampling-style algorithms for mean-variance MAB and provide comprehensive regret analyses for Gaussian and Bernoulli bandits with fewer assumptions. Our algorithms achieve the best known regret bounds for mean-variance MABs and also attain the information-theoretic bounds in some parameter regimes. Empirical simulations show that our algorithms significantly outperform existing LCB-based algorithms for all risk tolerances.

Keywords: Online Learning / Bandits · Reinforcement Learning Theory · Online Learning, Active Learning, and Bandits
[ Slides] 

Author Information

Qiuyu Zhu (National University of Singapore)
Vincent Tan (National University of Singapore)

More from the Same Authors