Timezone: »

 
Poster
Thompson Sampling with Diffusion Generative Prior
Yu-Guan Hsieh · Shiva Kasiviswanathan · Branislav Kveton · Patrick Bloebaum

Thu Jul 27 01:30 PM -- 03:00 PM (PDT) @ Exhibit Hall 1 #403
in Poster Session 5 »

In this work, we initiate the idea of using denoising diffusion models to learn priors for online decision making problems. We specifically focus on bandit meta-learning, aiming to learn a policy that performs well across bandit tasks of a same class. To this end, we train a diffusion model that learns the underlying task distribution and combine Thompson sampling with the learned prior to deal with new tasks at test time. Our posterior sampling algorithm carefully balances between the learned prior and the noisy observations that come from the learner's interaction with the environment. To capture realistic bandit scenarios, we propose a novel diffusion model training procedure that trains from incomplete and noisy data, which could be of independent interest. Finally, our extensive experiments clearly demonstrate the potential of the proposed approach.

[ PDF] 

Author Information

Yu-Guan Hsieh (University of Grenoble-Alpes)
Shiva Kasiviswanathan (Amazon)
Branislav Kveton (AWS AI Labs)
Patrick Bloebaum (Amazon Web Services)

More from the Same Authors