Skip to yearly menu bar Skip to main content


Poster

Meta-Learning Hypothesis Spaces for Sequential Decision-making

Parnian Kassraie · Jonas Rothfuss · Andreas Krause

Hall E #529

Keywords: [ MISC: Transfer, Multitask and Meta-learning ] [ PM: Bayesian Models and Methods ] [ T: Probabilistic Methods ] [ T: Online Learning and Bandits ] [ T: Learning Theory ] [ OPT: Optimization and Learning under Uncertainty ] [ MISC: Online Learning, Active Learning and Bandits ]


Abstract:

Obtaining reliable, adaptive confidence sets for prediction functions (hypotheses) is a central challenge in sequential decision-making tasks, such as bandits and model-based reinforcement learning. These confidence sets typically rely on prior assumptions on the hypothesis space, e.g., the known kernel of a Reproducing Kernel Hilbert Space (RKHS). Hand-designing such kernels is error prone, and misspecification may lead to poor or unsafe performance. In this work, we propose to meta-learn a kernel from offline data (Meta-KeL). For the case where the unknown kernel is a combination of known base kernels, we develop an estimator based on structured sparsity. Under mild conditions, we guarantee that our estimated RKHS yields valid confidence sets that, with increasing amounts of offline data, become as tight as those given the true unknown kernel. We demonstrate our approach on the kernelized bandits problem (a.k.a. Bayesian optimization), where we establish regret bounds competitive with those given the true kernel. We also empirically evaluate the effectiveness of our approach on a Bayesian optimization task.

Chat is not available.