Timezone: »

 
Poster
Optimistic Policy Optimization via Multiple Importance Sampling
Matteo Papini · Alberto Maria Metelli · Lorenzo Lupo · Marcello Restelli

Tue Jun 11 06:30 PM -- 09:00 PM (PDT) @ Pacific Ballroom #103
in Posters Tue »
Policy Search (PS) is an effective approach to Reinforcement Learning (RL) for solving control tasks with continuous state-action spaces. In this paper, we address the exploration-exploitation trade-off in PS by proposing an approach based on Optimism in the Face of Uncertainty. We cast the PS problem as a suitable Multi Armed Bandit (MAB) problem, defined over the policy parameter space, and we propose a class of algorithms that effectively exploit the problem structure, by leveraging Multiple Importance Sampling to perform an off-policy estimation of the expected return. We show that the regret of the proposed approach is bounded by $\widetilde{\mathcal{O}}(\sqrt{T})$ for both discrete and continuous parameter spaces. Finally, we evaluate our algorithms on tasks of varying difficulty, comparing them with existing MAB and RL algorithms.
Keywords: Theory and Algorithms

Author Information

Matteo Papini (Politecnico di Milano)
Alberto Maria Metelli (Politecnico di Milano)
Lorenzo Lupo (Politecnico di Milano)
Marcello Restelli (Politecnico di Milano)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors