Oral
Optimistic Policy Optimization via Multiple Importance Sampling
Matteo Papini · Alberto Maria Metelli · Lorenzo Lupo · Marcello Restelli

Tue Jun 11th 02:25 -- 02:30 PM @ Room 104
in Reinforcement Learning Theory »

Policy Search (PS) is an effective approach to Reinforcement Learning for solving control tasks with continuous state-action spaces. In this paper, we address the exploration-exploitation trade-off in PS by proposing an approach based on Optimism in Face of Uncertainty. We cast the PS problem as a suitable Multi Armed Bandit problem, defined over the policy parameter space, and we propose a class of algorithms that effectively exploit the problem structure, by leveraging Multiple Importance Sampling to perform an off-policy estimation of expected return. We show that the regret of the proposed approach is bounded by $\widetilde{\mathcal{O}}(\sqrt{T})$ for both discrete and continuous parameter spaces. Finally, we evaluate our algorithms on tasks of varying difficulty, comparing them with existing MAB and RL algorithms.

Author Information

Matteo Papini (Politecnico di Milano)
Alberto Maria Metelli (Politecnico di Milano)
Lorenzo Lupo (Politecnico di Milano)
Marcello Restelli (Politecnico di Milano)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors