Combining parametric and nonparametric models for off-policy evaluation
Omer Gottesman · Yao Liu · Scott Sussex · Emma Brunskill · Finale Doshi-Velez

Wed Jun 12th 04:40 -- 05:00 PM @ Room 104

We consider a model-based approach to perform batch off-policy evaluation in reinforcement learning. Our method takes a mixture-of-experts approach to combine parametric and non-parametric models of the environment such that the final value estimate has the least expected error. We do so by first estimating the local accuracy of each model and then using a planner to select which model to use at every time step as to minimize the return error estimate along entire trajectories. Across a variety of domains, our mixture-based approach outperforms the individual models alone as well as state-of-the-art importance sampling-based estimators.

Author Information

Omer Gottesman (Harvard University)
Yao Liu (Stanford University)
Scott Sussex (Harvard University)
Emma Brunskill (Stanford University)
Finale Doshi-Velez (Harvard University)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors