Timezone: »

 
Poster
Eventual Discounting Temporal Logic Counterfactual Experience Replay
Cameron Voloshin · Abhinav Verma · Yisong Yue

Tue Jul 25 02:00 PM -- 04:30 PM (PDT) @ Exhibit Hall 1 #118
in Poster Session 1 »

Linear temporal logic (LTL) offers a simplified way of specifying tasks for policy optimization that may otherwise be difficult to describe with scalar reward functions. However, the standard RL framework can be too myopic to find maximally LTL satisfying policies. This paper makes two contributions. First, we develop a new value-function based proxy, using a technique we call eventual discounting, under which one can find policies that satisfy the LTL specification with highest achievable probability. Second, we develop a new experience replay method for generating off-policy data from on-policy rollouts via counterfactual reasoning on different ways of satisfying the LTL specification. Our experiments, conducted in both discrete and continuous state-action spaces, confirm the effectiveness of our counterfactual experience replay approach.

[ PDF] 

Author Information

Cameron Voloshin (Caltech)
Abhinav Verma (The Pennsylvania State University)
Yisong Yue (Caltech & Latitude AI)
Yisong Yue

Yisong Yue is a Professor of Computing and Mathematical Sciences at Caltech and (via sabbatical) a Principal Scientist at Latitude AI. His research interests span both fundamental and applied pursuits, from novel learning-theoretic frameworks all the way to deep learning deployed in autonomous driving on public roads. His work has been recognized with multiple paper awards and nominations, including in robotics, computer vision, sports analytics, machine learning for health, and information retrieval. At Latitude AI, he is working on machine learning approaches to motion planning for autonomous driving.

More from the Same Authors