Stochastic Minimum-Cost Reach-Avoid Reinforcement Learning

We study stochastic minimum-cost reach-avoid reinforcement learning, where an agent must satisfy a reach-avoid specification with probability at least $p$ while minimizing expected cumulative costs in stochastic environments. Existing safe and constrained reinforcement learning methods typically fail to jointly enforce probabilistic reach-avoid constraints and optimize cost in the learning setting in stochastic environments. To address this challenge, we introduce reach-avoid probability certificates (RAPCs), which identify states from which stochastic reach-avoid constraints are satisfiable. Building on RAPCs, we develop a contraction-based Bellman formulation that serves as a principled surrogate for integrating reach-avoid considerations into reinforcement learning, enabling cost optimization under probabilistic constraints. We establish almost sure convergence of the proposed algorithms to locally optimal policies with respect to the resulting objective. Experiments in the MuJoCo simulator demonstrate improved cost performance and consistently higher reach-avoid satisfaction rates.

This paper studies how to train decision-making systems that must reach a desired goal while avoiding unsafe states in uncertain stochastic environments. Such problems appear in robotics and autonomous driving, where a system should complete its task safely and reliably while keeping its actions efficient. We propose a new reinforcement learning method that helps the system understand when the required safety-and-success condition can be satisfied. Based on this, the system can learn policies that lower the total cost while still meeting the required reach-and-avoid guarantee. Experiments in simulated robotic control tasks show that our method performs better than existing approaches in both cost and task satisfaction.