RiskZero: Plan More to Risk Less with a Learned Model

AlphaZero and MuZero have demonstrated superhuman performance across a range of strategic tasks. Yet their reliance on maximizing expected returns limits their use in real-world settings, where even high-return policies may incur rare but catastrophic failures. We introduce RiskZero to address this limitation; the first MuZero-family method for risk-sensitive decision-making, and planning with zero prior knowledge of environment dynamics. RiskZero learns distributional quantities to estimate trajectory-level risk, guiding search toward policies that explicitly avoid rare but severe outcomes. We establish theoretical convergence to optimal, stationary risk-sensitive policies and validate our approach on environments designed to test risk-sensitive learning from pixels, as well as on larger-scale combinatorial tasks. Across all settings, RiskZero consistently outperforms state-of-the-art risk-sensitive baselines, and improves sample efficiency, providing a general framework for safer and reliable model-based reinforcement learning under uncertainty.

We make decisions in an uncertain world, where one bad outcome can be the end; a single crash, a single failed treatment, or a single ruinous trade in the market. Powerful AI planners like AlphaZero and MuZero have mastered games like Chess and Atari by chasing the highest average score. But averages hide danger; a strategy that usually wins big can still fail rarely and disastrously. In the real world, there is often no second chance; the average is often the wrong objective. We built RiskZero to address, an AI agent that plans not just to win, but to avoid the worst. Like a careful person weighing their options before acting, RiskZero imagines many possible futures and learns to picture the full range of outcomes each choice may lead to - including the rare, and severe ones. Crucially, it judges risk over its entire journey - past and future - which we show is essential for making genuinely safe decisions. By folding this foresight into its search, RiskZero learns to steer clear of catastrophe while still performing well. We prove mathematically that this approach reliably finds safe strategies, and across video-game-like and large planning problems it consistently beats existing methods for risk-sensitive decision making while efficiently learning from far less experience. RiskZero is a step toward AI agents we can trust to make safer plans and act cautiously when the stakes are high.