Learning Anisotropic Value Geometry with Finsler Reinforcement Learning

We introduce **Finslerian Reinforcement Learning (FiRL)**, an RL framework that makes directional costs explicit and improves robustness to tail risk. FiRL incorporates a *Finsler metric* into the locomotion cost, expressing effort as $F(x,v)$ that depends on the state $x$ and motion $v$, so it can capture uphill versus downhill asymmetry, lateral slip, and other direction-dependent effects. To handle rare but catastrophic outcomes, FiRL optimizes a Conditional Value-at-Risk ($CVaR_\alpha$) objective. We derive the corresponding risk-sensitive Bellman equation and show that the resulting CVaR--Finsler Bellman operator is a $\gamma$-contraction. This guarantees a unique fixed-point value function, while the underlying Finsler cost induces an asymmetric path cost $d_F$ that satisfies a triangle inequality despite directional asymmetry. We then develop a FiRL actor--critic algorithm to learn policies under this anisotropic, risk-averse objective. Across simulation benchmarks and real-world robot trials, FiRL demonstrates safer and more energy-efficient locomotion behavior than strong baselines such as risk-neutral PPO. For instance, on a $12^\circ$ sloped Hopper task, FiRL reduces worst-case ($CVaR_{0.1}$) impact forces by over 35% and total energy cost by 15%, while also improving success rate.

Robots that walk over rough ground need to understand that not every movement has the same cost. Walking uphill is harder than walking downhill, and moving sideways on slippery ground can be risky. Many learning methods do not fully account for these differences, so a robot may look good on average but still fail in difficult or rare situations. This paper introduces FiRL, a method that helps robots choose safer and more efficient movements. FiRL teaches the robot to pay more attention to difficult directions, such as uphill or sideways motion, and to avoid rare failures like slipping, falling, or using too much energy. In simulation and real-robot trials, FiRL helped robots move more safely and efficiently than standard learning methods. This work is a step toward building robots that can handle uneven and uncertain terrain more reliably.