We propose Flow-Anchored Noise-conditioned Q-Learning (FAN), a highly efficient and high-performing offline reinforcement learning (RL) algorithm. Recent work has shown that expressive flow policies and distributional critics improve offline RL performance, but at a high computational cost. Specifically, flow policies require iterative sampling to produce a single action, and distributional critics require computation over multiple samples (e.g., quantiles) to estimate value. To address these inefficiencies while maintaining high performance, we introduce FAN. Our method employs a behavior regularization technique that uses a single flow policy iteration and requires a single Gaussian noise sample for distributional critics. Our theoretical analysis of convergence and performance bounds demonstrates that these simplifications not only improve efficiency but also lead to superior task performance. Experiments on robotic manipulation and locomotion tasks demonstrate that FAN achieves state-of-the-art performance while significantly reducing both training and inference runtimes. We release our code at https://github.com/brianlsy98/FAN.

Problem Training artificial intelligence (AI) to control complex robots is incredibly challenging. To make learning safe, scientists use a method that forces the AI to learn entirely from pre-recorded historical data rather than through risky trial-and-error in the real world. To prevent the AI from making dangerous mistakes, current systems use strict statistical guardrails to keep its decisions mirroring safe, recorded behaviors. However, these guardrails require the AI to run dozens of internal math simulations before making a single choice, making it far too slow and heavy to use in fast, real-world machines. # Approach To break this computational deadlock, we developed a new algorithm called Flow-Anchored Noise-conditioned Q-Learning (FAN). FAN strips away this massive computational slowdown by introducing elegant shortcuts to the AI's decision-making process. Instead of forcing the system to double-check its choices through repetitive mathematical loops, FAN uses a single-step lookahead to maintain safety, combined with a single randomized guess to evaluate future risks. # Impact When tested on complex robotic tasks, FAN achieved a major breakthrough for practical AI deployment. It speeds up the AI’s training time by at least 5 times and drastically accelerates how fast it can react in real-time, all while matching or beating the success rates of much heavier systems. Ultimately, FAN proves that AI does not need to be slow to be smart and safe, making it much easier to deploy advanced, highly responsive AI into everyday robotics.