Plug-and-Play Benchmarking of Reinforcement Learning Algorithms for Large-Scale Flow Control

Reinforcement learning (RL) has shown promising results in active flow control (AFC), yet progress in the field remains difficult to assess as existing studies rely on heterogeneous observation and actuation schemes, numerical setups, and evaluation protocols. Current AFC benchmarks attempt to address these issues but heavily rely on external computational fluid dynamics (CFD) solvers, are not fully differentiable, and provide limited 3D and multi-agent support. To overcome these limitations, we introduce FluidGym, the first standalone, fully differentiable benchmark suite for RL in AFC. Built entirely in PyTorch on top of the GPU-accelerated PICT solver, FluidGym runs in a single Python stack, requires no external CFD software, and provides standardized evaluation protocols. We present baseline results with PPO, SAC, DPC, and TD-MPC, and release all environments, datasets, and trained models as public resources. FluidGym enables systematic comparison of control methods, establishes a scalable foundation for future research in learning-based flow control, and is available at github.com/safe-autonomous-systems/fluidgym.

If you have ever looked out the window of a commercial airplane during landing, you have likely seen mechanical flaps on the wing adjust to control the airflow and keep the plane stable. Using AI to automate this kind of fluid control could substantially improve fuel efficiency and reduce emissions. However, developing these AI systems is incredibly difficult because researchers lack a common testing ground, relying instead on slow, fragmented physics software that makes it nearly impossible to fairly compare different methods. To solve this, we introduce FluidGym, the first all-in-one testing toolkit designed specifically for training AI in fluid flow control settings. Built entirely on modern AI software (PyTorch) and powered by graphics cards, FluidGym eliminates the need for external engineering programs, runs entirely in a single codebase, and provides standardized rules for evaluation. We also provide initial results using several popular AI algorithms and have made all of our code, data, and models publicly available. FluidGym provides the AI community with a shared, standardized testing ground. This makes it significantly easier for researchers to fairly compare their methods, accelerating the development of smarter, more efficient fluid control systems.