Model-Based Reinforcement Learning with SINDy

We draw on the latest advancements in the physics community to propose a novelmethod for discovering the governing non-linear dynamics of physical systemsin reinforcement learning (RL). We establish that this method is capable ofdiscovering the underlying dynamics using significantly fewer trajectories (aslittle as one rollout with $\leq 30$ time steps) than state of the art modellearning algorithms. Further, the technique learns a model that is accurateenough to induce near-optimal policies given significantly fewer trajectoriesthan those required by model-free algorithms. It brings the benefits ofmodel-based RL without requiring a model to be developed in advance, forsystems that have physics-based dynamics.To establish the validity and applicability of this algorithm, we conductexperiments on four classic control tasks. We found that an optimal policytrained on the discovered dynamics of the underlying system can generalizewell. Further, the learned policy performs well when deployed on the actualphysical system, thus bridging the model to real system gap. We furthercompare our method to state-of-the-art model-based and model-free approaches,and show that our method requires fewer trajectories sampled on the truephysical system compared other methods. Additionally, we explored approximatedynamics models and found that they also can perform well.