In this talk, we will introduce a continuous-time reinforcement learning (CTRL) framework. Our talk starts with a categorization of RL problems and naturally motivates a continuous-time perspective to RL. We then introduce a model-based CTRL approach, which solves physical control tasks using neural ordinary differential equations as a sub-routine. We conclude by briefly introducing recent approaches to CTRL.