Latent force models offer an interpretable alternative to purely data driven inference in dynamical systems. Uncertainty in the output variables is treated by deriving the kernel function of the low-dimensional latent forces directly from the dynamics. However, exact computation of posterior kernel terms is rarely tractable, requiring approximations for complex scenarios such as nonlinear dynamics. In this paper, we overcome these issues by posing the problem as meta-learning a general class of latent force models. By employing a deep kernel and a sensible embedding, we achieve extrapolation from a synthetic dataset to real experimental datasets. Moreover, our model is the first of its kind to scale up to large datasets.