Modelling Latent Dynamical Systems with Recognition-Parametrised Models

We introduce a new approach to learning latent Markovian dynamical processes underlying observed time series data: the recognition-parametrised latent dynamical system (RP-LDS). The RP-LDS resolves issues in two broad classes of state-of-the-art latent time series models, while maintaining expressivity through a complex neural network-based link between observations and latents. As opposed to generative or auto-encoding approaches, the RP-LDS does not learn an explicit model reconstructing observations from latents, thus allowing it to avoid parameter bias and focus model capacity on recognition. As opposed to contrastive approaches, the RP-LDS utilises efficient message-passing to propagate posterior uncertainty and achieve maximum-likelihood learning. The RP-LDS matches the performance of state-of-the-art methods on both linear and nonlinear toy problems. We apply the RP-LDS to video of a swinging pendulum with background distractors and show that it is able to recover the underlying latent system despite not being in model class.