Timezone: »

Deep Latent State Space Models for Time-Series Generation
Linqi Zhou · Michael Poli · Winnie Xu · Stefano Massaroli · Stefano Ermon

Wed Jul 26 02:00 PM -- 03:30 PM (PDT) @ Exhibit Hall 1 #113

Methods based on ordinary differential equations (ODEs) are widely used to build generative models of time-series. In addition to high computational overhead due to explicitly computing hidden states recurrence, existing ODE-based models fall short in learning sequence data with sharp transitions - common in many real-world systems - due to numerical challenges during optimization. In this work, we propose LS4, a generative model for sequences with latent variables evolving according to a state space ODE to increase modeling capacity. Inspired by recent deep state space models (S4), we achieve speedups by leveraging a convolutional representation of LS4 which bypasses the explicit evaluation of hidden states. We show that LS4 significantly outperforms previous continuous-time generative models in terms of marginal distribution, classification, and prediction scores on real-world datasets in the Monash Forecasting Repository, and is capable of modeling highly stochastic data with sharp temporal transitions. LS4 sets state-of-the-art for continuous-time latent generative models, with significant improvement of mean squared error and tighter variational lower bounds on irregularly-sampled datasets, while also being x100 faster than other baselines on long sequences.

Author Information

Linqi Zhou (Stanford University)
Michael Poli (Stanford University)
Winnie Xu (University of Toronto)
Stefano Massaroli (Mila)
Stefano Ermon (Stanford University)

More from the Same Authors