Timezone: »
We present PredRNN++, a recurrent network for spatiotemporal predictive learning. In pursuit of a great modeling capability for short-term video dynamics, we make our network deeper in time by leveraging a new recurrent structure named Causal LSTM with cascaded dual memories. To alleviate the gradient propagation difficulties in deep predictive models, we propose a Gradient Highway Unit, which provides alternative quick routes for the gradient flows from outputs back to long-range previous inputs. The gradient highway units work seamlessly with the causal LSTMs, enabling our model to capture the short-term and the long-term video dependencies adaptively. Our model achieves state-of-the-art prediction results on both synthetic and real video datasets, showing its power in modeling entangled motions.
Author Information
Yunbo Wang (Tsinghua University)
Zhifeng Gao (Tsinghua University)
Mingsheng Long (Tsinghua University)
Jianmin Wang (Tsinghua University)
Philip Yu (UIC)
Related Events (a corresponding poster, oral, or spotlight)
-
2018 Poster: PredRNN++: Towards A Resolution of the Deep-in-Time Dilemma in Spatiotemporal Predictive Learning »
Wed. Jul 11th 04:15 -- 07:00 PM Room Hall B #194
More from the Same Authors
-
2023 Poster: Estimating Heterogeneous Treatment Effects: Mutual Information Bounds and Learning Algorithms »
Xingzhuo Guo · Yuchen Zhang · Jianmin Wang · Mingsheng Long -
2023 Poster: CLIPood: Generalizing CLIP to Out-of-Distributions »
Yang Shu · Xingzhuo Guo · Jialong Wu · Ximei Wang · Jianmin Wang · Mingsheng Long -
2023 Poster: Solving High-Dimensional PDEs with Latent Spectral Models »
Haixu Wu · Tengge Hu · huakun luo · Jianmin Wang · Mingsheng Long -
2022 : Deoscillated Adaptive Graph Collaborative Filtering »
Zhiwei Liu · Lin Meng · Fei Jiang · Jiawei Zhang · Philip Yu -
2022 Poster: Flowformer: Linearizing Transformers with Conservation Flows »
Haixu Wu · Jialong Wu · Jiehui Xu · Jianmin Wang · Mingsheng Long -
2022 Spotlight: Flowformer: Linearizing Transformers with Conservation Flows »
Haixu Wu · Jialong Wu · Jiehui Xu · Jianmin Wang · Mingsheng Long -
2021 Poster: LogME: Practical Assessment of Pre-trained Models for Transfer Learning »
Kaichao You · Yong Liu · Jianmin Wang · Mingsheng Long -
2021 Spotlight: LogME: Practical Assessment of Pre-trained Models for Transfer Learning »
Kaichao You · Yong Liu · Jianmin Wang · Mingsheng Long -
2021 Poster: Representation Subspace Distance for Domain Adaptation Regression »
Xinyang Chen · Sinan Wang · Jianmin Wang · Mingsheng Long -
2021 Spotlight: Representation Subspace Distance for Domain Adaptation Regression »
Xinyang Chen · Sinan Wang · Jianmin Wang · Mingsheng Long -
2021 Poster: Self-Tuning for Data-Efficient Deep Learning »
Ximei Wang · Jinghan Gao · Mingsheng Long · Jianmin Wang -
2021 Poster: Zoo-Tuning: Adaptive Transfer from A Zoo of Models »
Yang Shu · Zhi Kou · Zhangjie Cao · Jianmin Wang · Mingsheng Long -
2021 Spotlight: Self-Tuning for Data-Efficient Deep Learning »
Ximei Wang · Jinghan Gao · Mingsheng Long · Jianmin Wang -
2021 Spotlight: Zoo-Tuning: Adaptive Transfer from A Zoo of Models »
Yang Shu · Zhi Kou · Zhangjie Cao · Jianmin Wang · Mingsheng Long -
2020 Poster: Unsupervised Transfer Learning for Spatiotemporal Predictive Networks »
Zhiyu Yao · Yunbo Wang · Mingsheng Long · Jianmin Wang -
2019 Poster: Transferable Adversarial Training: A General Approach to Adapting Deep Classifiers »
Hong Liu · Mingsheng Long · Jianmin Wang · Michael Jordan -
2019 Poster: Transferability vs. Discriminability: Batch Spectral Penalization for Adversarial Domain Adaptation »
Xinyang Chen · Sinan Wang · Mingsheng Long · Jianmin Wang -
2019 Oral: Transferability vs. Discriminability: Batch Spectral Penalization for Adversarial Domain Adaptation »
Xinyang Chen · Sinan Wang · Mingsheng Long · Jianmin Wang -
2019 Oral: Transferable Adversarial Training: A General Approach to Adapting Deep Classifiers »
Hong Liu · Mingsheng Long · Jianmin Wang · Michael Jordan -
2017 Poster: Deep Transfer Learning with Joint Adaptation Networks »
Mingsheng Long · Han Zhu · Jianmin Wang · Michael Jordan -
2017 Poster: Kernelized Support Tensor Machines »
Lifang He · Chun-Ta Lu · Guixiang Ma · Shen Wang · Linlin Shen · Philip Yu · Ann Ragin -
2017 Talk: Deep Transfer Learning with Joint Adaptation Networks »
Mingsheng Long · Han Zhu · Jianmin Wang · Michael Jordan -
2017 Talk: Kernelized Support Tensor Machines »
Lifang He · Chun-Ta Lu · Guixiang Ma · Shen Wang · Linlin Shen · Philip Yu · Ann Ragin