Timezone: »

 
Oral
PredRNN++: Towards A Resolution of the Deep-in-Time Dilemma in Spatiotemporal Predictive Learning
Yunbo Wang · Zhifeng Gao · Mingsheng Long · Jianmin Wang · Philip Yu

Wed Jul 11 05:50 AM -- 06:00 AM (PDT) @ Victoria
in Deep Learning (Neural Network Architectures) 3 »

We present PredRNN++, a recurrent network for spatiotemporal predictive learning. In pursuit of a great modeling capability for short-term video dynamics, we make our network deeper in time by leveraging a new recurrent structure named Causal LSTM with cascaded dual memories. To alleviate the gradient propagation difficulties in deep predictive models, we propose a Gradient Highway Unit, which provides alternative quick routes for the gradient flows from outputs back to long-range previous inputs. The gradient highway units work seamlessly with the causal LSTMs, enabling our model to capture the short-term and the long-term video dependencies adaptively. Our model achieves state-of-the-art prediction results on both synthetic and real video datasets, showing its power in modeling entangled motions.

[ PDF]  [ Video] 

Author Information

Yunbo Wang (Tsinghua University)
Zhifeng Gao (Tsinghua University)
Mingsheng Long (Tsinghua University)
Jianmin Wang (Tsinghua University)
Philip Yu (UIC)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors