Timezone: »

 
Poster
Video Prediction via Example Guidance
Jingwei Xu · Harry (Huazhe) Xu · Bingbing Ni · Xiaokang Yang · Trevor Darrell

Thu Jul 16 09:00 AM -- 09:45 AM & Thu Jul 16 08:00 PM -- 08:45 PM (PDT) @
in Poster Session 44 »

In video prediction tasks, one major challenge is to capture the multi-modal nature of future contents and dynamics. In this work, we propose a simple yet effective framework that can efficiently predict plausible future states. The key insight is that the potential distribution of a sequence could be approximated with analogous ones in a repertoire of training pool, namely, expert examples. By further incorporating a novel optimization scheme into the training procedure, plausible predictions can be sampled efficiently from distribution constructed from the retrieved examples. Meanwhile, our method could be seamlessly integrated with existing stochastic predictive models; significant enhancement is observed with comprehensive experiments in both quantitative and qualitative aspects. We also demonstrate the generalization ability to predict the motion of unseen class, i.e., without access to corresponding data during training phase.

Keywords: Computer Vision · Time Series and Sequence Models · Applications - Computer Vision

Author Information

Jingwei Xu (Shanghai Jiao Tong University)
Harry (Huazhe) Xu (UC Berkeley)

I am a 2nd year phd at UC Berkeley doing RL and vision under Prof. Trevor Darrell. I also actively collaborate with Prof. Sergey Levine and Prof. Tengyu Ma

Bingbing Ni (Shanghai Jiao Tong University)
Xiaokang Yang (Shanghai Jiao Tong University of China)
Trevor Darrell (University of California at Berkeley)

More from the Same Authors