Future predictions on sequence data (e.g., videos or audios) require the algorithms to capture non-Markovian and compositional properties of high-level semantics. Context-free grammars are natural choices to capture such properties, but traditional grammar parsers (e.g., Earley parser) only take symbolic sentences as inputs. In this paper, we generalize the Earley parser to parse sequence data which is neither segmented nor labeled. This generalized Earley parser integrates a grammar parser with a classifier to find the optimal segmentation and labels, and makes top-down future predictions. Experiments show that our method significantly outperforms other approaches for future human activity prediction.
Siyuan Qi (UCLA)
I am a third year Ph.D. Candidate in the Computer Science Department at the University of California, Los Angeles advised by Professor Song-Chun Zhu.
Baoxiong Jia (Peking University)
Song-Chun Zhu (UCLA)
Related Events (a corresponding poster, oral, or spotlight)
2018 Poster: Generalized Earley Parser: Bridging Symbolic Grammars and Sequence Data for Future Prediction »
Fri Jul 13th 04:15 -- 07:00 PM Room Hall B
More from the Same Authors
2021 : [12:02 - 12:47 PM UTC] Invited Talk 1: Explainable AI: How Machines Gain Justified Trust from Humans »
2020 Poster: Closed Loop Neural-Symbolic Learning via Integrating Neural Perception, Grammar Parsing, and Symbolic Reasoning »
Qing Li · Siyuan Huang · Yining Hong · Yixin Chen · Ying Nian Wu · Song-Chun Zhu