Timezone: »

 
Poster
Recurrent Hierarchical Topic-Guided RNN for Language Generation
Dandan Guo · Bo Chen · Ruiying Lu · Mingyuan Zhou

Tue Jul 14 07:00 AM -- 07:45 AM & Tue Jul 14 08:00 PM -- 08:45 PM (PDT) @
in Poster Session 1 »

To simultaneously capture syntax and global semantics from a text corpus, we propose a new larger-context recurrent neural network (RNN) based language model, which extracts recurrent hierarchical semantic structure via a dynamic deep topic model to guide natural language generation. Moving beyond a conventional RNN-based language model that ignores long-range word dependencies and sentence order, the proposed model captures not only intra-sentence word dependencies, but also temporal transitions between sentences and inter-sentence topic dependencies. For inference, we develop a hybrid of stochastic-gradient Markov chain Monte Carlo and recurrent autoencoding variational Bayes. Experimental results on a variety of real-world text corpora demonstrate that the proposed model not only outperforms larger-context RNN-based language models, but also learns interpretable recurrent multilayer topics and generates diverse sentences and paragraphs that are syntactically correct and semantically coherent.

Keywords: Approximate Inference · Bayesian Methods · Deep Generative Models · Probabilistic Inference - Models and Probabilistic Programming

Author Information

Dandan Guo (National Laboratory of Radar Signal Processing, Xidian University)
Bo Chen (School of Electronic Engineering, Xidian University)

Bo Chen, Ph.D., Professor. Before joining the Department of Electronic Engineering in Xidian University in 2013, I was a post-doc researcher, research scientist and senior research scientist at the Department of Electrical and Computer Engineering in Duke University. In 2013 and 2014, I was elected into the Program for New Century Excellent Talents in University and the Program for Thousand Youth Talents respectively. I am interested in developing statistical machine learning methods for the complex and large-scale data. My current interests are in statistical signal processing, statistical machine learning, deep learning and their applications to radar target detection and recognition.

Ruiying Lu (xidian university)
Mingyuan Zhou (University of Texas at Austin)

More from the Same Authors