Timezone: »

 
Poster
Sawtooth Factorial Topic Embeddings Guided Gamma Belief Network
Zhibin Duan · Dongsheng Wang · Bo Chen · CHAOJIE WANG · Wenchao Chen · yewen li · Jie Ren · Mingyuan Zhou

Wed Jul 21 09:00 AM -- 11:00 AM (PDT) @ Virtual
in Poster Session 3 »

Hierarchical topic models such as the gamma belief network (GBN) have delivered promising results in mining multi-layer document representations and discovering interpretable topic taxonomies. However, they often assume in the prior that the topics at each layer are independently drawn from the Dirichlet distribution, ignoring the dependencies between the topics both at the same layer and across different layers. To relax this assumption, we propose sawtooth factorial topic embedding guided GBN, a deep generative model of documents that captures the dependencies and semantic similarities between the topics in the embedding space. Specifically, both the words and topics are represented as embedding vectors of the same dimension. The topic matrix at a layer is factorized into the product of a factor loading matrix and a topic embedding matrix, the transpose of which is set as the factor loading matrix of the layer above. Repeating this particular type of factorization, which shares components between adjacent layers, leads to a structure referred to as sawtooth factorization. An auto-encoding variational inference network is constructed to optimize the model parameter via stochastic gradient descent. Experiments on big corpora show that our models outperform other neural topic models on extracting deeper interpretable topics and deriving better document representations.

Keywords: Probabilistic Methods
[ Slides] 

Author Information

Zhibin Duan (Xidian University)
Dongsheng Wang (Xidian University)
Bo Chen (School of Electronic Engineering, Xidian University)

Bo Chen, Ph.D., Professor. Before joining the Department of Electronic Engineering in Xidian University in 2013, I was a post-doc researcher, research scientist and senior research scientist at the Department of Electrical and Computer Engineering in Duke University. In 2013 and 2014, I was elected into the Program for New Century Excellent Talents in University and the Program for Thousand Youth Talents respectively. I am interested in developing statistical machine learning methods for the complex and large-scale data. My current interests are in statistical signal processing, statistical machine learning, deep learning and their applications to radar target detection and recognition.

CHAOJIE WANG (XIDIAN UNIVERSITY)
Wenchao Chen (Xi'dian University)
yewen li (Xidian University)
Jie Ren (Xidian University)
Mingyuan Zhou (University of Texas at Austin)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors