Timezone: »
Topic models have been widely explored as probabilistic generative models of documents. Traditional inference methods have sought closed-form derivations for updating the models, however as the expressiveness of these models grows, so does the difficulty of performing fast and accurate inference over their parameters. This paper presents alternative neural approaches to topic modelling by providing parameterisable distributions over topics which permit training by backpropagation in the framework of neural variational inference. In addition, with the help of a stick-breaking construction, we propose a recurrent network that is able to discover a notionally unbounded number of topics, analogous to Bayesian non-parametric topic models. Experimental results on the MXM Song Lyrics, 20NewsGroups and Reuters News datasets demonstrate the effectiveness and efficiency of these neural topic models.
Author Information
Yishu Miao (University of Oxford)
Edward Grefenstette (Deepmind)
Phil Blunsom (Oxford University and DeepMind)
Related Events (a corresponding poster, oral, or spotlight)
-
2017 Poster: Discovering Discrete Latent Topics with Neural Variational Inference »
Wed. Aug 9th 08:30 AM -- 12:00 PM Room Gallery #71
More from the Same Authors
-
2022 Poster: Evolving Curricula with Regret-Based Environment Design »
Jack Parker-Holder · Minqi Jiang · Michael Dennis · Mikayel Samvelyan · Jakob Foerster · Edward Grefenstette · Tim Rocktäschel -
2022 Spotlight: Evolving Curricula with Regret-Based Environment Design »
Jack Parker-Holder · Minqi Jiang · Michael Dennis · Mikayel Samvelyan · Jakob Foerster · Edward Grefenstette · Tim Rocktäschel -
2021 Poster: Prioritized Level Replay »
Minqi Jiang · Edward Grefenstette · Tim Rocktäschel -
2021 Spotlight: Prioritized Level Replay »
Minqi Jiang · Edward Grefenstette · Tim Rocktäschel -
2020 Poster: Learning Reasoning Strategies in End-to-End Differentiable Proving »
Pasquale Minervini · Sebastian Riedel · Pontus Stenetorp · Edward Grefenstette · Tim Rocktäschel -
2019 Poster: CompILE: Compositional Imitation Learning and Execution »
Thomas Kipf · Yujia Li · Hanjun Dai · Vinicius Zambaldi · Alvaro Sanchez-Gonzalez · Edward Grefenstette · Pushmeet Kohli · Peter Battaglia -
2019 Oral: CompILE: Compositional Imitation Learning and Execution »
Thomas Kipf · Yujia Li · Hanjun Dai · Vinicius Zambaldi · Alvaro Sanchez-Gonzalez · Edward Grefenstette · Pushmeet Kohli · Peter Battaglia -
2017 Workshop: Learning to Generate Natural Language »
Yishu Miao · Wang Ling · Tsung-Hsien Wen · Kris Cao · Daniela Gerz · Phil Blunsom · Chris Dyer -
2017 Poster: Latent Intention Dialogue Models »
Tsung-Hsien Wen · Yishu Miao · Phil Blunsom · Stephen J Young -
2017 Talk: Latent Intention Dialogue Models »
Tsung-Hsien Wen · Yishu Miao · Phil Blunsom · Stephen J Young