Timezone: »

 
Poster
Einsum Networks: Fast and Scalable Learning of Tractable Probabilistic Circuits
Robert Peharz · Steven Lang · Antonio Vergari · Karl Stelzner · Alejandro Molina · Martin Trapp · Guy Van den Broeck · Kristian Kersting · Zoubin Ghahramani

Wed Jul 15 12:00 PM -- 12:45 PM & Thu Jul 16 01:00 AM -- 01:45 AM (PDT) @ Virtual
in Poster Session 26 »

Probabilistic circuits (PCs) are a promising avenue for probabilistic modeling, as they permit a wide range of exact and efficient inference routines. Recent ``deep-learning-style'' implementations of PCs strive for a better scalability, but are still difficult to train on real-world data, due to their sparsely connected computational graphs. In this paper, we propose Einsum Networks (EiNets), a novel implementation design for PCs, improving prior art in several regards. At their core, EiNets combine a large number of arithmetic operations in a single monolithic einsum-operation, leading to speedups and memory savings of up to two orders of magnitude, in comparison to previous implementations. As an algorithmic contribution, we show that the implementation of Expectation-Maximization (EM) can be simplified for PCs, by leveraging automatic differentiation. Furthermore, we demonstrate that EiNets scale well to datasets which were previously out of reach, such as SVHN and CelebA, and that they can be used as faithful generative image models.

Keywords: Generative Models · Graphical Models · Probabilistic Inference - Models and Probabilistic Programming
[ Slides] 

Author Information

Robert Peharz (Eindhoven University of Technology)
Steven Lang (Technical University of Darmstadt)
Antonio Vergari (University of California, Los Angeles)
Karl Stelzner (TU Darmstadt)
Alejandro Molina (TU Darmstadt)
Martin Trapp (Graz University of Technology)
Guy Van den Broeck (University of California, Los Angeles)
Kristian Kersting (TU Darmstadt)
Zoubin Ghahramani (University of Cambridge & Uber)

Zoubin Ghahramani is a Professor at the University of Cambridge, and Chief Scientist at Uber. He is also Deputy Director of the Leverhulme Centre for the Future of Intelligence, was a founding Director of the Alan Turing Institute and co-founder of Geometric Intelligence (now Uber AI Labs). His research focuses on probabilistic approaches to machine learning and AI. In 2015 he was elected a Fellow of the Royal Society.

More from the Same Authors