Timezone: »

 
Poster
Variable Skipping for Autoregressive Range Density Estimation
Eric Liang · Zongheng Yang · Ion Stoica · Pieter Abbeel · Yan Duan · Peter Chen

Tue Jul 14 09:00 AM -- 09:45 AM & Tue Jul 14 09:00 PM -- 09:45 PM (PDT) @
in Poster Session 3 »

Deep autoregressive models compute point likelihood estimates of individual data points. However, many applications (i.e., database cardinality estimation), require estimating range densities, a capability that is under-explored by current neural density estimation literature. In these applications, fast and accurate range density estimates over high-dimensional data directly impact user-perceived performance. In this paper, we explore a technique for accelerating range density estimation over deep autoregressive models. This technique, called variable skipping, exploits the sparse structure of range density queries to avoid sampling unnecessary variables during approximate inference. We show that variable skipping provides 10-100x efficiency improvements when targeting challenging high-quantile error metrics, enables complex applications such as text pattern matching, and can be realized via a simple data augmentation procedure without changing the usual maximum likelihood objective.

Keywords: Approximate Inference · Deep Generative Models · Deep Sequence Models · Hardware and Systems · Deep Learning - Generative Models and Autoencoders

Author Information

Eric Liang (University of California, Berkeley)
Zongheng Yang (UC Berkeley)
Ion Stoica (UC Berkeley)
Pieter Abbeel (UC Berkeley & Covariant)
Yan Duan (COVARIANT.AI)
Peter Chen (COVARIANT.AI)

More from the Same Authors