Timezone: »

 
Oral
Rao-Blackwellized Stochastic Gradients for Discrete Distributions
Runjing Liu · Jeffrey Regier · Nilesh Tripuraneni · Michael Jordan · Jon McAuliffe

Tue Jun 11 04:25 PM -- 04:30 PM (PDT) @ Room 101
in Bayesian Methods »

We wish to compute the gradient of an expectation over a finite or countably infinite sample space having K ≤ ∞ categories. When K is indeed infinite, or finite but very large, the relevant summation is intractable. Accordingly, various stochastic gradient estimators have been proposed. In this paper, we describe a technique that can be applied to reduce the variance of any such estimator, without changing its bias—in particular, unbiasedness is retained. We show that our technique is an instance of Rao-Blackwellization, and we demonstrate the improvement it yields on a semi-supervised classification problem and a pixel attention task.

[ Slides]  [ Video] 

Author Information

Runjing Liu (UC Berkeley)
Jeffrey Regier (UC Berkeley)
Nilesh Tripuraneni (UC Berkeley)
Michael Jordan (UC Berkeley)
Jon McAuliffe (Voleon Group and University of California at Berkeley)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors