Timezone: »
To address the challenge of backpropagating the gradient through categorical variables, we propose the augment-REINFORCE-swap-merge (ARSM) gradient estimator that is unbiased and has low variance. ARSM first uses variable augmentation, REINFORCE, and Rao-Blackwellization to re-express the gradient as an expectation under the Dirichlet distribution, then uses variable swapping to construct differently expressed but equivalent expectations, and finally shares common random numbers between these expectations to achieve significant variance reduction. Experimental results show ARSM closely resembles the performance of the true gradient for optimization in univariate settings; outperforms existing estimators by a large margin when applied to categorical variational auto-encoders; and provides a "try-and-see self-critic" variance reduction method for discrete-action policy gradient, which removes the need of estimating baselines by generating a random number of pseudo actions and estimating their action-value functions.
Author Information
Mingzhang Yin (University of Texas at Austin)
Yuguang Yue (University of Texas at Austin)
Mingyuan Zhou (University of Texas at Austin)
Related Events (a corresponding poster, oral, or spotlight)
-
2019 Oral: ARSM: Augment-REINFORCE-Swap-Merge Estimator for Gradient Backpropagation Through Categorical Variables »
Thu Jun 13th 11:35 -- 11:40 PM Room Grand Ballroom
More from the Same Authors
-
2020 Poster: On hyperparameter tuning in general clustering problemsm »
Xinjie Fan · Yuguang Yue · Purnamrita Sarkar · Y. X. Rachel Wang -
2020 Poster: Thompson Sampling via Local Uncertainty »
Zhendong Wang · Mingyuan Zhou -
2020 Poster: Bayesian Graph Neural Networks with Adaptive Connection Sampling »
Arman Hasanzadeh · Ehsan Hajiramezanali · Shahin Boluki · Mingyuan Zhou · Nick Duffield · Krishna Narayanan · Xiaoning Qian -
2020 Poster: Recurrent Hierarchical Topic-Guided RNN for Language Generation »
Dandan Guo · Bo Chen · Ruiying Lu · Mingyuan Zhou -
2019 Poster: Convolutional Poisson Gamma Belief Network »
CHAOJIE WANG · Bo Chen · SUCHENG XIAO · Mingyuan Zhou -
2019 Poster: Locally Private Bayesian Inference for Count Models »
Aaron Schein · Steven Wu · Alexandra Schofield · Mingyuan Zhou · Hanna Wallach -
2019 Oral: Convolutional Poisson Gamma Belief Network »
CHAOJIE WANG · Bo Chen · SUCHENG XIAO · Mingyuan Zhou -
2019 Oral: Locally Private Bayesian Inference for Count Models »
Aaron Schein · Steven Wu · Alexandra Schofield · Mingyuan Zhou · Hanna Wallach -
2018 Poster: Inter and Intra Topic Structure Learning with Word Embeddings »
He Zhao · Lan Du · Wray Buntine · Mingyuan Zhou -
2018 Oral: Inter and Intra Topic Structure Learning with Word Embeddings »
He Zhao · Lan Du · Wray Buntine · Mingyuan Zhou -
2018 Poster: Semi-Implicit Variational Inference »
Mingzhang Yin · Mingyuan Zhou -
2018 Oral: Semi-Implicit Variational Inference »
Mingzhang Yin · Mingyuan Zhou -
2017 Poster: Deep Latent Dirichlet Allocation with Topic-Layer-Adaptive Stochastic Gradient Riemannian MCMC »
Yulai Cong · Bo Chen · Hongwei Liu · Mingyuan Zhou -
2017 Talk: Deep Latent Dirichlet Allocation with Topic-Layer-Adaptive Stochastic Gradient Riemannian MCMC »
Yulai Cong · Bo Chen · Hongwei Liu · Mingyuan Zhou