Timezone: »

 
Poster
Variational (Gradient) Estimate of the Score Function in Energy-based Latent Variable Models
Fan Bao · Kun Xu · Chongxuan Li · Lanqing Hong · Jun Zhu · Bo Zhang

Tue Jul 20 09:00 PM -- 11:00 PM (PDT) @
in Poster Session 2 »

This paper presents new estimates of the score function and its gradient with respect to the model parameters in a general energy-based latent variable model (EBLVM). The score function and its gradient can be expressed as combinations of expectation and covariance terms over the (generally intractable) posterior of the latent variables. New estimates are obtained by introducing a variational posterior to approximate the true posterior in these terms. The variational posterior is trained to minimize a certain divergence (e.g., the KL divergence) between itself and the true posterior. Theoretically, the divergence characterizes upper bounds of the bias of the estimates. In principle, our estimates can be applied to a wide range of objectives, including kernelized Stein discrepancy (KSD), score matching (SM)-based methods and exact Fisher divergence with a minimal model assumption. In particular, these estimates applied to SM-based methods outperform existing methods in learning EBLVMs on several image datasets.

Keywords: Computer Vision · Deep Learning · Algorithms -> Image Segmentation; Algorithms -> Similarity and Distance Learning; Algorithms · Spectral Methods; Applications

Author Information

Fan Bao (Tsinghua University)
Kun Xu (Tsinghua University)
Chongxuan Li (Tsinghua University)
Lanqing Hong (Huawei)
Jun Zhu (Tsinghua University)
Bo Zhang (Tsinghua University)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors