Skip to yearly menu bar Skip to main content


Poster

Handling the Positive-Definite Constraint in the Bayesian Learning Rule

Wu Lin · Mark Schmidt · Mohammad Emtiyaz Khan

Keywords: [ Approximate Inference ] [ Probabilistic Inference - Approximate, Monte Carlo, and Spectral Methods ]


Abstract:

The Bayesian learning rule is a natural-gradient variational inference method, which not only contains many existing learning algorithms as special cases but also enables the design of new algorithms. Unfortunately, when variational parameters lie in an open constraint set, the rule may not satisfy the constraint and requires line-searches which could slow down the algorithm. In this work, we address this issue for positive-definite constraints by proposing an improved rule that naturally handles the constraints. Our modification is obtained by using Riemannian gradient methods, and is valid when the approximation attains a block-coordinate natural parameterization (e.g., Gaussian distributions and their mixtures). Our method outperforms existing methods without any significant increase in computation. Our work makes it easier to apply the rule in the presence of positive-definite constraints in parameter spaces.

Chat is not available.