Can I read Fast and Simple Natural-Gradient Variational Inference with Mixture of Exponential-family Approximations on EtoBox?

Fast and Simple Natural-Gradient Variational Inference with Mixture of Exponential-family Approximations by Lin, Wu; Khan, Mohammad Emtiyaz; Schmidt, Mark is a scholarly article available to read on EtoBox.

What is Fast and Simple Natural-Gradient Variational Inference with Mixture of Exponential-family Approximations about?

Natural-gradient methods enable fast and simple algorithms for variational inference, but due to computational difficulties, their use is mostly limited to \emph{minimal} exponential-family (EF) approximations. In this paper, we extend their application to estimate \emph{structured} approximations such as mixtures of EF distributions. Such approximations can fit complex, multimodal posterior distributions and are generally more accurate than unimodal EF approximations. By using a \emph{minimal conditional-EF} representation of such approximations, we derive simple natural-gradient updates. Our empirical results demonstrate a faster convergence of our natural-gradient method compared to black-box gradient-based methods with reparameterization gradients. Our work expands the scope of natural gradients for Bayesian inference and makes them more widely applicable than before.

Author

Lin, Wu; Khan, Mohammad Emtiyaz; Schmidt, Mark

Published

2019

Language

EN