Can I read STANLEY: Stochastic Gradient Anisotropic Langevin Dynamics for Learning Energy-Based Models on EtoBox?

STANLEY: Stochastic Gradient Anisotropic Langevin Dynamics for Learning Energy-Based Models by Karimi, Belhal; Xie, Jianwen; Li, Ping is a scholarly article available to read on EtoBox.

What is STANLEY: Stochastic Gradient Anisotropic Langevin Dynamics for Learning Energy-Based Models about?

We propose in this paper, STANLEY, a STochastic gradient ANisotropic LangEvin dYnamics, for sampling high dimensional data. With the growing efficacy and potential of Energy-Based modeling, also known as non-normalized probabilistic modeling, for modeling a generative process of different natures of high dimensional data observations, we present an end-to-end learning algorithm for Energy-Based models (EBM) with the purpose of improving the quality of the resulting sampled data points. While the unknown normalizing constant of EBMs makes the training procedure intractable, resorting to Markov Chain Monte Carlo (MCMC) is in general a viable option. Realizing what MCMC entails for the EBM training, we propose in this paper, a novel high dimensional sampling method, based on an anisotropic stepsize and a gradient-informed covariance matrix, embedded into a discretized Langevin diffusion. We motivate the necessity for an anisotropic update of the negative samples in the Markov Chain by the nonlinearity of the backbone of the EBM, here a Convolutional Neural Network. Our resulting method, namely STANLEY, is an optimization algorithm for training Energy-Based models via our newly introdu

Author

Karimi, Belhal; Xie, Jianwen; Li, Ping

Published

2023

Language

EN