Open AccessProceedings Article
Stochastic Gradient Riemannian Langevin Dynamics on the Probability Simplex
Sam Patterson,Yee Whye Teh +1 more
- Vol. 26, pp 3102-3110
TLDR
A new method, Stochastic gradient Riemannian Langevin dynamics, which is simple to implement and can be applied to large scale data is proposed and achieves substantial performance improvements over the state of the art online variational Bayesian methods.Abstract:
In this paper we investigate the use of Langevin Monte Carlo methods on the probability simplex and propose a new method, Stochastic gradient Riemannian Langevin dynamics, which is simple to implement and can be applied to large scale data. We apply this method to latent Dirichlet allocation in an online mini-batch setting, and demonstrate that it achieves substantial performance improvements over the state of the art online variational Bayesian methods.read more
Citations
More filters
Journal Article
Riemann manifold Langevin and Hamiltonian Monte Carlo methods
Mark Girolami,Ben Calderhead +1 more
TL;DR: The methodology proposed automatically adapts to the local structure when simulating paths across this manifold, providing highly efficient convergence and exploration of the target density, and substantial improvements in the time‐normalized effective sample size are reported when compared with alternative sampling approaches.
Proceedings Article
Stochastic Gradient Hamiltonian Monte Carlo
TL;DR: A variant that uses second-order Langevin dynamics with a friction term that counteracts the effects of the noisy gradient, maintaining the desired target distribution as the invariant distribution is introduced.
Posted Content
Adding Gradient Noise Improves Learning for Very Deep Networks
Arvind Neelakantan,Luke Vilnis,Quoc V. Le,Ilya Sutskever,Lukasz Kaiser,Karol Kurach,James Martens +6 more
TL;DR: This paper explores the low-overhead and easy-to-implement optimization technique of adding annealed Gaussian noise to the gradient, which it is found surprisingly effective when training these very deep architectures.
Posted Content
A Complete Recipe for Stochastic Gradient MCMC
TL;DR: In this article, a general recipe for constructing MCMCMCMC samplers, including stochastic gradient versions, based on continuous Markov processes specified via two matrices is provided.
Proceedings Article
Bayesian Sampling Using Stochastic Gradient Thermostats
TL;DR: This work shows that one can leverage a small number of additional variables to stabilize momentum fluctuations induced by the unknown noise inynamics-based sampling methods.
References
More filters
Journal ArticleDOI
Latent dirichlet allocation
TL;DR: This work proposes a generative model for text and other collections of discrete data that generalizes or improves on several previous models including naive Bayes/unigram, mixture of unigrams, and Hofmann's aspect model.
Journal ArticleDOI
Inference of population structure using multilocus genotype data
TL;DR: Pritch et al. as discussed by the authors proposed a model-based clustering method for using multilocus genotype data to infer population structure and assign individuals to populations, which can be applied to most of the commonly used genetic markers, provided that they are not closely linked.
Proceedings Article
Latent Dirichlet Allocation
TL;DR: This paper proposed a generative model for text and other collections of discrete data that generalizes or improves on several previous models including naive Bayes/unigram, mixture of unigrams, and Hof-mann's aspect model, also known as probabilistic latent semantic indexing (pLSI).
Journal ArticleDOI
A Stochastic Approximation Method
Herbert Robbins,Sutton Monro +1 more
TL;DR: In this article, a method for making successive experiments at levels x1, x2, ··· in such a way that xn will tend to θ in probability is presented.
BookDOI
Markov Chain Monte Carlo in Practice
TL;DR: The Markov Chain Monte Carlo Implementation Results Summary and Discussion MEDICAL MONITORING Introduction Modelling Medical Monitoring Computing Posterior Distributions Forecasting Model Criticism Illustrative Application Discussion MCMC for NONLINEAR HIERARCHICAL MODELS.