Summary. Markov chain Monte Carlo and sequential Monte Carlo methods have emerged as the two main tools to sample from high dimensional probability distributions. Although asymptotic convergence of Markov chain Monte Carlo algorithms is ensured under weak assumptions, the performance of these algorithms is unreliable when the proposal distributions that are used to explore the space are poorly chosen and/or if highly correlated variables are updated independently. We show here how it is possible to build efficient high dimensional proposal distributions by using sequential Monte Carlo methods. This allows us not only to improve over standard Markov chain Monte Carlo schemes but also to make Bayesian inference feasible for a large class of statistical models where this was not previously so. We demonstrate these algorithms on a non-linear state space model and a Levy-driven stochastic volatility model.

https://rss.onlinelibrary.wiley.com/doi/pdf/10.1111/j.1467-9868.2009.00736.x

Particle Markov chain Monte Carlo methods

We propose to combine two quite powerful ideas that have recently appeared in the Markov chain Monte Carlo literature: adaptive Metropolis samplers and delayed rejection. The ergodicity of the resulting non-Markovian sampler is proved, and the efficiency of the combination is demonstrated with various examples. We present situations where the combination outperforms the original methods: adaptation clearly enhances efficiency of the delayed rejection algorithm in cases where good proposal distributions are not available. Similarly, delayed rejection provides a systematic remedy when the adaptation process has a slow start.

DRAM: Efficient adaptive MCMC

The paper proposes Metropolis adjusted Langevin and Hamiltonian Monte Carlo sampling methods defined on the Riemann manifold to resolve the shortcomings of existing Monte Carlo algorithms when sampling from target densities that may be high dimensional and exhibit strong correlations. The methods provide fully automated adaptation mechanisms that circumvent the costly pilot runs that are required to tune proposal densities for Metropolis–Hastings or indeed Hamiltonian Monte Carlo and Metropolis adjusted Langevin algorithms. This allows for highly efficient sampling even in very high dimensions where different scalings may be required for the transient and stationary phases of the Markov chain. The methodology proposed exploits the Riemann geometry of the parameter space of statistical models and thus automatically adapts to the local structure when simulating paths across this manifold, providing highly efficient convergence and exploration of the target density. The performance of these Riemann manifold Monte Carlo methods is rigorously assessed by performing inference on logistic regression models, log-Gaussian Cox point processes, stochastic volatility models and Bayesian estimation of dynamic systems described by non-linear differential equations. Substantial improvements in the time-normalized effective sample size are reported when compared with alternative sampling approaches. MATLAB code that is available from http://www.ucl.ac.uk/statistics/research/rmhmc allows replication of all the results reported.

/pdf/riemann-manifold-langevin-and-hamiltonian-monte-carlo-254effw0xm.pdf

Riemann manifold Langevin and Hamiltonian Monte Carlo methods

Simulation and the monte carlo method

Bayesian inference often requires integrating some function with respect to a posterior distribution. Monte Carlo methods are sampling algorithms that allow to compute these integrals numerically when they are not analytically tractable. We review here the basic principles and the most common Monte Carlo algorithms, among which rejection sampling, importance sampling and Monte Carlo Markov chain (MCMC) methods. We give intuition on the theoretical justification of the algorithms as well as practical advice, trying to relate both. We discuss the application of Monte Carlo in experimental physics, and point to landmarks in the literature for the curious reader.

Monte Carlo methods

We look at adaptive Markov chain Monte Carlo algorithms that generate stochastic processes based on sequences of transition kernels, where each transition kernel is allowed to depend on the history of the process. We show under certain conditions that the stochastic process generated is ergodic, with appropriate stationary distribution. We use this result to analyse an adaptive version of the random walk Metropolis algorithm where the scale parameter σ is sequentially adapted using a Robbins-Monro type algorithm in order to find the optimal scale parameter σopt. We close with a simulation example.

/pdf/on-adaptive-markov-chain-monte-carlo-algorithms-1rxjch8a3x.pdf

On adaptive Markov chain Monte Carlo algorithms

This paper extends some adaptive schemes that have been developed for the Random Walk Metropolis algorithm to more general versions of the Metropolis-Hastings (MH) algorithm, particularly to the Metropolis Adjusted Langevin algorithm of Roberts and Tweedie (1996). Our simulations show that the adaptation drastically improves the performance of such MH algorithms. We study the convergence of the algorithm. Our proves are based on a new approach to the analysis of stochastic approximation algorithms based on mixingales theory.

http://dept.stat.lsa.umich.edu/~yvesa/atmala.pdf

An Adaptive Version for the Metropolis Adjusted Langevin Algorithm with a Truncated Drift

Iterated filtering algorithms are stochastic optimization procedures for latent variable models that recursively combine parameter perturbations with latent variable reconstruction. Previously, theoretical support for these algorithms has been based on the use of conditional moments of perturbed parameters to approximate derivatives of the log likelihood function. Here, a theoretical approach is introduced based on the convergence of an iterated Bayes map. An algorithm supported by this theory displays substantial numerical improvement on the computational challenge of inferring parameters of a partially observed Markov process.

https://www.pnas.org/content/pnas/112/3/719.full.pdf

Inference for dynamic and latent variable models via iterated, perturbed Bayes maps

A large number of statistical models are “doubly-intractable”:
the likelihood normalising term, which is a function of the model parameters,
is intractable, as well as the marginal likelihood (model evidence). This
means that standard inference techniques to sample from the posterior, such
as Markov chain Monte Carlo (MCMC), cannot be used. Examples include,
but are not confined to, massive Gaussian Markov random fields, autologistic
models and Exponential random graph models. A number of approximate
schemes based on MCMC techniques, Approximate Bayesian computation
(ABC) or analytic approximations to the posterior have been suggested, and
these are reviewed here. Exact MCMC schemes, which can be applied to a
subset of doubly-intractable distributions, have also been developed and are
described in this paper. As yet, no general method exists which can be applied
to all classes of models with doubly-intractable posteriors.
In addition, taking inspiration from the Physics literature, we study an alternative
method based on representing the intractable likelihood as an infinite
series. Unbiased estimates of the likelihood can then be obtained by
finite time stochastic truncation of the series via Russian Roulette sampling,
although the estimates are not necessarily positive. Results from the Quantum
Chromodynamics literature are exploited to allow the use of possibly negative
estimates in a pseudo-marginal MCMC scheme such that expectations
with respect to the posterior distribution are preserved. The methodology is
reviewed on well-known examples such as the parameters in Ising models,
the posterior for Fisher–Bingham distributions on the d-Sphere and a largescale
Gaussian Markov Random Field model describing the Ozone Column
data. This leads to a critical assessment of the strengths and weaknesses of
the methodology with pointers to ongoing research.

/pdf/on-russian-roulette-estimates-for-bayesian-inference-with-1vxac6e52w.pdf

On Russian Roulette Estimates for Bayesian Inference with Doubly-Intractable Likelihoods

We consider optimal temperature spacings for Metropolis-coupled Markov chain Monte Carlo (MCMCMC) and Simulated Tempering algorithms. We prove that, under certain conditions, it is optimal (in terms of maximising the expected squared jumping distance) to space the temperatures so that the proportion of temperature swaps which are accepted is approximately 0.234. This generalises related work by physicists, and is consistent with previous work about optimal scaling of random-walk Metropolis algorithms.

/pdf/towards-optimal-scaling-of-metropolis-coupled-markov-chain-5brmvteip1.pdf

Yves F. Atchadé

Papers

On adaptive Markov chain Monte Carlo algorithms

An Adaptive Version for the Metropolis Adjusted Langevin Algorithm with a Truncated Drift

Inference for dynamic and latent variable models via iterated, perturbed Bayes maps

On Russian Roulette Estimates for Bayesian Inference with Doubly-Intractable Likelihoods

Towards optimal scaling of metropolis-coupled Markov chain Monte Carlo