From the Publisher:
Artificial "neural networks" are now widely used as flexible models for regression classification applications, but questions remain regarding what these models mean, and how they can safely be used when training data is limited. Bayesian Learning for Neural Networks shows that Bayesian methods allow complex neural network models to be used without fear of the "overfitting" that can occur with traditional neural network learning methods. Insight into the nature of these complex Bayesian models is provided by a theoretical investigation of the priors over functions that underlie them. Use of these models in practice is made possible using Markov chain Monte Carlo techniques. Both the theoretical and computational aspects of this work are of wider statistical interest, as they contribute to a better understanding of how Bayesian methods can be applied to complex problems. Presupposing only the basic knowledge of probability and statistics, this book should be of interest to many researchers in statistics, engineering, and artificial intelligence. Software for Unix systems that implements the methods described is freely available over the Internet.

Bayesian learning for neural networks

Hamiltonian dynamics can be used to produce distant proposals for the Metropolis algorithm, thereby avoiding the slow exploration of the state space that results from the diffusive behaviour of simple random-walk proposals. Though originating in physics, Hamiltonian dynamics can be applied to most problems with continuous state spaces by simply introducing fictitious "momentum" variables. A key to its usefulness is that Hamiltonian dynamics preserves volume, and its trajectories can thus be used to define complex mappings without the need to account for a hard-to-compute Jacobian factor - a property that can be exactly maintained even when the dynamics is approximated by discretizing time. In this review, I discuss theoretical and practical aspects of Hamiltonian Monte Carlo, and present some of its variations, including using windows of states for deciding on acceptance or rejection, computing trajectories using fast approximations, tempering during the course of a trajectory to handle isolated modes, and short-cut methods that prevent useless trajectories from taking much computation time.

MCMC using Hamiltonian dynamics

Dynamic strength of molecular adhesion bonds.

One of Feynman's early applications of path integrals was to superfluid $^{4}\mathrm{He}$. He showed that the thermodynamic properties of Bose systems are exactly equivalent to those of a peculiar type of interacting classical "ring polymer." Using this mapping, one can generalize Monte Carlo simulation techniques commonly used for classical systems to simulate boson systems. In this review, the author introduces this picture of a boson superfluid and shows how superfluidity and Bose condensation manifest themselves. He shows the excellent agreement between simulations and experimental measurements on liquid and solid helium for such quantities as pair correlations, the superfluid density, the energy, and the momentum distribution. Major aspects of computational techniques developed for a boson superfluid are discussed: the construction of more accurate approximate density matrices to reduce the number of points on the path integral, sampling techniques to move through the space of exchanges and paths quickly, and the construction of estimators for various properties such as the energy, the momentum distribution, the superfluid density, and the exchange frequency in a quantum crystal. Finally the path-integral Monte Carlo method is compared to other quantum Monte Carlo methods.

Path integrals in the theory of condensed helium

Hamiltonian dynamics can be used to produce distant proposals for the Metropolis algorithm, thereby avoiding the slow exploration of the state space that results from the diffusive behaviour of simple random-walk proposals. Though originating in physics, Hamiltonian dynamics can be applied to most problems with continuous state spaces by simply introducing fictitious “momentum” variables. A key to its usefulness is that Hamiltonian dynamics preserves volume, and its trajectories can thus be used to define complex mappings without the need to account for a hard-to-compute Jacobian factor — a property that can be exactly maintained even when the dynamics is approximated by discretizing time. In this review, I discuss theoretical and practical aspects of Hamiltonian Monte Carlo, and present some of its variations, including using windows of states for deciding on acceptance or rejection, computing trajectories using fast approximations, tempering during the course of a trajectory to handle isolated modes, and short-cut methods that prevent useless trajectories from taking much computation time.

/pdf/mcmc-using-hamiltonian-dynamics-1w7tfbaxia.pdf

MCMC Using Hamiltonian Dynamics

A new Monte Carlo simulation procedure is developed which is expected to produce more rapid convergence than the standard Metropolis method. The trial particle moves are chosen in accord with a Brownian dynamics algorithm rather than at random. For two model systems, a string of point masses joined by harmonic springs and a cluster of charged soft spheres, the new procedure is compared to the standard one and shown to manifest a more rapid convergence rate for some important energetic and structural properties.

J. D. Doll

Papers

Brownian dynamics as smart Monte Carlo simulation