We explore in this chapter questions of existence and uniqueness for solutions to stochastic differential equations and offer a study of their properties. This endeavor is really a study of diffusion processes. Loosely speaking, the term diffusion is attributed to a Markov process which has continuous sample paths and can be characterized in terms of its infinitesimal generator.

Stochastic Differential Equations

This chapter introduces the finite element method (FEM) as a tool for solution of classical electromagnetic problems. Although we discuss the main points in the application of the finite element method to electromagnetic design, including formulation and implementation, those who seek deeper understanding of the finite element method should consult some of the works listed in the bibliography section.

Introduction to the Finite Element Method

: The unpolarized absorption and circular dichroism spectra of the fundamental vibrational transitions of the chiral molecule, 4-methyl-2-oxetanone, are calculated ab initio. Harmonic force fields are obtained using Density Functional Theory (DFT), MP2, and SCF methodologies and a 5S4P2D/3S2P (TZ2P) basis set. DFT calculations use the Local Spin Density Approximation (LSDA), BLYP, and Becke3LYP (B3LYP) density functionals. Mid-IR spectra predicted using LSDA, BLYP, and B3LYP force fields are of significantly different quality, the B3LYP force field yielding spectra in clearly superior, and overall excellent, agreement with experiment. The MP2 force field yields spectra in slightly worse agreement with experiment than the B3LYP force field. The SCF force field yields spectra in poor agreement with experiment.The basis set dependence of B3LYP force fields is also explored: the 6-31G* and TZ2P basis sets give very similar results while the 3-21G basis set yields spectra in substantially worse agreements with experiment. jg

Ab initio calculation of vibrational absorption and circular dichroism spectra using density functional force fields

Stochastic equations in infinite dimensions

The paper proposes Metropolis adjusted Langevin and Hamiltonian Monte Carlo sampling methods defined on the Riemann manifold to resolve the shortcomings of existing Monte Carlo algorithms when sampling from target densities that may be high dimensional and exhibit strong correlations. The methods provide fully automated adaptation mechanisms that circumvent the costly pilot runs that are required to tune proposal densities for Metropolis-Hastings or indeed Hamiltonian Monte Carlo and Metropolis adjusted Langevin algorithms. This allows for highly efficient sampling even in very high dimensions where different scalings may be required for the transient and stationary phases of the Markov chain. The methodology proposed exploits the Riemann geometry of the parameter space of statistical models and thus automatically adapts to the local structure when simulating paths across this manifold, providing highly efficient convergence and exploration of the target density. The performance of these Riemann manifold Monte Carlo methods is rigorously assessed by performing inference on logistic regression models, log-Gaussian Cox point processes, stochastic volatility models and Bayesian estimation of dynamic systems described by non-linear differential equations. Substantial improvements in the time-normalized effective sample size are reported when compared with alternative sampling approaches. MATLAB code that is available from http://www.ucl.ac.uk/statistics/research/rmhmc allows replication of all the results reported.

Riemann manifold Langevin and Hamiltonian Monte Carlo methods

We consider estimation of scalar functions that determine the dynamics of diffusion processes. It has been recently shown that nonparametric maximum likelihood estimation is ill-posed in this context. We adopt a probabilistic approach to regularize the problem by the adoption of a prior distribution for the unknown functional. A Gaussian prior measure is chosen in the function space by specifying its precision operator as an appropriate differential operator. We establish that a Bayesian–Gaussian conjugate analysis for the drift of one-dimensional nonlinear diffusions is feasible using high-frequency data, by expressing the loglikelihood as a quadratic function of the drift, with sufficient statistics given by the local time process and the end points of the observed path. Computationally efficient posterior inference is carried out using a finite element method. We embed this technology in partially observed situations and adopt a data augmentation approach whereby we iteratively generate missing data paths and draws from the unknown functional. Our methodology is applied to estimate the drift of models used in molecular dynamics and financial econometrics using high- and low-frequency observations. We discuss extensions to other partially observed schemes and connections to other types of nonparametric inference.

https://homepages.warwick.ac.uk/~masdr/JOURNALPUBS/stuart95.pdf

Nonparametric estimation of diffusions: a differential equations approach

Hypoelliptic diffusion processes can be used to model a variety of phenomena in applications ranging from molecular dynamics to audio signal analysis. We study parameter estimation for such processes in situations where we observe some components of the solution at discrete times. Since exact likelihoods for the transition densities are typically not known, approximations are used that are expected to work well in the limit of small intersample times Δt and large total observation times N Δt. Hypoellipticity together with partial observation leads to ill conditioning requiring a judicious combination of approximate likelihoods for the various parameters to be estimated. We combine these in a deterministic scan Gibbs sampler alternating between missing data in the unobserved solution components, and parameters. Numerical experiments illustrate asymptotic consistency of the method when applied to simulated data. The paper concludes with an application of the Gibbs sampler to molecular dynamics data.

/pdf/parameter-estimation-for-partially-observed-hypoelliptic-581d1c52se.pdf

Parameter estimation for partially observed hypoelliptic diffusions

Hypoelliptic diffusion processes can be used to model a variety of phenomena in applications ranging from molecular dynamics to audio signal analysis. We study parameter estimation for such processes in situations where we observe some components of the solution at discrete times. Since exact likelihoods for the transition densities are typically not known, approximations are used that are expected to work well in the limit of small inter-sample times $\Delta t$ and large total observation times $N\Delta t$. Hypoellipticity together with partial observation leads to ill-conditioning requiring a judicious combination of approximate likelihoods for the various parameters to be estimated. We combine these in a deterministic scan Gibbs sampler alternating between missing data in the unobserved solution components, and parameters. Numerical experiments illustrate asymptotic consistency of the method when applied to simulated data. The paper concludes with application of the Gibbs sampler to molecular dynamics data.

Parameter Estimation for Partially Observed Hypoelliptic Diffusions

Posterior consistency via precision operators for Bayesian nonparametric drift estimation in SDEs

Many applications require efficient sampling from Gaussian distributions. The method of choice depends on the dimension of the problem as well as the structure of the covariance- (Σ) or precision matrix (Q). The most common black-box routine for computing a sample is based on Cholesky factorization. In high dimensions, computing the Cholesky factor of Σ or Q may be prohibitive due to accumulation of more non-zero entries in the factor than is possible to store in memory. We compare different methods for computing the samples iteratively adapting ideas from numerical linear algebra. These methods assume that matrix vector products, Qv, are fast to compute. We show that some of the methods are competitive and faster than Cholesky sampling and that a parallel version of one method on a Graphical Processing Unit (GPU) using CUDA can introduce a speed-up of up to 30x. Moreover, one method is used to sample from the posterior distribution of petroleum reservoir parameters in a North Sea field, given seismic reflection data on a large 3D grid.

/pdf/iterative-numerical-methods-for-sampling-from-high-32fezd58ox.pdf

Yvo Pokern

Papers

Nonparametric estimation of diffusions: a differential equations approach

Parameter estimation for partially observed hypoelliptic diffusions

Parameter Estimation for Partially Observed Hypoelliptic Diffusions

Posterior consistency via precision operators for Bayesian nonparametric drift estimation in SDEs

Iterative numerical methods for sampling from high dimensional Gaussian distributions