Filtering and smoothing methods are used to produce an accurate estimate of the state of a time-varying system based on multiple observational inputs (data). Interest in these methods has exploded in recent years, with numerous applications emerging in fields such as navigation, aerospace engineering, telecommunications, and medicine. This compact, informal introduction for graduate students and advanced undergraduates presents the current state-of-the-art filtering and smoothing methods in a unified Bayesian framework. Readers learn what non-linear Kalman filters and particle filters are, how they are related, and their relative advantages and disadvantages. They also discover how state-of-the-art Bayesian parameter estimation methods can be combined with state-of-the-art filtering and smoothing algorithms. The book’s practical and algorithmic approach assumes only modest mathematical prerequisites. Examples include MATLAB computations, and the numerous end-of-chapter exercises include computational assignments. MATLAB/GNU Octave source code is available for download at www.cambridge.org/sarkka, promoting hands-on work with the methods.

贝叶斯滤波与平滑 (Bayesian filtering and smoothing)

The paper proposes Metropolis adjusted Langevin and Hamiltonian Monte Carlo sampling methods defined on the Riemann manifold to resolve the shortcomings of existing Monte Carlo algorithms when sampling from target densities that may be high dimensional and exhibit strong correlations. The methods provide fully automated adaptation mechanisms that circumvent the costly pilot runs that are required to tune proposal densities for Metropolis-Hastings or indeed Hamiltonian Monte Carlo and Metropolis adjusted Langevin algorithms. This allows for highly efficient sampling even in very high dimensions where different scalings may be required for the transient and stationary phases of the Markov chain. The methodology proposed exploits the Riemann geometry of the parameter space of statistical models and thus automatically adapts to the local structure when simulating paths across this manifold, providing highly efficient convergence and exploration of the target density. The performance of these Riemann manifold Monte Carlo methods is rigorously assessed by performing inference on logistic regression models, log-Gaussian Cox point processes, stochastic volatility models and Bayesian estimation of dynamic systems described by non-linear differential equations. Substantial improvements in the time-normalized effective sample size are reported when compared with alternative sampling approaches. MATLAB code that is available from http://www.ucl.ac.uk/statistics/research/rmhmc allows replication of all the results reported.

Riemann manifold Langevin and Hamiltonian Monte Carlo methods

Stochastic optimization problems arise in decision-making problems under uncertainty, and find various applications in economics and finance. On the other hand, problems in finance have recently led to new developments in the theory of stochastic control. This volume provides a systematic treatment of stochastic optimization problems applied to finance by presenting the different existing methods: dynamic programming, viscosity solutions, backward stochastic differential equations, and martingale duality methods. The theory is discussed in the context of recent developments in this field, with complete and detailed proofs, and is illustrated by means of concrete examples from the world of finance: portfolio allocation, option hedging, real options, optimal investment, etc. This book is directed towards graduate students and researchers in mathematical finance, and will also benefit applied mathematicians interested in financial applications and practitioners wishing to know more about the use of stochastic optimization methods in finance.

Continuous-time stochastic control and optimization with financial applications / Huyen Pham

Thank you for downloading introduction to stochastic control theory. Maybe you have knowledge that, people have look hundreds times for their chosen novels like this introduction to stochastic control theory, but end up in infectious downloads. Rather than reading a good book with a cup of tea in the afternoon, instead they juggled with some infectious bugs inside their laptop. introduction to stochastic control theory is available in our digital library an online access to it is set as public so you can get it instantly. Our book servers saves in multiple countries, allowing you to get the most less latency time to download any of our books like this one. Merely said, the introduction to stochastic control theory is universally compatible with any devices to read.

Introduction To Stochastic Control Theory

Lévy Processes” (eight papers), “III. Empirical Processes” (four papers), and “IV. Stochastic Differential Equations” (four papers). Here are some comments about the individual papers: In I.2 (paper 2 of Part I) the covariance representation method, which relies on Clark’s formula on path spaces, is used to obtain concentration inequalities for functionals of Brownian motion on a manifold, allowing one to obtain tail estimates for this Brownian motion. In I.4 a transportation inequality for the canonical Gaussian measure in R is obtained and applied to Khintchine–Kahane inequalities for norms of random series with nonsymmetric Bernoulli coefficients. In II.1 exponential inequalities for U -statistics of order two are presented; these rely upon the Talagrand inequality for empirical processes but also use martingale type inequalities. In II.2 the unconditional convergence of a Gaussian [and, more generally, independent, identically distributed (iid)] series in a Banach space is studied. Applications to Karhunen–Love representations of Gaussian processes are given. In II.3 estimates of tail properties and moments of multidimensional chaos generated by positive random variables with log concave tails are given. In II.4 a quantitative technique for studying the asymptotic distribution of sequences of Markov processes in infinite dimensions is proposed. The proof relies on the properties of an associated sequence of exponential martingales. In II.5 it is shown that a moving average process driven by a symmetric Lévy process and with a kernel with finite total 2-variation admits an almost surely bounded version. In II.6 a Markovian approach to the entropic convergence in the central limit theorem is presented. The emphasis is on the speed of convergence, as well as relaxing a spectral gap assumption. In II.7 a new version of the Khintchine–Kahane inequality for general Bernoulli random variables is presented with the help of hypercontractive methods. In III.1 necessary and sufficient conditions for the moderate deviations of empirical processes and sums of iid random vectors on a separable Banach space are given. In III.2 exponential concentration inequalities for subadditive functions of independent random variables are obtained. As a consequence, Talagrand’s inequality for empirical processes is refined thanks to further developments of the entropy method introduced by M. Ledoux. In III.3 ratio limit theorems for empirical processes are obtained with the help of concentration inequalities. In III.4 asymptotic distributions of trimmed Wasserstein distances between the true and the empirical distribution function are obtained via weighted approximation results for uniform empirical processes. In IV.1 sharp rates of convergence for splitting-up approximations of stochastic partial differential equations are obtained. The error is estimated in terms of Sobolev’s norm. In IV.4 the existence and uniqueness of a strong solution for a stochastic differential equation driven by a fractional Brownian motion with Hurst index H < 1/2 and with a possibly time-dependent drift which satisfies a suitable integrability condition is obtained. In short, the book presents a wide variety of inequalities which are established for either stochastic processes or sequences of random variables. In 2006 this is still an active domain of research in transportation problems.

Statistical Inference for Ergodic Diffusion Processes

We propose a computational method (with acronym ALDI) for sampling from a given target distribution based on first-order (overdamped) Langevin dynamics which satisfies the property of affine invari...

https://epubs.siam.org/doi/pdf/10.1137/19M1304891

Affine Invariant Interacting Langevin Dynamics for Bayesian Inference

Optimal control of diffusion processes is intimately connected to the problem of solving certain Hamilton–Jacobi–Bellman equations. Building on recent machine learning inspired approaches towards high-dimensional PDEs, we investigate the potential of iterative diffusion optimisation techniques, in particular considering applications in importance sampling and rare event simulation, and focusing on problems without diffusion control, with linearly controlled drift and running costs that depend quadratically on the control. More generally, our methods apply to nonlinear parabolic PDEs with a certain shift invariance. The choice of an appropriate loss function being a central element in the algorithmic design, we develop a principled framework based on divergences between path measures, encompassing various existing methods. Motivated by connections to forward-backward SDEs, we propose and study the novel log-variance divergence, showing favourable properties of corresponding Monte Carlo estimators. The promise of the developed approach is exemplified by a range of high-dimensional and metastable numerical examples.

/pdf/solving-high-dimensional-hamilton-jacobi-bellman-pdes-using-3ibtbylm4w.pdf

Solving high-dimensional Hamilton–Jacobi–Bellman PDEs using neural networks: perspectives from the theory of controlled diffusions and measures on path space

In this paper we introduce and analyse Langevin samplers that consist of perturbations of the standard underdamped Langevin dynamics. The perturbed dynamics is such that its invariant measure is the same as that of the unperturbed dynamics. We show that appropriate choices of the perturbations can lead to samplers that have improved properties, at least in terms of reducing the asymptotic variance. We present a detailed analysis of the new Langevin sampler for Gaussian target distributions. Our theoretical results are supported by numerical experiments with non-Gaussian target measures.

/pdf/using-perturbed-underdamped-langevin-dynamics-to-efficiently-2yabmkbjwe.pdf

Using Perturbed Underdamped Langevin Dynamics to Efficiently Sample from Probability Distributions

In this paper we derive spectral gap estimates for several Piecewise Deterministic Markov Processes, namely the Randomized Hamiltonian Monte Carlo, the Zig-Zag process and the Bouncy Particle Sampler. The hypocoercivity technique we use, presented in (Dolbeault et al., 2015), produces estimates with explicit dependence on the parameters of the dynamics. Moreover the general framework we consider allows to compare quantitatively the bounds found for the different methods.

Hypocoercivity of piecewise deterministic Markov process-Monte Carlo

In this work, we establish $\mathrm{L}^2$-exponential convergence for a broad class of Piecewise Deterministic Markov Processes recently proposed in the context of Markov Process Monte Carlo methods and covering in particular the Randomized Hamiltonian Monte Carlo, the Zig-Zag process and the Bouncy Particle Sampler. The kernel of the symmetric part of the generator of such processes is non-trivial, and we follow the ideas recently introduced by (Dolbeault et al., 2009, 2015) to develop a rigorous framework for hypocoercivity in a fairly general and unifying set-up, while deriving tractable estimates of the constants involved in terms of the parameters of the dynamics. As a by-product we characterize the scaling properties of these algorithms with respect to the dimension of classes of problems, therefore providing some theoretical evidence to support their practical relevance.

Nikolas Nüsken

Papers

Affine Invariant Interacting Langevin Dynamics for Bayesian Inference

Solving high-dimensional Hamilton–Jacobi–Bellman PDEs using neural networks: perspectives from the theory of controlled diffusions and measures on path space

Using Perturbed Underdamped Langevin Dynamics to Efficiently Sample from Probability Distributions

Hypocoercivity of piecewise deterministic Markov process-Monte Carlo

Hypocoercivity of Piecewise Deterministic Markov Process-Monte Carlo