Convergence of Probability Measures. By P. Billingsley. Chichester, Sussex, Wiley, 1968. xii, 253 p. 9 1/4“. 117s.

Convergence of Probability Measures

This book is intended as an introduction to optimal stochastic control for continuous time Markov processes and to the theory of viscosity solutions. The authors approach stochastic control problems by the method of dynamic programming. The text provides an introduction to dynamic programming for deterministic optimal control problems, as well as to the corresponding theory of viscosity solutions. A new Chapter X gives an introduction to the role of stochastic optimal control in portfolio optimization and in pricing derivatives in incomplete markets. Chapter VI of the First Edition has been completely rewritten, to emphasize the relationships between logarithmic transformations and risk sensitivity. A new Chapter XI gives a concise introduction to two-controller, zero-sum differential games. Also covered are controlled Markov diffusions and viscosity solutions of Hamilton-Jacobi-Bellman equations. The authors have tried, through illustrative examples and selective material, to connect stochastic control theory with other mathematical areas (e.g. large deviations theory) and with applications to engineering, physics, management, and finance. In this Second Edition, new material on applications to mathematical finance has been added. Concise introductions to risk-sensitive control theory, nonlinear H-infinity control and differential games are also included.

/pdf/controlled-markov-processes-and-viscosity-solutions-4n8p6ywj5v.pdf

Controlled Markov processes and viscosity solutions

Non-Gaussian processes of Ornstein–Uhlenbeck (OU) type offer the possibility of capturing important distributional deviations from Gaussianity and for flexible modelling of dependence structures. This paper develops this potential, drawing on and extending powerful results from probability theory for applications in statistical analysis. Their power is illustrated by a sustained application of OU processes within the context of finance and econometrics. We construct continuous time stochastic volatility models for financial assets where the volatility processes are superpositions of positive OU processes, and we study these models in relation to financial data and theory.

Non-Gaussian Ornstein–Uhlenbeck-based models and some of their uses in financial economics

This celebrated book has been prepared with readers' needs in mind, remaining a systematic treatment of the subject whilst retaining its vitality. The second volume follows on from the first, concentrating on stochastic integrals, stochastic differential equations, excursion theory and the general theory of processes. Much effort has gone into making these subjects as accessible as possible by providing many concrete examples that illustrate techniques of calculation, and by treating all topics from the ground up, starting from simple cases. Many of the examples and proofs are new; some important calculational techniques appeared for the first time in this book. Together with its companion volume, this book helps equip graduate students for research into a subject of great intrinsic interest and wide application in physics, biology, engineering, finance and computer science.

Diffusions, Markov processes, and martingales

This report presents a unified approach for the study of constrained Markov decision processes with a countable state space and unbounded costs. We consider a single controller having several objectives; it is desirable to design a controller that minimize one of cost objective, subject to inequality constraints on other cost objectives. The objectives that we study are both the expected average cost, as well as the expected total cost (of which the discounted cost is a special case). We provide two frameworks: the case were costs are bounded below, as well as the contracting framework. We characterize the set of achievable expected occupation measures as well as performance vectors. This allows us to reduce the original control dynamic problem into an infinite Linear Programming. We present a Lagrangian approach that enables us to obtain sensitivity analysis. In particular, we obtain asymptotical results for the constrained control problem: convergence of both the value and the policies in the time horizon and in the discount factor. Finally, we present and several state truncation algorithms that enable to approximate the solution of the original control problem via finite linear programs.

/pdf/constrained-markov-decision-processes-3ndxrfjbtt.pdf

Constrained Markov Decision Processes

1. Essentials of Probability Theory and Mathematical Statistics.- 2. Martingales and Related Processes: Discrete Time.- 3. Martingales and Related Processes: Continuous Time.- 4. The Wiener Process, the Stochastic Integral over the Wiener Process, and Stochastic Differential Equations.- 5. Square Integrable Martingales and Structure of the Functionals on a Wiener Process.- 6. Nonnegative Supermartingales and Martingales, and the Girsanov Theorem.- 7. Absolute Continuity of Measures corresponding to the Ito Processes and Processes of the Diffusion Type.- 8. General Equations of Optimal Nonlinear Filtering, Interpolation and Extrapolation of Partially Observable Random Processes.- 9. Optimal Filtering, Interpolation and Extrapolation of Markov Processes with a Countable Number of States.- 10. Optimal Linear Nonstationary Filtering.

Statistics of random processes

I.- 1. Basic Concepts and the Review of Results of "The General Theory of Stochastic Processes".- 1. Stochastic basis. Random times, sets and processes.- 2. Optional and predictable ?-algebras of random sets.- 3. Predictable and totally inaccessible random times. Classification of Markov times. Section theorems.- 4. Martingales and local martingales.- 5. Square integrable martingales.- 6. Increasing processes. Compensators (dual predictable projections). The Doob-Meyer decomposition.- 7. The structure of local martingales.- 8. Quadratic characteristic and quadratic variation.- 9. Inequalities for local martingales.- 2. Semimartingales. I. Stochastic Integral.- 1. Semimartingales and quasimartingales.- 2. Stochastic integral with respect to a local martingale and a semimartingale. Construction and properties.- 3. Ito's formula. I.- 4. Doleans equation. Stochastic exponential.- 5. Multiplicative decomposition of positive semimartingales.- 6. Convergence sets and the strong law of large numbers for special martingales.- 3. Random Measures and their Compensators.- 1. Optional and predictable random measures.- 2. Compensators of random measures. Conditional mathematical expectation with respect to the ?-algebra P?.- 3. Integer-valued random measures.- 4. Multivariate point processes.- 5. Stochastic integral with respect to a martingale measure ?-?.- 6. Ito's formula. II.- 4. Semimartingales. II Canonical Representation.- 1. Canonical representation. Triplet of predictable characteristics of a semimartingale.- 2. Stochastic exponential constructed by the triplet of a semimartingale.- 3. Martingale characterization of semimartingales by means of stochastic exponentials.- 4. Characterization of semimartingales with conditionally independent increments.- 5. Semimartingales and change of probability measures. Transformation of triplets.- 6. Semimartingales and reduction of a flow of ?-algebras.- 7. Semimartingales and random change of time.- 8. Semimartingales and integral representation of martingales.- 9. Gaussian martingales and semimartingales.- 10. Filtration of special semimartingales.- 11. Semimartingales and helices. Ergodic theorems.- 12. Semimartingales - stationary processes.- 13. Exponential inequalities for large deviation probabilities.- II.- 5. Weak Convergence of Finite-Dimensional Distributions of Semimartingales to Distributions of Processes with Conditionally Independent Increments.- 1. Method of stochastic exponentials. I. Convergence of conditional characteristic functions.- 2. Method of stochastic exponentials. II. Weak convergence of finite dimensional distributions.- 3. Weak convergence of finite dimensional distributions of point processes and semimartingales to distributions of point processes.- 4. Weak convergence of finite dimensional distributions of semimartingales to distributions of a left quasi-continuous semimartingale with conditionally independent increments.- 5. The central limit theorem. I. "Classical" version.- 6. The central limit theorem. II. "Nonclassical" version.- 7. Evaluation of a convergence rate for marginal distributions in the central limit theorem.- 8. A martingale method of proving the central limit theorem for strictly stationary sequences. Relation to mixing conditions.- 6. The Space D. Relative Compactness of Probability Distributions of Semimartingales.- 1. The space D. Skorohod's topology.- 2. Continuous functions on R+ x D.- 3. Conditions on adapted processes sufficient for relative compactness of families of their distributions.- 4. Relative compactness of probability distributions of semimartingales.- 5. Conditions necessary for the weak convergence of probability distributions of semimartingales.- 7. Weak Convergence of Distributions of Semimartingales to Distributions of Processes with Conditionally Independent Increments.- 1. The functional central limit theorem (invariance principle).- 2. Weak convergence of distributions of semimartingales to distributions of point processes.- 3. Weak convergence of distributions of semimartingales to the distribution of a left quasi-continuous semimartingale, with conditionally independent increments.- 8. Weak Convergence of Distributions of Semimartingales to the Distribution of a Semimartingale.- 1. Convergence of stochastic exponentials and weak convergence of distributions of semimartingales.- 2. Weak convergence to the distribution of a left quasi-continuous semimartingale.- 3. Diffusion approximation.- 4. Weak convergence to a distribution of a point process with a continuous compensator.- 5. Weak convergence of in variant measures.- III.- 9. Invariance Principle and Diffusion Approximation for Models Generated by Stationary Processes.- 1. Generalization of Donsker's invariance principle.- 2. Invariance principle for strictly stationary processes.- 3. Invariance principle for a Markov process.- 4. Diffusion approximation for systems with a "broad bandwidth noise" (scalar case).- 5. Diffusion approximation with a "broad bandwidth noise" (vector case).- 6. Ergodic theorem and invariant principle in case of nonhomogeneous time averaging.- 7. Stochastic version of Bogoljubov's averaging principle.- 10. Diffusion Approximation for Semimartingales with a Normal Reflexion in a Convex Region.- 1. Skorohod's problem on normal reflection.- 2. Semimartingale with normal reflection.- 3. Diffusion approximation with normal reflection.- 4. Diffusion approximation with reflection for queueing models with autonomious service.- Historic-Bibliographical notes.

Theory of martingales

A Functional Central Limit Theorem for Semimartingales

Statistics and control of stochastic processes

We study the distance in variation between probability measures defined on a measurable space (Ω, ℱ) with right-continuous filtration (ℱt)t
≦0. To every pair of probability measures P and 
$$\tilde P$$

 an increasing predictable process 
$$h = h(P,{\text{ }}\tilde P)$$

 (called the Hellinger process) is associated. For the variation distance 
$$\left\| {P_T - \tilde P_T } \right\|$$

 between the restrictions of P and 
$$\tilde P$$

 to ℱ
 T
 (T is a stopping time), lower and upper bounds are obtained in terms of h. For example, in the case when 
$$P_0 = \tilde P_0 $$

 , 
$$2(1 - (E{\text{ }}\exp {\text{ }}( - h_T ))^{1/2} ) \leqq \left\| {P_T - \tilde P_T } \right\| \leqq 4(Eh_T )^{1/2} $$

 In the cases where P and 
$$\tilde P$$

 are distributions of multivariate point processes, diffusion-type processes or semimartingales h are expressed explicitly in terms of given predictable characteristics.

R. Sh. Liptser

Papers

Statistics of random processes

Theory of martingales

A Functional Central Limit Theorem for Semimartingales

Statistics and control of stochastic processes

On the variation distance for probability measures defined on a filtered space