Preface.- 1. Conditional Expectation and Linear Parabolic PDEs.- 2. Stochastic Control and Dynamic Programming.- 3. Optimal Stopping and Dynamic Programming.- 4. Solving Control Problems by Verification.- 5. Introduction to Viscosity Solutions.- 6. Dynamic Programming Equation in the Viscosity Sense.- 7. Stochastic Target Problems.- 8. Second Order Stochastic Target Problems.- 9. Backward SDEs and Stochastic Control.- 10. Quadratic Backward SDEs.- 11. Probabilistic Numerical Methods for Nonlinear PDEs.- 12. Introduction to Finite Differences Methods.- References.

Optimal Stochastic Control, Stochastic Target Problems, and Backward SDE

Using systematically a tricky idea of N.V. Krylov, we obtain general results on the rate of convergence of a certain class of monotone approximation schemes for stationary Hamilton-Jacobi-Bellman equations with variable coefficients. This result applies in particular to control schemes based on the dynamic programming principle and to finite difference schemes despite, here, we are not able to treat the most general case. General results have been obtained earlier by Krylov for finite difference schemes in the stationary case with constant coefficients and in the time-dependent case with variable coefficients by using control theory and probabilistic methods. In this paper we are able to handle variable coefficients by a purely analytical method. In our opinion this way is far simpler and, for the cases we can treat, it yields a better rate of convergence than Krylov obtains in the variable coefficients case.

/pdf/on-the-convergence-rate-of-approximation-schemes-for-1bt0nky2ny.pdf

On the convergence rate of approximation schemes for Hamilton-Jacobi-Bellman equations

We consider the probabilistic numerical scheme for fully nonlinear PDEs suggested in \cite{cstv}, and show that it can be introduced naturally as a combination of Monte Carlo and finite differences scheme without appealing to the theory of backward stochastic differential equations. Our first main result provides the convergence of the discrete-time approximation and derives a bound on the discretization error in terms of the time step. An explicit implementable scheme requires to approximate the conditional expectation operators involved in the discretization. This induces a further Monte Carlo error. Our second main result is to prove the convergence of the latter approximation scheme, and to derive an upper bound on the approximation error. Numerical experiments are performed for the approximation of the solution of the mean curvature flow equation in dimensions two and three, and for two and five-dimensional (plus time) fully-nonlinear Hamilton-Jacobi-Bellman equations a! rising in the theory of portfolio optimization in financial mathematics.

/pdf/a-probabilistic-numerical-method-for-fully-nonlinear-24kziqi3ad.pdf

A probabilistic numerical method for fully nonlinear parabolic PDEs

We study the numerical approximation of boundary optimal control problems governed by semilinear elliptic partial differential equations with pointwise constraints on the control. The control is the trace of the state on the boundary of the domain, which is assumed to be a convex, polygonal, open set in ${\mathbb R}^2$. Piecewise linear finite elements are used to approximate the control as well as the state. We prove that the error estimates are of order $O(h^{1 - 1/p})$ for some $p > 2$, which is consistent with the $W^{1 - 1/p,p}(\Gamma)$-regularity of the optimal control.

/pdf/error-estimates-for-the-numerical-approximation-of-dirichlet-2c7kkus1ik.pdf

Error Estimates for the Numerical Approximation of Dirichlet Boundary Control for Semilinear Elliptic Equations

. We obtain nonsymmetric upper and lower bounds on the rate of convergence of general monotone approximation/numerical schemes for parabolic Hamilton-Jacobi-Bellman equations by introducing a new notion of consistency. Our results are robust and general - they improve and extend earlier results by Krylov, Barles, and Jakobsen. We apply our general results to various schemes including Crank-Nicholson type finite difference schemes, splitting methods, and the classical approximation by piecewise constant controls. In the first two cases our results are new, and in the last two cases the results are obtained by a new method which we develop here.

/pdf/error-bounds-for-monotone-approximation-schemes-for-8sdhwlwv8m.pdf

Error bounds for monotone approximation schemes for parabolic Hamilton-Jacobi-Bellman equations

We analyze a class of numerical schemes for solving the HJB equation for stochastic control problems, which enters the framework of Markov chain approximations and generalizes the usual finite difference method. The latter is known to be monotonic, and hence valid, only if the scaled covariance matrix is dominant diagonal. We generalize this result by, given the set of neighboring points allowed to enter the scheme, showing how to compute effectively the class of covariance matrices that is consistent with this set of points. We perform this computation for several cases in dimensions 2, 3, and 4.

/pdf/consistency-of-generalized-finite-difference-schemes-for-the-sxuu9fcsfy.pdf

Consistency of Generalized Finite Difference Schemes for the Stochastic HJB Equation

This paper analyses the implementation of the generalized finite differences method for the HJB equation of stochastic control, introduced by two of the authors in (Bonnans and Zidani, SIAM J. Numer. Anal. 41 (2003) 1008-1021). The computation of coefficients needs to solve at each point of the grid (and for each control) a linear programming problem. We show here that, for two dimensional problems, this linear programming problem can be solved in O(pmax )o perations, wherepmax is the size of the stencil. The method is based on a walk on the Stern-Brocot tree, and on the related filling of the set of positive semidefinite matrices of size two.

/pdf/a-fast-algorithm-for-the-two-dimensional-hjb-equation-of-4gv8ol9pd7.pdf

A fast algorithm for the two dimensional HJB equation of stochastic control

This paper discusses a class of state constrained optimal control problems, for which it is possible to formulate second-order necessary or sufficient conditions for local optimality or quadratic growth that do not involve all curvature terms for the constraints. This kind of result is classical in the case of polyhedric control constraints. Our theory of optimization problems with partially polyhedric constraints allows to extend these results to the case when the control constraints are polyhedric, in the presence of state constraints satisfying some specific hypotheses. The analysis is based on the assumption that some strict semilinearized qualification condition is satisfied. We apply the theory to some optimal control problems of elliptic equations with state and control constraints.

/pdf/optimal-control-problems-with-partially-polyhedric-uu1igosw68.pdf

Optimal Control Problems with Partially Polyhedric Constraints

In this paper we investigate optimal control problems governed by variational inequalities. We present a method for deriving optimality conditions in the form of Pontryagin's principle. The main tools used are the Ekeland's variational principle combined with penalization and spike variation techniques.

/pdf/pontryagin-maximum-principle-for-optimal-control-of-14z149fdku.pdf

Pontryagin Maximum Principle for Optimal Control of Variational Inequalities

We obtain error bounds for monotone approximation schemes of a particular Isaacs equation. This is an extension of the theory for estimating errors for the Hamilton-Jacobi-Bellman equation. To obtain the upper error bound, we consider the 'Krylov regularization' of the Isaacs equation to build an approximate sub-solution of the scheme. To get the lower error bound, we extend the method of Barles & Jakobsen (2005, SIAM J. Numer. Anal.) which consists in introducing a switching system whose solutions are local super-solutions of the Isaacs equation.

/pdf/error-estimates-for-stochastic-differential-games-the-4u8kbg9vc5.pdf

Housnaa Zidani

Papers

Consistency of Generalized Finite Difference Schemes for the Stochastic HJB Equation

A fast algorithm for the two dimensional HJB equation of stochastic control

Optimal Control Problems with Partially Polyhedric Constraints

Pontryagin Maximum Principle for Optimal Control of Variational Inequalities

Error estimates for stochastic differential games: the adverse stopping case