In this paper, we prove a comparison result between semicontinuous viscosity sub- and supersolutions growing at most quadratically of second-order degenerate parabolic Hamilton--Jacobi--Bellman and Isaacs equations. As an application, we characterize the value function of a finite horizon stochastic control problem with unbounded controls as the unique viscosity solution of the corresponding dynamic programming equation.

https://hal.archives-ouvertes.fr/hal-00455993/document

Uniqueness Results for Second-Order Bellman--Isaacs Equations under Quadratic Growth Assumptions and Applications

Introduction to the Mathematical Theory of Control Processes, Volume I: Linear Equations and Quadratic Criteria, Volume II: Nonlinear Processes

We consider the optimal control problem of feeding in minimal time a tank where several species compete for a single resource, with the objective being to reach a given level of the resource. We allow controls to be bounded measurable functions of time plus possible impulses. For the one-species case, we show that the immediate one-impulse strategy (filling the whole reactor with one single impulse at the initial time) is optimal when the growth function is monotonic. For nonmonotonic growth functions with one maximum, we show that a particular singular arc strategy (precisely defined in section 3) is optimal. These results extend and improve former ones obtained for the class of measurable controls only. For the two-species case with monotonic growth functions, we give conditions under which the immediate one-impulse strategy is optimal. We also give optimality conditions for the singular arc strategy (at a level that depends on the initial condition) to be optimal. The possibility for the immediate one-impulse strategy to be nonoptimal while both growth functions are monotonic is a surprising result and is illustrated with the help of numerical simulations.

/pdf/minimal-time-sequential-batch-reactors-with-bounded-and-3avkfdvrf1.pdf

Minimal Time Sequential Batch Reactors with Bounded and Impulse Controls for One or More Species

We consider the classical pursuit-evasion problem and an approximation scheme based on Dynamic Programming We prove the convergence of the scheme to the value function of the game by using some recent results and methods of the theory of viscosity solutions to the Isaacs equations The more restrictive assumption is the continuity of the value function, but we can eliminate it when dealing with control problems with a single player We test the algorithm on two simple examples with explicit solution

Fully Discrete Schemes for the Value Function of Pursuit-Evasion Games

Unbounded stochastic control problems may lead to Hamilton-Jacobi-Bellman equations whose Hamiltonians are not always defined, especially when the diffusion term is unbounded with respect to the control. We obtain existence and uniqueness of viscosity solutions growing at most like o(1+|x|p) at infinity for such HJB equations and more generally for degenerate parabolic equations with a superlinear convex gradient nonlinearity. If the corresponding control problem has a bounded diffusion with respect to the control, then our results apply to a larger class of solutions, namely those growing like O(1+|x|p) at infinity. This latter case encompasses some equations related to backward stochastic differential equations.

Convex Hamilton-Jacobi Equations Under Superlinear Growth Conditions on Data

We investigate existence, uniqueness, and regular- ity properties for a class of H-J-B equations arising in non-linear control problems with unbounded controls. These equations involve Hamiltonians which are superlinear in the adjoint vari- able, and they have been already studied in the case when the growth in the adjoint variable is, in a sense, uniform with re- spect to the state variable. For instance, this is the case of the linear-quadratic problem. On the contrary, our results concern Hamiltonians that are superlinear in the adjoint variable, pos- sibly not uniformly with respect to the state variable. Actually, this is the general situation one has to deal with when consid- ering optimal control problems with a nonlinear dynamics (e.g. by slightly perturbing the linear quadratic problem). We also investigate situations where the fast growth of the Hamilton- ian in the adjoint variable degenerates into a very discontinuity. Such Hamiltonians arise quite naturally in those optimal con- trol problems where, roughly speaking, the dynamics and the cost display the same growth in the control variable.

/pdf/hamilton-jacobi-bellman-equations-with-fast-gradient-3qg8ck62ez.pdf

Hamilton-Jacobi-Bellman equations with fast gradient-dependence

The research on a class of asymptotic exit-time problems with a vanishing Lagrangian, begun in [M. Motta and C. Sartori, Nonlinear Differ. Equ. Appl. Springer (2014).] for the compact control case, is extended here to the case of unbounded controls and data, including both coercive and non-coercive problems. We give sufficient conditions to have a well-posed notion of generalized control problem and obtain regularity, characterization and approximation results for the value function of the problem.

/pdf/on-asymptotic-exit-time-control-problems-lacking-coercivity-4ajsmgjv37.pdf

On asymptotic exit-time control problems lacking coercivity

Necessary and sufficient conditions for the regularity of the minimum time function and minimum energy function for a control system with controls in $L^p([0,+\infty[,{\Bbb R^m})$ and $p\ge 1$ are given in terms of topological properties of the reachable sets. In particular, standard local controllability assumptions are sufficient to yield the continuity of both value functions for linear systems and $p\ge1$ and the Holder continuity of the minimum time function for nonlinear systems and p>1.

Minimum Time with Bounded Energy, Minimum Energy with Bounded Time

We investigate, via the dynamic programming approach, a finite fuel nonlinear singular stochastic control problem of Bolza type. We prove that the associated value function is continuous and that its continuous extension to the closure of the domain coincides with the value function of a nonsingular control problem, for which we prove the existence of an optimal control. Moreover, such a continuous extension is characterized as the unique viscosity solution of a quasi-variational inequality with suitable boundary conditions of mixed type.

Finite Fuel Problem in Nonlinear Singular Stochastic Control

We establish uniqueness of viscosity solutions
for some boundary value problems arising from stochastic optimal
 control problems with unbounded, possibly singular, controls.
They involve
a nonlinear degenerate second order
Bellman-Isaacs equation and mixed boundary conditions
(Dirichlet, generalized Dirichlet and state constrained conditions).

Caterina Sartori

Papers

Hamilton-Jacobi-Bellman equations with fast gradient-dependence

On asymptotic exit-time control problems lacking coercivity

Minimum Time with Bounded Energy, Minimum Energy with Bounded Time

Finite Fuel Problem in Nonlinear Singular Stochastic Control

Uniqueness results for boundary value problems arising from finite fuel and other singular and unbounded stochastic control problems