# A curse-of-dimensionality-free numerical method for a class of HJB PDE's

Abstract: Max-plus methods have been explored for solution of first-order, nonlinear Hamilton-Jacobi-Bellman partial differential equations (HJB PDEs) and corresponding nonlinear control problems. These methods exploit the max-plus linearity of the associated semigroups. Although these methods provide advantages, they still suffer from the curse-of-dimensionality. Here we consider HJB PDEs where the Hamiltonian takes the form of a (pointwise) maximum of linear/quadratic forms. We obtain a numerical method not subject to the curse-of-dimensionality. The method is based on construction of the dual-space semigroup corresponding to the HJB PDE. This dual-space semigroup is constructed from the dual-space semigroups corresponding to the constituent Hamiltonians.

## Summary (1 min read)

### 1. INTRODUCTION

- One approach to nonlinear control is through Dynamic Programming (DP).
- Various approaches have been taken to solution of the HJB PDE.
- If the state dimension is n, then one has 100n grid points.
- They employ a max-plus basis function expansion of the solution, and the numerical methods obtain the coefficients in the basis expansion.

### 2. REVIEW OF THEORY

- Note that due to space limitations, the proofs of the results cannot be included here.
- As indicated above, the authors suppose the individual Hm are linear/quadratic Hamiltonians.
- These assumptions guarantee the existence of the V m as locally bounded functions which are zero at the origin (cf. (McEneaney, 1998)).
- Pm is the smallest symmetric, positive definite solution of (6) The method the authors will use to obtain these value functions/HJB PDE solutions will be through the associated semigroups.

### 3. MAX-PLUS DUAL OPERATORS

- LetSβ = Sβ(IRn) be the set of functions mapping IRn into IR which are uniformly semiconvex with constant β.
- The following semiconvex duality result (Fleming and McEneaney, 2000), (McEneaney, 2003) requires only a small modification of convex duality and Legendre/Fenchel transform results (c.f. (Rockafellar and Wets, 1997)).
- Semiconcavity is the obvious analogue of semiconvexity.
- In particular, one has the following Theorem 3.8.

### 4. DISCRETE TIME APPROXIMATION

- The method developed here will not involve any discretization over space nor any basis functions.
- Of course this is obvious since otherwise one could not avoid the curse-of-dimensionality.
- The discretization will be over time, where approximate µ processes will be constant over the length of each time-step.
- The corresponding max-plus linear operator is B̂τ = ⊕ m∈M B̂mτ .
- The following result on propagation of the semiconvex dual will also come in handy.

### 5. ALGORITHM AND EXAMPLES

- Due to space limitations, the authors cannot give the steps in the actual algorithm that is generated by the above theory.
- Further, the computational cost growth in space dimension n is limited to cubic growth.
- The quoted computational times were obtained with a standard 2001 PC.
- The backsubstitution error has been scaled by dividing by |x|2 + 10−5.

Did you find this useful? Give us your feedback

...read more

##### Citations

46 citations

### Cites methods from "A curse-of-dimensionality-free nume..."

...In the first case, several attempts have been made to address the difficulty inherent in solving such nonlinear PDEs, aswell as the curse of dimensionality, with various different methods and approaches (Beard, Saridis, & Wen, 1997; Lasserre, Henrion, Prieur, & Trelat, 2008; McEneaney, 2007) for deterministic control problems, while a stochastic setting is considered in Gorodetsky, Karaman, and Marzouk (2015), Horowitz and Burdick (2014) and Horowitz, Damle, and Burdick (2014)....

[...]

42 citations

32 citations

### Cites methods from "A curse-of-dimensionality-free nume..."

...In [29], [30], [31], [32], a new class of methods for first-order HJB PDEs was introduced, and these methods are not subject to the curse-of-dimensionality....

[...]

9 citations

##### References

2,561 citations

### "A curse-of-dimensionality-free nume..." refers methods in this paper

...The most common methods by far all fall into the class of finite element methods (cf. (Bardi and Capuzzo-Dolcetta, 1997), (Dupuis and Boué, 1999), among many others)....

[...]

...(Bardi and Capuzzo-Dolcetta, 1997), (Dupuis and Boué, 1999), among many others)....

[...]

^{1}

425 citations

398 citations

259 citations

209 citations