scispace - formally typeset
Search or ask a question
Journal ArticleDOI

A curse-of-dimensionality-free numerical method for a class of HJB PDE's

01 Jan 2005-IFAC Proceedings Volumes (Elsevier)-Vol. 38, Iss: 1, pp 532-537
TL;DR: In this article, the authors consider HJB PDEs where the Hamiltonian takes the form of a (pointwise) maximum of linear/quadratic forms and obtain a numerical method not subject to the curse of dimensionality.
About: This article is published in IFAC Proceedings Volumes.The article was published on 2005-01-01 and is currently open access. It has received 4 citations till now. The article focuses on the topics: Semigroup & Hamilton–Jacobi–Bellman equation.

Summary (1 min read)

1. INTRODUCTION

  • One approach to nonlinear control is through Dynamic Programming (DP).
  • Various approaches have been taken to solution of the HJB PDE.
  • If the state dimension is n, then one has 100n grid points.
  • They employ a max-plus basis function expansion of the solution, and the numerical methods obtain the coefficients in the basis expansion.

2. REVIEW OF THEORY

  • Note that due to space limitations, the proofs of the results cannot be included here.
  • As indicated above, the authors suppose the individual Hm are linear/quadratic Hamiltonians.
  • These assumptions guarantee the existence of the V m as locally bounded functions which are zero at the origin (cf. (McEneaney, 1998)).
  • Pm is the smallest symmetric, positive definite solution of (6) The method the authors will use to obtain these value functions/HJB PDE solutions will be through the associated semigroups.

3. MAX-PLUS DUAL OPERATORS

  • LetSβ = Sβ(IRn) be the set of functions mapping IRn into IR which are uniformly semiconvex with constant β.
  • The following semiconvex duality result (Fleming and McEneaney, 2000), (McEneaney, 2003) requires only a small modification of convex duality and Legendre/Fenchel transform results (c.f. (Rockafellar and Wets, 1997)).
  • Semiconcavity is the obvious analogue of semiconvexity.
  • In particular, one has the following Theorem 3.8.

4. DISCRETE TIME APPROXIMATION

  • The method developed here will not involve any discretization over space nor any basis functions.
  • Of course this is obvious since otherwise one could not avoid the curse-of-dimensionality.
  • The discretization will be over time, where approximate µ processes will be constant over the length of each time-step.
  • The corresponding max-plus linear operator is B̂τ = ⊕ m∈M B̂mτ .
  • The following result on propagation of the semiconvex dual will also come in handy.

5. ALGORITHM AND EXAMPLES

  • Due to space limitations, the authors cannot give the steps in the actual algorithm that is generated by the above theory.
  • Further, the computational cost growth in space dimension n is limited to cubic growth.
  • The quoted computational times were obtained with a standard 2001 PC.
  • The backsubstitution error has been scaled by dividing by |x|2 + 10−5.

Did you find this useful? Give us your feedback

Citations
More filters
Journal ArticleDOI
TL;DR: This scheme is shown to be capable of learning the optimal control without requiring an initial guess and to enhance the efficiency of the proposed scheme when treating more complex nonlinear systems, an iterative algorithm based on Girsanov's theorem on the change of measure is derived.

59 citations


Cites methods from "A curse-of-dimensionality-free nume..."

  • ...In the first case, several attempts have been made to address the difficulty inherent in solving such nonlinear PDEs, aswell as the curse of dimensionality, with various different methods and approaches (Beard, Saridis, & Wen, 1997; Lasserre, Henrion, Prieur, & Trelat, 2008; McEneaney, 2007) for deterministic control problems, while a stochastic setting is considered in Gorodetsky, Karaman, and Marzouk (2015), Horowitz and Burdick (2014) and Horowitz, Damle, and Burdick (2014)....

    [...]

Journal ArticleDOI
TL;DR: In this article, the authors considered the matrix differential Riccati equation (DRE) as a finite-dimensional solution to a deterministic linear/quadratic control problem and proposed a semiconvex dual of the associated semigroup.

44 citations

Journal ArticleDOI
TL;DR: This work obtains specific error bounds for a previously obtained numerical method not subject to the curse-of-dimensionality of HJB PDEs where the Hamiltonian takes the form of a (pointwise) maximum of linear/quadratic forms.
Abstract: In previous work of the first author and others, max-plus methods have been explored for solution of first-order, nonlinear Hamilton-Jacobi-Bellman partial differential equations (HJB PDEs) and corresponding nonlinear control problems. Although max-plus basis expansion and max-plus finite-element methods can provide substantial computational-speed advantages, they still generally suffer from the curse-of-dimensionality. Here we consider HJB PDEs where the Hamiltonian takes the form of a (pointwise) maximum of linear/quadratic forms. The approach to solution will be rather general, but in order to ground the work, we consider only constituent Hamiltonians corresponding to long-run average-cost-per-unit-time optimal control problems for the development. We consider a previously obtained numerical method not subject to the curse-of-dimensionality. The method is based on construction of the dual-space semigroup corresponding to the HJB PDE. This dual-space semigroup is constructed from the dual-space semigroups corresponding to the constituent linear/quadratic Hamiltonians. The dual-space semigroup is particularly useful due to its form as a max-plus integral operator with kernel obtained from the originating semigroup. One considers repeated application of the dual-space semigroup to obtain the solution. Although previous work indicated that the method was not subject to the curse-of-dimensionality, it did not indicate any error bounds or convergence rate. Here we obtain specific error bounds.

36 citations


Cites methods from "A curse-of-dimensionality-free nume..."

  • ...In [29], [30], [31], [32], a new class of methods for first-order HJB PDEs was introduced, and these methods are not subject to the curse-of-dimensionality....

    [...]

Proceedings ArticleDOI
01 Dec 2009
TL;DR: It is shown that a similar, albeit more abstract, approach can be applied to deterministic game problems by finding reduced-complexity approximations to min-max sums of max-plus affine functions.
Abstract: In recent years, idempotent methods (specifically, max-plus methods) have been developed for solution of nonlinear control problems. It was thought that idempotent linearity of the associated semigroup was required for application of these techniques. It is now known that application of the max-plus distributive property allows one to apply the max-plus curse-of-dimensionality-free approach to stochastic control problems. Here, we see that a similar, albeit more abstract, approach can be applied to deterministic game problems. The main difficulty is a curse-of-complexity growth in the computational cost. Attenuation of this effect requires finding reduced-complexity approximations to min-max sums of max-plus affine functions. We demonstrate that this problem can be reduced to a pruning problem.

10 citations

References
More filters
Book
18 Dec 1997
TL;DR: In this paper, the main ideas on a model problem with continuous viscosity solutions of Hamilton-Jacobi equations are discussed. But the main idea of the main solutions is not discussed.
Abstract: Preface.- Basic notations.- Outline of the main ideas on a model problem.- Continuous viscosity solutions of Hamilton-Jacobi equations.- Optimal control problems with continuous value functions: unrestricted state space.- Optimal control problems with continuous value functions: restricted state space.- Discontinuous viscosity solutions and applications.- Approximation and perturbation problems.- Asymptotic problems.- Differential Games.- Numerical solution of Dynamic Programming.- Nonlinear H-infinity control by Pierpaolo Soravia.- Bibliography.- Index

2,747 citations


"A curse-of-dimensionality-free nume..." refers methods in this paper

  • ...The most common methods by far all fall into the class of finite element methods (cf. (Bardi and Capuzzo-Dolcetta, 1997), (Dupuis and Boué, 1999), among many others)....

    [...]

  • ...(Bardi and Capuzzo-Dolcetta, 1997), (Dupuis and Boué, 1999), among many others)....

    [...]

Journal ArticleDOI
John Rust1
TL;DR: In this paper, random versions of successive approximations and multigrid algorithms for computing approximate solutions to a class of finite and infinite horizon Markovian decision problems (MDPs) were introduced.
Abstract: This paper introduces random versions of successive approximations and multigrid algorithms for computing approximate solutions to a class of finite and infinite horizon Markovian decision problems (MDPs). We prove that these algorithms succeed in breaking the curse of dimensionality for a subclass of MDPs known as discrete decision processes (DDPs).

435 citations

Book
30 Apr 1997
TL;DR: In this article, a generalized solution of Bellman's Differential Equation and multiplicative additive asymptotics is presented, which is based on the Maslov Optimziation Theory.
Abstract: Preface. 1. Idempotent Analysis. 2. Analysis of Operators on Idempotent Semimodules. 3. Generalized Solutions of Bellman's Differential Equation. 4. Quantization of the Bellman Equation and Multiplicative Asymptotics. References. Appendix: (P. Del Moral) Maslov Optimziation Theory. Optimality versus Randomness. Index.

425 citations

Journal ArticleDOI
TL;DR: In this article, a nonlinear projection on subsemimodules is introduced, where the projection of a point is the maximal approximation from below of the point in the sub-semimmodule.

273 citations

Journal ArticleDOI
TL;DR: In this paper, an algebraic approach to idempotent functional analysis is presented, which is an abstract version of the traditional functional analysis developed by V. P. Maslov and his collaborators.
Abstract: This paper is devoted to Idempotent Functional Analysis, which is an “abstract” version of Idempotent Analysis developed by V. P. Maslov and his collaborators. We give a brief survey of the basic ideas of Idempotent Analysis. The correspondence between concepts and theorems of traditional Functional Analysis and its idempotent version is discussed in the spirit of N. Bohr's correspondence principle in quantum theory. We present an algebraic approach to Idempotent Functional Analysis. Basic notions and results are formulated in algebraic terms; the essential point is that the operation of idempotent addition can be defined for arbitrary infinite sets of summands. We study idempotent analogs of the basic principles of linear functional analysis and results on the general form of a linear functional and scalar products in idempotent spaces.

222 citations

Frequently Asked Questions (1)
Q1. What are the contributions in "A curse-of-dimensionality-free numerical method for a class of hjb pde’s" ?

Here the authors consider HJB PDEs where the Hamiltonian takes the form of a ( pointwise ) maximum of linear/quadratic forms.