Turbulence, Coherent Structures, Dynamical Systems and SymmetryP. Holmes, J. L. Lumley, G. Berkooz, and C. W. Rowley, 2nd ed., Cambridge University Press, Cambridge, England, U.K., 2012, 386 pp., $90

An approach to solve finite time horizon suboptimal feedback control problems for partial differential equations is proposed by solving dynamic programming equations on adaptive sparse grids. A semi-discrete optimal control problem is introduced and the feedback control is derived from the corresponding value function. The value function can be characterized as the solution of an evolutionary Hamilton---Jacobi Bellman (HJB) equation which is defined over a state space whose dimension is equal to the dimension of the underlying semi-discrete system. Besides a low dimensional semi-discretization it is important to solve the HJB equation efficiently to address the curse of dimensionality. We propose to apply a semi-Lagrangian scheme using spatially adaptive sparse grids. Sparse grids allow the discretization of the value functions in (higher) space dimensions since the curse of dimensionality of full grid methods arises to a much smaller extent. For additional efficiency an adaptive grid refinement procedure is explored. The approach is illustrated for the wave equation and an extension to equations of Schrodinger type is indicated. We present several numerical examples studying the effect the parameters characterizing the sparse grid have on the accuracy of the value function and the optimal trajectory.

https://hal.archives-ouvertes.fr/hal-01185912/document

Suboptimal Feedback Control of PDEs by Solving HJB Equations on Adaptive Sparse Grids

We propose new and original mathematical connections between Hamilton–Jacobi (HJ) partial differential equations (PDEs) with initial data and neural network architectures. Specifically, we prove that some classes of neural networks correspond to representation formulas of HJ PDE solutions whose Hamiltonians and initial data are obtained from the parameters of the neural networks. These results do not rely on universal approximation properties of neural networks; rather, our results show that some classes of neural network architectures naturally encode the physics contained in some HJ PDEs. Our results naturally yield efficient neural network-based methods for evaluating solutions of some HJ PDEs in high dimension without using grids or numerical approximations. We also present some numerical results for solving some inverse problems involving HJ PDEs using our proposed architectures.

Overcoming the curse of dimensionality for some Hamilton–Jacobi partial differential equations via neural network architectures

We present an accelerated algorithm for the solution of static Hamilton--Jacobi--Bellman equations related to optimal control problems. Our scheme is based on a classic policy iteration procedure, which is known to have superlinear convergence in many relevant cases provided the initial guess is sufficiently close to the solution. This limitation often degenerates into a behavior similar to a value iteration method, with an increased computation time. The new scheme circumvents this problem by combining the advantages of both algorithms with an efficient coupling. The method starts with a coarse-mesh value iteration phase and then switches to a fine-mesh policy iteration procedure when a certain error threshold is reached. A delicate point is to determine this threshold in order to avoid cumbersome computations with the value iteration and at the same time to ensure the convergence of the policy iteration method to the optimal solution. We analyze the methods and efficient coupling in a number of examples...

An efficient policy iteration algorithm for dynamic programming equations

Handling obstacles in pedestrian simulations: Models and optimization

The classical dynamic programming (DP) approach to optimal control problems is based on the characterization of the value function as the unique viscosity solution of a Hamilton--Jacobi--Bellman eq...

An Efficient DP Algorithm on a Tree-Structure for Finite Horizon Optimal Control Problems

In this paper infinite horizon optimal control problems for nonlinear high-dimensional dynamical systems are studied. Nonlinear feedback laws can be computed via the value function characterized as the unique viscosity solution to the corresponding Hamilton--Jacobi--Bellman (HJB) equation which stems from the dynamic programming approach. However, the bottleneck is mainly due to the curse of dimensionality, and HJB equations are solvable only in a relatively small dimension. Therefore, a reduced-order model is derived for the dynamical system, using the method of proper orthogonal decomposition (POD). The resulting errors in the HJB equations are estimated by an a priori error analysis, which is utilized in the numerical approximation to ensure a desired accuracy for the POD method. Numerical experiments illustrates the theoretical findings.

/pdf/error-analysis-for-pod-approximations-of-infinite-horizon-1kfv9x293o.pdf

Error Analysis for POD Approximations of Infinite Horizon Problems via the Dynamic Programming Approach

We present an algorithm for the approximation of a finite horizon optimal control problem for advection-diffusion equations The method is based on the coupling between an adaptive POD representation of the solution and a Dynamic Programming approximation scheme for the corresponding evolutive Hamilton-Jacobi equation We discuss several features regarding the adaptivity of the method, the role of error estimate indicators to choose a time subdivision of the problem and the computation of the basis functions Some test problems are presented to illustrate the method

An adaptive POD approximation method for the control of advection-diffusion equations

We present an algorithm for the approximation of a finite horizon optimal control problem for advection-diffusion equations. The method is based on the coupling between an adaptive POD representation of the solution and a Dynamic Programming approximation scheme for the corresponding evolutive Hamilton–Jacobi equation. We discuss several features regarding the adaptivity of the method, the role of error estimate indicators to choose a time subdivision of the problem and the computation of the basis functions. Some test problems are presented to illustrate the method.

An Adaptive POD Approximation Method for the Control of Advection-Diffusion Equations

The reconstruction of a 3D object or a scene is a classical inverse problem in Computer Vision. In the case of a single image this is called the Shape-from-Shading (SfS) problem and it is known to be ill-posed even in a simplified version like the vertical light source case. A huge number of works deals with the orthographic SfS problem based on the Lambertian reflectance model, the most common and simplest model which leads to an eikonal type equation when the light source is on the vertical axis. In this paper we want to study non-Lambertian models since they are more realistic and suitable whenever one has to deal with different kind of surfaces, rough or specular. We will present a unified mathematical formulation of some popular orthographic non-Lambertian models, considering vertical and oblique light directions as well as different viewer positions. These models lead to more complex stationary nonlinear partial differential equations of Hamilton-Jacobi type which can be regarded as the generalization of the classical eikonal equation corresponding to the Lambertian case. However, all the equations corresponding to the models considered here (Oren-Nayar and Phong) have a similar structure so we can look for weak solutions to this class in the viscosity solution framework. Via this unified approach, we are able to develop a semi-Lagrangian approximation scheme for the Oren-Nayar and the Phong model and to prove a general convergence result. Numerical simulations on synthetic and real images will illustrate the effectiveness of this approach and the main features of the scheme, also comparing the results with previous results in the literature.

Maurizio Falcone

Papers

An Efficient DP Algorithm on a Tree-Structure for Finite Horizon Optimal Control Problems

Error Analysis for POD Approximations of Infinite Horizon Problems via the Dynamic Programming Approach

An adaptive POD approximation method for the control of advection-diffusion equations

An Adaptive POD Approximation Method for the Control of Advection-Diffusion Equations

Analysis and approximation of some Shape-from-Shading models for non-Lambertian surfaces