An efficient Newton-type scheme for the approximate on-line solution of optimization problems as they occur in optimal feedback control is presented. The scheme allows a fast reaction to disturbances by delivering approximations of the exact optimal feedback control which are iteratively refined during the runtime of the controlled process. The contractivity of this real-time iteration scheme is proven, and a bound on the loss of optimality---compared with the theoretical optimal solution---is given. The robustness and excellent real-time performance of the method is demonstrated in a numerical experiment, the control of an unstable system, namely, an airborne kite that shall fly loops.

/pdf/a-real-time-iteration-scheme-for-nonlinear-optimization-in-2hl0dw431p.pdf

A Real-Time Iteration Scheme for Nonlinear Optimization in Optimal Feedback Control

Automatic Differentiation (AD) is a maturing computational technology. It has become a mainstream tool used by practicing scientists and computer engineers. The rapid advance of hardware computing power and AD tools has enabled practitioners to generate derivative enhanced versions of their code for a broad range of applications in applied research and development. Automatic Differentiation of Algorithms provides a comprehensive and authoritative survey of all recent developments, new techniques, and tools for AD use.

Automatic Differentiation Of Algorithms: From Simulation To Optimization

This paper proposes to use Nesterov's fast gradient method for the solution of linear quadratic model predictive control (MPC) problems with input constraints. The main focus is on the method's a priori computational complexity certification which consists of deriving lower iteration bounds such that a solution of pre-specified suboptimality is obtained for any possible state of the system. We investigate cold- and warm-starting strategies and provide an easily computable lower iteration bound for cold-starting and an asymptotic characterization of the bounds for warm-starting. Moreover, we characterize the set of MPC problems for which small iteration bounds and thus short solution times are expected. The theoretical findings and the practical relevance of the obtained lower iteration bounds are underpinned by various numerical examples and compared to certification results for a primal-dual interior point method.

Computational Complexity Certification for Real-Time MPC With Input Constraints Based on the Fast Gradient Method

A review On reinforcement learning: Introduction and applications in industrial process control

Efficient Solution of Dynamic Optimization and NMPC Problems

Naive implementations of Newton's method for unconstrainedN-stage discrete-time optimal control problems with Bolza objective functions tend to increase in cost likeN
3 asN increases. However, if the inherent recursive structure of the Bolza problem is properly exploited, the cost of computing a Newton step will increase only linearly withN. The efficient Newton implementation scheme proposed here is similar to Mayne's DDP (differential dynamic programming) method but produces the Newton step exactly, even when the dynamical equations are nonlinear. The proposed scheme is also related to a Riccati treatment of the linear, two-point boundary-value problems that characterize optimal solutions. For discrete-time problems, the dynamic programming approach and the Riccati substitution differ in an interesting way; however, these differences essentially vanish in the continuous-time limit.

Efficient dynamic programming implementations of Newton's method for unconstrained optimal control problems

We formulate a locally superlinearly convergent projected Newton method for constrained minimization in a Cartesian product of balls. For discrete-time,N-stage, input-constrained optimal control problems with Bolza objective functions, we then show how the required scaled tangential component of the objective function gradient can be approximated efficiently with a differential dynamic programming scheme; the computational cost and the storage requirements for the resulting modified projected Newton algorithm increase linearly with the number of stages. In calculations performed for a specific control problem with 10 stages, the modified projected Newton algorithm is shown to be one to two orders of magnitude more efficient than a standard unscaled projected gradient method.

J. C. Dunn

Papers

Efficient dynamic programming implementations of Newton's method for unconstrained optimal control problems

A projected Newton method in a Cartesian product of balls