A class of optimal state-delay control problems

TL;DR: It is shown that this optimal state-delay control problem can be formulated as a nonlinear programming problem in which the cost function is an implicit function of the decision variables and an efficient numerical method for determining thecost function’s gradient is developed.

Abstract: We consider a general nonlinear time-delay system with state-delays as control variables. The problem of determining optimal values for the state-delays to minimize overall system cost is a non-standard optimal control problem–called an optimal state-delay control problem–that cannot be solved using existing optimal control techniques. We show that this optimal control problem can be formulated as a nonlinear programming problem in which the cost function is an implicit function of the decision variables. We then develop an efficient numerical method for determining the cost function’s gradient. This method, which involves integrating an auxiliary impulsive system backwards in time, can be combined with any standard gradient-based optimization method to solve the optimal state-delay control problem effectively. We conclude the paper by discussing applications of our approach to parameter identification and delayed feedback control.

Summary

2. Problem formulation

• Let T denote the set of all such admissible state-delay vectors.
• Let Z denote the set of all such admissible parameter vectors.

Any vector

• The authors assume that the following conditions are satisfied.
• These time points are called characteristic times in the optimal control literature [2, 17, 18] .
• As the authors will see, cost functions with characteristic times arise in parameter identification problems, where the aim is to minimize the discrepancy between predicted and observed system output at a set of sample times.
• The authors optimal state-delay control problem is defined formally below.

3. Gradient computation

• Problem (P) can be viewed as a nonlinear optimization problem in which the decision vectors τ and ζ influence the cost function J implicitly through the governing dynamic system (1)-( 2).
• Thus, if the gradient of J can be computed for each admissible control pair, then Problem (P) can be solved using existing gradient-based optimization methods, such as sequential quadratic programming (see [20, 21] ).

3.3. Solving Problem (P)

• Evolves forward in time (starting from an initial condition), while the auxiliary system (6)-( 8) evolves backwards in time (starting from a terminal condition).
• Thus, since the state and auxiliary systems evolve in opposite directions, and the auxiliary system depends on the solution of the state system, these two systems cannot be solved simultaneously.
• Instead, the state system is solved first in Step 1, and then the solution of the state system is used to solve the auxiliary system in Step 2.
• In practice, numerical integration methods are used to solve the state and auxiliary systems.
• The integrals in the gradient formulae ( 9) and ( 19) can be evaluated using standard numerical quadrature rules.

4.1. Problem formulation

• Using the algorithm in Section 3.3, only 2n differential equations need to be solved.
• Thus, their new method is ideal for online applications in which efficiency is paramount.

6. Conclusion

• The authors new method is applicable to systems with nonlinear terms containing more than one state-delay.
• The authors have restricted their attention in this paper to systems with time-invariant time-delays.
• Such problems arise in the control of crushing processes [19] and mixing tanks with recycle loops [27] .

Figures

Figure 3: Optimal states of the Chen chaotic system in Section 5.3

Table 1: Control values and switching times for control functions (29) and (30).

Figure 2: Optimal angular displacement for the closed-loop inverted pendulum system

Table 2: Numerical convergence of the cost values for the example in Section 4.2.

Figure 1: Numerical convergence of the output trajectory for run 5 in Section 4.2.

Full text

NOTICE: This is the author’s version of a work that was accepted for publication in
Nonlinear Analysis: Real World Applications. Changes resulting from the publishing
process, such as peer review, editing, corrections, structural formatting, and other
quality control mechanisms may not be reflected in this document. Changes may
have been made to this work since it was submitted for publication. A definitive
version was subsequently published in Nonlinear Analysis: Real World Applications,
Vol. 14 (2013). doi: 10.1016/j.nonrwa.2012.10.017

A class of optimal state-delay control problems
Qinqin Chai
a,b,∗
, Ryan Loxton
b
, Kok Lay Teo
b
, Chunhua Yang
a
a
School of Information Science & Engineering, Central South University,
Changsha, P. R. China
b
Department of Mathematics & Statistics, Curtin University, Perth, Australia
Abstract
We consider a g eneral nonlinear time-delay system with state-delays as con-
trol variables. The problem of determining optimal values for the state-delays
to minimize overall system cost is a non- standard optimal control problem—
called an optimal state-delay cont r ol problem—that cannot be solved using
existing techniques. We show that this optimal control problem can be for-
mulated as a nonlinear programming problem in which the cost function is
an implicit function o f the decision var iables. We then develop an efficient
numerical method for determining the cost function’s gradient. This method,
which involves integrating an impulsive dynamic system backwards in time,
can be combined with any standard gradient -based optimization method to
solve the optimal state-delay control problem effectively. We conclude the
paper by discussing a pplications of our approach to parameter identification
and delayed feedback control.
Keywords: time-delay, optimal control, nonlinear optimization, parameter
identification, delayed feedback contr ol
∗
Corresponding author
Email addresses: kppqing@163.com (Qinqin Chai), r.loxton@curtin.edu.au (Ryan
Loxton), k.l.teo@curtin.edu.au (Kok Lay Teo), ychh@csu.edu.cn (Chunhua Yang)
Preprint submitted to Nonlinear Analysis Series B July 27, 2014

1. Introduction1
Time-delay systems arise in many real-world applications—e.g. evapo-2
ration and purification processes [1 , 2], aerospace models [3], and human3
immune response [4]. Over the past two decades, various optimal control4
methods have been developed fo r time-delay systems. Well-known tools in-5
clude the necessary conditions for o ptima lity [5, 6] and numerical methods6
based on the control parameterization technique [7, 8]. These existing opti-7
mal control methods are restricted to time-delay systems in which the delays8
are fixed and known. In this paper, we consider a new class of optimal control9
problems in which the delays are not fixed, but are instead control variables10
to be chosen optimally. Such problems ar e called optimal state-delay control11
problems.12
As an example of an optimal state-delay control problem, consider a13
system of delay-differential equations with unknown delays. This delay-14
differential system is a dynamic model for some phenomenon under con-15
sideration. The problem is to choose values for the unknown delays (and16
possibly other model parameters) so that the system output predicted by17
the model is consistent with exp erimental data. This so-called parameter18
identification problem can be for mulated as an optimal state-delay control19
problem in which the delays and model parameters are decision variables,20
and the cost function measures the least-squares error between predicted21
and observed system output.22
Parameter ident ification for time-delay systems has been an active area23
of research over the past decade. Existing techniques fo r parameter identi-24
fication include interpolation methods [9], genetic algorithms [10], and the25
delay operator transform method [11]. These techniques are mainly designed26
for single-delay linear systems. In contrast, the computational approach27
to be developed in this paper, which is based on formulating and solving28
the parameter identification pro blem as an optimal state-delay control prob-29
lem, can handle systems with nonlinear dynamics and multiple time-delays.30
This computational approach is motiva t ed by our earlier work in [12 ], which31
2

presents a parameter identification algor it hm based on nonlinear program-32
ming techniques. This algorithm has two limitations: (i) it is only applicable33
to systems in which each nonlinear term contains a single delay and no un-34
known parameters; and (ii) it involves integrating a large number of auxiliary35
delay-differential systems (o ne auxiliary system for each unknown delay a nd36
model parameter). The new approach to be developed in t his paper does not37
suffer fr om these limitations. In particular, our new approach only requires38
the integration of one auxiliary system, regardless o f the number of delays39
and par ameters in the underlying dynamic model.40
Another important application of optimal state-delay control problems41
is in delayed feedback control. In delayed feedback control, the system’s42
input function is chosen to be a linear function of the delayed state, as op-43
posed to traditional feedback control in which the input is a function of the44
current (undelayed) state. Voluntarily introducing delays via delayed feed-45
back control can be beneficial for certain types of systems; see, for example,46
[13, 14, 1 5]. The problem of choo sing optimal values for the delays in a de-47
layed feedback controller can be f ormulated as an optimal state-delay control48
problem.49
Our goal in this paper is to develop a unified computational approach50
for solving optimal state-delay control problems. A key aspect of our work51
is t he derivation of an auxiliary impulsive system, which turns out to be52
the analogue of the costate system in classical optimal control. We derive53
formulae fo r the cost function’s gradient in terms of the solution of this im-54
pulsive system. On this basis, the opt ima l state-delay control problem can55
be solved by combining numerical integration and nonlinear programming56
techniques. This a pproach has proven very effective for the two specific ap-57
plications mentioned above—parameter identification and delayed feedback58
control.59
The remainder of the paper is organized as follows. We first formulate the60
optimal state-delay control problem in Section 2, befor e introducing the aux-61
iliary impulsive system and deriving gradient formular in Section 3. Section 462
3

is devoted to parameter identification problems, and Section 5 is devoted to63
delayed feedback control. We make some concluding remarks in Section 6.64
2. Problem formulation65
Consider the following nonlinear time-delay system:
˙
x(t) = f (x(t), x(t − τ
1
), . . . , x(t − τ
m
), ζ), t ∈ [0, T], (1)
x(t) = φ(t, ζ), t ≤ 0, (2)
where T > 0 is a given terminal time; x(t) = [x
1
(t), . . . , x
n
(t)]
⊤
∈ R
n
is66
the state vector; τ
i
, i = 1, . . . , m are state-delays; ζ = [ζ
1
, . . . , ζ
r
]
⊤
∈ R
r
is a67
vector of system parameters; and f : R
(m+1)n
×R
r
→ R
n
and φ : R×R
r
→ R
n
68
are given f unctions.69
System (1)-(2) is controlled via the state-delays and system parameters—
these must be chosen optimally so that the system behaves in the best pos-
sible manner. We impose the following bound constraints:
a
i
≤ τ
i
≤ b
i
, i = 1, . . . , m, (3)
and
c
j
≤ ζ
j
≤ d
j
, j = 1, . . . , r, (4)
where a
i
and b
i
are given constants such that 0 ≤ a
i
< b
i
, and c
j
and d
j
are70
given constants such that c
j
< d
j
.71
Any vector τ = [τ
1
, . . . , τ
m
]
⊤
∈ R
m
satisfying ( 3) is called an admissible72
state-delay vector. Let T denote t he set of all such admissible state-delay73
vectors.74
Any vector ζ = [ζ
1
, . . . , ζ
r
]
⊤
∈ R
r
satisfying (4) is called an admissible75
parameter vector. Let Z denote the set of all such admissible parameter76
vectors.77
Any combined pair (τ , ζ) ∈ T × Z is called an admissi b l e control pair for78
system ( 1)-(2).79
We assume that the following conditions are satisfied.80
4

Frequently asked questions

Q1. What have the authors contributed in "A class of optimal state-delay control problems" ?

The authors consider a general nonlinear time-delay system with state-delays as control variables. The authors show that this optimal control problem can be formulated as a nonlinear programming problem in which the cost function is an implicit function of the decision variables. The authors conclude the paper by discussing applications of their approach to parameter identification and delayed feedback control.