LUND UNIVERSITY
PO Box 117
221 00 Lund
+46 46-222 00 00
Analysis of Recursive Stochastic Algorithms
Ljung, Lennart
1976
Document Version:
Publisher's PDF, also known as Version of record
Link to publication
Citation for published version (APA):
Ljung, L. (1976).
Analysis of Recursive Stochastic Algorithms
. (Technical Reports TFRT-7097). Department of
Automatic Control, Lund Institute of Technology (LTH).
Total number of authors:
1
General rights
Unless other specific re-use rights are stated the following general rights apply:
Copyright and moral rights for the publications made accessible in the public portal are retained by the authors
and/or other copyright owners and it is a condition of accessing publications that users recognise and abide by the
legal requirements associated with these rights.
• Users may download and print one copy of any publication from the public portal for the purpose of private study
or research.
• You may not further distribute the material or use it for any profit-making activity or commercial gain
• You may freely distribute the URL identifying the publication in the public portal
Read more about Creative commons licenses: https://creativecommons.org/licenses/
Take down policy
If you believe that this document breaches copyright please contact us providing details, and we will remove
access to the work immediately and investigate your claim.
A¡,ÏALYSIS
0F
RIüJRSI\¿Ë,
ST0üi{STIC
AI"G0RtTT{,,rs
f
lennart
Ljung
Depanürcnt
of
Autornatic
Contnol
Lund
ïnstitute
of
?echnology
S-220
07
Lurrd,
$rr¿eden
Abstract
Recur"sive
algonithms,
whene
r.andom
obsenvations
enter^
are
studied
in
a fain-
ly
general
frrarræwot'k.
An
ínpontant
featr¡ne
is
that
the obsenvations
rnay
de-
penci
on
pneviotts
rrout¡:utÊ't
of
the
algonithm.
The
considered
class
of
algo-
nithms
containsr
e
9,
stochastic
apprrrxirnation
algonít¡rrns,
necur.sive
identi-
fication
algonithns
arid algonithr"ns
fon
adaptíve
cont:rol
of
linear.
systerns.
ft
is
shcrnn
how
a
deterrn:í-nistic
diffe::entíal
equation
can
be
associated
with
the
algor"íthn.
Problems
like
convergence
with
pnobabiliry
one,
possíb1e
con-
vengence
points
and
asymptotic
behavior¡:-
of
the
algor.ithm
can
all
be
studied
in
terrns
of
ttris
differential
equation.
ïheorems
stating
the
precise
relation-
ships
betr¡een
the
díffe¡.ential
equation
and
the algonithm
arne
given
as
well
as
exanples
of
applications
of
the
nesults
to
pnoblems
in
identifícation
and
ad,ap*
tive
contnol.
A rnajon
pa::t
of
this wotlk
has
been
supporrted
by
the Swedish
Boarl
fon lech-
nical
Ðevelopnent
unden
cont:ract
No.
7g3546.
t
1
1.
ÏATTRODLICTTON
Recr.¡r"sive
algorithms,
whene
stoclrastic
observations
enter
are
eorrnon
in many
fields.
Tn
the control
and
estimation
litenatr-ne
such
algorithms
are
rridely
discussedr
e
g
in
connection
with
adaptive
contnol,
(adaptive)
filte¡ing
and
on-line
identificatíon.
Tire
convergence
analysis
of
the algorithns
ís
not sel*
dcm
diffict-rlt.
As a rule,
special
teehniques
for
analysis
are
used for
eacþr
t¡pe
of application
and
often
the
convergence
pr.operties
have
to
'be
studied
only
by
sjmulation
Ïn
this
papen
a
genenal
apprnach
to the
enalysis
of
the
asSrnptotic
behaviou:r
of
recursive
algorithms
is
ciescribed.
In
effect,
the
convengenee
analysis
is
reduced
tcl
stability
analysis
of
a
deternrinistico
orriina:¡¡
differentiat
equa-
tion.
This
technique
ís
believed
to be a
fainly genenal
toÕl
arrd
to
have
a
wide
applicability.
Applications
to various pnoblens
have been
published
Ín
[1]r
1"21,
[3J,
[aJ
and
scrnre
thecry
Ì^ras
presented
in
[SJ.
The
objective
of
the
present
papen
is
to
give
a comprehensive
presentation
of
forunl
results
and
useful
techniques
fon
the
conver.gence
analysis,
as wel1
as
to
Íllustr"ate
urith
sevenal
exanples
how
the
tedrniques
can be applied.
fn Section
2 a
genenal
recunsive
algonitfun
is
descríbed
and
d.iscussed.
A
heu-
nistic
trealrnent
of
the conveïlgence
problern
is
given
in Section
B
and
this
leads
to
the basic
ideas
of
the
present
apprnach,
Seetion
4
contains
a dÍs*
cussion
of
the
conditions
whích
ar"e
ùrposed
on,the
algoníthm
jn
order
to
pnove
the
fornal
results.
These
theorems
a::e
gíven
in
Sectíons
S and
6. The
thecrems
suggest
ccr-tain
tecfrniques
fon
the
converagence
analysis,
årrd
these
aspects
ane
treated
in
section
7.
several.
exanples
of horo¡
the
theoreng rnåy
be
used, sone
of
then
revíewing prevíous
applications
are
given
in section
B.
2.
TT{H
ArßORITTü'Ï
A
genenal
rectusive
algonithm
can
be r,,nritten
x(t)
=
x(t-t)
+
v(t)Q[t;x{t-t)rw(t)i
{1)
where
x(')
is
a
sequence
of
n-d:i:nensional
coltrtn
vectors,
whieh
are
the objects
2
of
our i¡tenest.
!'le
shall
refen
to x(.
)
as
ttthe
gslj{ngFåÍ,
and
they
couldo
e
g,
be the
current
estimates
of sone
wrl<not^¡n
pararreter
vestor.
They
could,
hcmeven,
also
be
pananetens
that
deterrnine
a feedbaek
.1aw
of an
adaptive
con-
trolleno
etc,
andwe
shall
be
precise
about
the cùraracter.
ofx(.)
only
in
the
exanples
below.
The
sequenee y
(.
)
is
thnoLlghoub
the
papen
assu¡',ed
to
be å
se-
quence
of
positive
scalans. The
m-di¡rensional
vecton
t{t) is ar¡
obsenvation
obtained
at
tj.me
to and
these
are
the
objeets
that
cause x(t-1
) to
be
updated
to
talc.e
new
inforEnation
into account.
(The
notion
Itobsenvatíon'r
does
nÕt
håve
"to
be
taken
literally.
The
va:riable
p
may
very
well
be the result
of certain
treatnent
of
actual
rtËasureilents.)
The
observations
are in
genenal
functions
of
the
previous
estinates
x(.) and
of
a
sequence
of r^andom
vecto:rs
e(.).
This
rneåns
that
the obsenvatíon
is a
random
variable,
vrhích
nray be
affected
by
pne-
rrious
estj¡iates.
This
ís
the
case,
e
g,
fon
adaptive
systems, wlren
the
Ínput
signal
is
detenn:Ì¡ed
on
t̡e
basís of
pnevious
estimates. If
the
expenÍnrent
de*
signenhas
scxne
test
sigral
athis
disposal,
this
rfl¿ybe
included
ín
e(.).
The
function
Q(. i'r.
)
from
RxRn*Rm
into
Rn is
a
deterqninistic function
wíth
5oÍÊ
regular-ity
conditions
to
be specified
belor^r" lhis
funeti.on,
togethen
w.ith
the
choice
of
the scålår
trgain'
sequence
y(.
) dete::nr-ine entírely
the
aLgonÍthm.
ï'rle
shall
not
work
wíth
corryletely genenal
dependence of
rp(t)
on
x(.); sonæ
re-
sults
fon
thÍs
case
ar€
gírren
in
[6
J,
but
the foilowing
struct.u:re
for
the
ge-
neration
of
t.o(.
)
i¿ill
be
used:
ç(t)
=
A(x(t*1))u(t-t)
+
n(x(t-r))e(t)
(a)
Her.e
A(')
and
B(.
)
are
mlm
and
mln.matnix
funetions.
Req+r"k:
rt
is
per.haps
n¡¡re
natunar
ta
thirrk
of
an obsenvatíon
õ(t)
as
the
(lower*
dimensional)
output
of a
dynaruical
systern
Uke
(Z),
õ(t)
=
C(x(t-t))u{t).
How-
everl,
this
case
is
natr.lrrally
srrbsumed
in
the
pnesent
one,
since
Ç(t)
nray ente:l
in
Q
only
as
the
combination
õ(t).
The
assunption
(2)
seems
to be
appnopniate
fon
nrany applícations.
The
sãnìe
re-
sults
at¡ those
belcn^¡
can
be obtaj¡red
also
fon
non linear
dyr.amics.
9(t)
=
e(t;ç(t-t
)
rx(t-1
),e(t))
and tåe
proofs
fon
this
cðse âre grrren
in
t?1.
(3)