On Geometric Ergodicity of CHARME Models
J¨urgen Franke
∗
Jean-Pierre Stockis
Joseph Tadjuidje Kamgaing
University of Kaiserslautern
January 10, 2007
Abstract
In this paper we consider a CHARME Model, a class of generalized mixture of
nonlinear nonparametric AR-ARCH time series. We apply the theory of Markov models
to derive as ymptotic stability of this model. Indeed, the goal is to provide some sets
of conditions under which our model is geometric ergodic and therefore satisfies some
mixing conditions. This result can be considered as the basis toward an asymptotic
theory for our model.
Keywords: Nonparametric AR-ARCH; Mixture Models; Markov chain; Geometric Ergodicity
∗
University of Kaiserslautern, Department of Mathematics,Erwin-Schroedinger-Str., 67663 Kaisers-
lautern, Germany. E-mail address: franke@mathematik.uni-kl.de
The work was supported by the Deutsche Forschungsgemeinschaft (DFG) as part of the priority research
program 1114 Mathematical Methods of Time Series and Digital Image Analysis, the center of excellence
Dependable Adaptive Systems and Mathematical Modeling funded by the state of Rhineland-Palatinate as
well as the ”Graduiertenkolleg Mathematik und Praxis” and by the Fraunhofer ITWM.
1
1 Introduction
Nonparametric conditional heteroscedastic autoregressive (nonlinear CHARN) models of the form
X
t
= m(X
t−1
, · · · , X
t−p
) + σ(X
t−1
, · · · , X
t−p
)
t
, (1.1)
m and σ unknown functions,
t
independent identically distributed (i.i.d.) random variables with
mean 0, play an important role in many fields of application, for example in econometrics or
finance, see for example H¨ardle and Tsybakov [2], Franke, Neumann and Stockis [6], Hafner
[15]. Theoretical results about stability properties of this processes are available. In particular, the
important property of geometric ergodicity is obtained under some conditions.
In practice, it is often not realistic to assume that the observed process has the same trend function
m and the same volatility function σ at each time instant. In this paper we are analyzing the
so-called Conditional Heteroscedastic Autoregressive Mixture of Experts, henceforth CHARME,
models. Here, a hidden Markov chain {Q
t
} with values in a finite set of states {1, 2, · · · , K} drives
the dynamics of {X
t
} and our model is defined as follows
X
t
=
K
X
k=1
S
tk
(m
k
(X
t−1
, · · · , X
t−p
) + σ
k
(X
t−1
, · · · , X
t−p
)
t
) (1.2)
with
S
tk
=
(
1 for Q
t
= k
0 otherwise
(1.3)
m
k
, σ
k
, k = 1, · · · , K unknown functions,
t
i.i.d. random variables with mean 0.
Notice that for sake of simplicity of notation, we take the same number of comp onents p in each
trend function m
k
and volatility function σ
k
. This is done without loss of generality if we take p
large enough.
We call this mo dels CHARME since many authors using a mixture of models, e.g. in engineering
are calling them mixture of experts as soon as nonparametric functions estimates, typically neural
networks, are considered, see, e.g. M¨uller et al. [8], Jacob et al. [12], or Jiang and Tanner [10].
CHARME is quite useful for modeling time series data which are piecewise stationary such that
their dynamics switch sometimes from one state to another. A typical example is given by stock
returns if the market changes from a quiesce nt to a volatile phase. Tadjuidje [16] gives some
applications of such models to financial data in the context of asset management and risk analysis
where the state functions m
k
, σ
k
, k = 1, · · · , K, are e stimate d by neural networks.
Independently of the type of estimates considered, a crucial c ondition for developing a theory for
estimation and testing in the setting of CHARME is the existence of a stochastic proces s satisfying
(1.2) which is geometric ergodic. In this paper we investigate separately the case p = 1 (s ec tion
1) and the case p ≥ 1 (section 2) since they differ somewhat with respect to the formulation and
proof. In particular, the case p = 1 is interesting on its own. We formulate for both cases two
different sets of conditions.
2 First conditions for geometric ergodicity of CHARME
proce sses
We focus on our CHARME model (1.2) and make the following assumptions
A. 1 The process {Q
t
} with values on {1, · · · , K} is a first order strictly stat ionary Markov chain
which is irreducible and aperiodic with probability distribution (π
1
, · · · , π
K
) and transition proba-
bility matrix A = (a
ij
)
1≤i,j≤K
.
2
Obviously, {S
t
= (S
t1
, · · · , S
tK
)
0
} inherits the properties of {Q
t
}.
A. 2 Let G
t−1
= σ{X
r
, r ≤ t −1} be the σ-algebra generated by {X
r
, r ≤ t − 1} and G
t−1
any event
in G
t−1
. Then
P (Q
t
= j | Q
t−1
= i, G
t−1
) = P (Q
t
= j | Q
t−1
= i), ∀ i, j
This assumption means that the hidden process Q
t
is indep e ndent of the past observations given
its own past, i.e. Q
t−1
.
A. 3 Given (Q
t−1
, X
t−1
, X
t−2
, · · · ), Q
t
is uncorrelated with the innovation
t
.
A. 4
t
is independent of X
t−1
, X
t−2
, · · · .
A. 5 The functions m
k
and σ
k
are bounded on compact sets for all k, there exists a δ such that
σ
k
(u) ≥ δ > 0, for all k, u.
A. 6 The i.i.d. random variables
t
have a density f which is continuous and positive everywhere.
These assumptions are reasonable conditions for hidden Markov chain models, see e.g. Francq and
Roussignol [11] or Francq, Roussignol and Zakoian [9].
Now, we restrict ourselves for the rest of this section to the case p = 1, i.e. m
k
, σ
k
are functions on
the real line. We first assume
A. 7 The i.i.d. random variables
t
have mean 0 and variance σ
2
= 1
A. 8
max
l∈{1,··· ,K}
lim sup
|x|−→∞
P
k
a
lk
(m
2
k
(x) + σ
2
k
(x))
x
2
< 1
A.8 is the generalization of the well-known sufficient condition for geometric ergodicity in the case
of model (1.1). Now we need a Markov chain representing the transformed mixture proces s: under
assumptions A.1 to A.4 it is easily seen that if we define, as previously, S
t
= (S
t1
, · · · , S
tK
)
0
, then,
ζ
t
= (S
t
, X
t
)
0
is a Markov chain.
Theorem 1 Under A.1 to A.8, {ζ
t
} is geometrically ergodic.
Proof: We are going to prove that the conditions of Theorem 15.0.1, (iii) of Meyn and Tweedie
[7], pp 354 − 355, are satisfied.
• {ζ
t
} is ϕ-irreducible if we take ϕ as the product of the stationary probability distribution
measure on {1, · · · , K} and the Lebesgue measure on R
This can be proven as follows:
Let A = A
1
× A
2
be such that ϕ(A) > 0. Then A
1
contains at least one integer between 1 and K
and it is enough to prove that there exists t such that
P
S
t+1
X
t+1
∈ {e} × A
2
| S
1
= s
l
, X
1
= x
> 0
3
with e a unit vector with the the kth component equal 1 and s
l
a unit vector with the lth component
equal 1. By definition,
P
S
2
X
2
∈ {e} × A
2
| S
1
= s
l
, X
1
= x
= P (Q
2
= k, X
2
∈ A
2
| S
1
= s
l
, X
1
= x)
= P (X
2
∈ A
2
| Q
2
= k, S
1
= s
l
, X
1
= x) P (Q
2
= k | Q
1
= l, X
1
= x)
= a
lk
P (m
k
(x) + σ
k
(x)
2
∈ A
2
)
= a
lk
Z
A
2
1
σ
k
(x)
f
u − m
k
(x)
σ
k
(x)
du
= a
lk
b
k
(x) with b
k
(x) > 0
Further,
P
S
3
X
3
∈ {e} × A
2
| S
1
= s
l
, X
1
= x
=
K
X
j=1
a
lj
a
jk
Z
A
2
Z
R
1
σ
k
(y)
f
u − m
k
(y)
σ
k
(y)
1
σ
j
(x)
f
y − m
j
(x)
σ
j
(x)
dydu
=
K
X
j=1
a
lj
a
jk
b
jk
(x) with b
jk
(x) > 0
and doing so iteratively, we obtain
P
S
t+1
X
t+1
∈ {e} × A
2
| S
1
= s
l
, X
1
= x
=
K
X
j,··· ,j
t−1
a
lj
1
· · · a
j
t−1
k
b
j,··· ,j
t−1
(x)
which is strictly greater than 0 for some t because of the irreducibility of {Q
t
} and the fact that
b
j,··· ,j
t−1
(x) > 0.
• Analogously it c an easily be seen that {ζ
t
} is aperiodic.
• In the drift criterion of Theorem 15.0.1, (iii) mentioned previously app ears the notion of a
petite set. In our case, it can be shown that each compact set is indeed a s mall set and thus
a petite set, see for example, Bhattacharya and Lee [3] and Lee and Shin [5].
• So, to apply the drift criterion, we need to find a function g(ζ) > 1, β > 0 and M > 0 such
that
E
g(ζ
t
) | ζ
t−1
=
s
l
x
− g
s
l
x
g
s
l
x
≤ −β for kζ
t−1
k > M
Let
g(ζ
t
) = 1 + X
2
t
.
Then,
E
g(ζ
t
) | ζ
t−1
=
s
l
x
− g
s
l
x
g
s
l
x
=
P
k
(m
2
k
(x) + σ
2
k
(x))E(S
tk
| S
t−1
= s
l
) − x
2
1 + x
2
≤
P
k
(m
2
k
(x) + σ
2
k
(x))a
lk
x
2
− 1
4
and the conclusion is obtained by A.8.
Howeve r, in financial time series which are very often heavy-tailed, the existence of σ
2
= var(
t
) is
not necessarily guaranteed. Therefore, instead of A.7 and A.8 we assume
A. 9 The i.i.d. random variables
t
are such that E(|
t
|
α
) < ∞ for some 0 < α ≤ 1
A. 10
max
l∈{1,··· ,K}
lim sup
|x|−→∞
P
k
a
lk
(|m
k
(x)|
α
+ σ
α
k
(x)E|
t
|
α
)
|x|
α
< 1
with α as in A.9
Theorem 2 Under Assumptions A.1 to A.6, A.9 and A.10, {ζ
t
} is geometrically ergodic.
Proof: The only part of this proof which is not similar to the proof of Lemma 1 is the drift criterion.
Here we consider
g(ζ) = 1 + |X
t
|
α
.
Then,
E
g(ζ
t
) | ζ
t−1
=
s
l
x
− g
s
l
x
g
s
l
x
≤
P
k
(|m
k
(x)|
α
+ σ
α
k
(x)E|
t
|
α
)E(|S
tk
|
α
| S
t−1
= s
l
) − |x|
α
1 + |x|
α
≤
P
k
a
lk
(|m
k
(x)|
α
+ σ
α
k
(x)E|
t
|
α
)
|x|
α
− 1
and we conclude the proof by using A.10.
3 Geometric ergodicity for higher order CHARME pro-
cesses
We now follow a slightly different route to geometric ergodicity of CHARME processes. We first
state an auxiliary result that we are going to use for proving a condition for geometric ergodicity.
Lemma 1 Let φ, ψ be random variables with values in R
d
, C ⊂ R
d
a measurable set, g : R
d
−→ R
measurable and bounded on C satisfying g ≥ 1. If there exist constants 0 < r < 1, B > 0 such that
E(g(φ) | ψ = x) < rg(x), if x 6∈ C
E(g(φ) | ψ = x) < B, if x ∈ C
then, there exist β > 0, b < ∞ such that
E(g(φ) | ψ = x) − g(x) < −βg(x) + bI
C
(x).
5