Applied Mathematical Sciences, Vol. 4, 2010, no. 50, 2485 - 2495
Wavelet Neural Networks for Nonlinear
Time Series Analysis
K. K. Minu, M. C. Lineesh and C. Jessy John
Department of Mathematics
National Institute of Technology Calicut
NIT Campus P O - 673 601, India
lineesh@nitc.ac.in
Abstract
A wavelet network is an important tool for analyzing time series
especially when it is nonlinear and non-stationary. It takes advantage
of high resolution of wavelets and learning and feed forward nature of
Neural Networks. Wavelets are a class of functions such that multiple
resolution nature of wavelets provides a natural frame work for the anal-
ysis of time series. The power of this network to approximate functions
from given input-output data is proved and it has utilized the local-
ization property of a wavelet to focus on local properties. Guaranteed
upper bounds on the accuracy of approximation is established. Here
we are analyzing the time series of number of terrorist attacks in the
world measured on monthly basis during the period February 1968 to
January 2007 for establishing the superiority of this method over other
existing methods. The simulation results show that the model is capable
of producing a reasonable accuracy within s everal steps.
Mathematics Subject Classification: 37M10, 65T60, 92B20
Keywords: Non Stationary-nonlinear Time Series; Wavelet Networks;
Scaling Coefficients; Wavelet Coefficients; Terrorist Attack Time Series; GARCH
Model
1 Introduction
One of the greatest threats modern nations encounter is the terrorist attacks
of various groups. It is usually unpredictable and unexpected. It may be na-
tional or trans national. The cause of it may vary from local to international
but every nation faces it in one form or other. Prediction in time series is to
2486 K. K. Minu, M. C. Lineesh and C. Jessy John
model an existing data series in order to predict unknown future values ac-
curately[14]. Linear models do not adequately represent nonlinear series and
there are no single powerful tool for the analysis of nonlinear time series[12].
Wavelets can be considered as functions generated from a basic function by
translations and dilations. The basic function is called the mother wavelet.
Wavelet transforms involve representing a general function in terms of sim-
ple fixed building blocks at different resolutions[9]. They are generated from
a single fixed function by changing translation and scale. The continuous
wavelet transform considers a family {Ψ
t
(a, b)} and satisfies the admissibility
condition expressed in terms of its Fourier transform. For Discrete Wavelet
Transform(DWT), scale and translation parameters are chosen in such a way
that at level m the wavelets are given by Ψ
m,n
(t) = 2
−
m
2
Ψ(2
−m
t − n) where m
and n are integers. Orthonormal basis and multiresolution analysis represents
a function at various levels of resolution by projecting the function into an
increasing sequence of subspaces.
Authors like Wei, W. W. W [16] concentrated on the analysis and forecasting
of stationary time series. Many of these studies are based on the statistical
concepts like correlation and regression analysis. On the other hand several
researchers have been looking for better ways to design neural networks[14].
Hence it is of great importance to analyze the relationship between neural net-
works, approximation theory and functional analysis. In functional analysis
any continuous function can be represented as a weighted sum of orthogonal
basis functions. Such expansions can be easily represented as neural networks
which can be designed for the desired error rate using the properties of or-
thonormal expansions. In order to take full advantage of orthonormality of
basis functions, and localized learning, we need a set of basis functions which
are local and orthogonal[15]. Wavelets are functions with these features. They
have generated a tremendous interest in both theoretical and applied areas over
the past few years. Wavelet networks are a class of neural networks that em-
ploy wavelets as activation functions[15]. These have been recently researched
as an alternative approach to the neural networks with sigmoidal activation.
In recent years, wavelet transformation is proposed for the analysis of time se-
ries. Researchers like Priestley, 1996; Morettin, 1997; Gao, 1997; Percival and
Walden, 1999 focused on periodogram analysis of a time series. Bjorn, 1995;
Soltani(2000); Renaud, 2003 are some groups studying time series prediction
using wavelets.
2 Wavelets
A wavelet is a real or complex valued function Ψ(.) satisfying the following
conditions;
R
∞
−∞
Ψ(u)du = 0 and
R
∞
−∞
|Ψ
2
(u)|du = 1.
Wavelet neural networks 2487
There are two functions in wavelet transform namely the scale function(father
wavelet) and the mother wavelet. These two functions give a function family
that can be used for reconstructing a signal. This is the basic idea of Mul-
tiresolution Analysis(MRA). Some commonly used wavelet families are Haar
Wavelet, Meyer Wavelet, Daubechies Wavelet, Mexican Hat Wavelet, Coiflet
Wavelet and Last Assymetric[13].
3 Neural Networks and Time Series Analysis
Feedforward neural networks are composed of layers of neurons in which the
input layer of neurons is connected to the output layer of neurons through one
or more layers of intermediate neurons. The training process of the neural
network involves adjusting the weights till a desired input/output relationship
is obtained. The majority of adaptation learning algorithms are based on the
Widrow-Hoff back-propagation algorithm. Feed forward neural networks have
been proposed for analyzing a given time series.
The standard neural network method of performing time series prediction is
to induce the function using any feedforward function approximating neural
network architecture, such as, a standard MLP, an RBF architecture, or a
Cascade correlation model, using a set of N-tuples as inputs and a single output
as the target value of the network. This method is often called the sliding
window technique as the N-tuple input slides over the full training set.
The neural network forecaster can be described as follows
z
k+1
= NN(z
k
, z
k−1
, ..., z
k−d
, e
k
, e
k−1
, ..., e
k−d
); (1)
where z is either original observations or processed data, and {e
k
, e
k−1
, ..., e
k−d
}
are residuals.
4 Discrete wavelet transform - Decomposition
and Reconstruction
The discrete wavelet transform of a given time series {X
t+1
: t = 0, 1, 2, ..., N − 1}
is defined by
Ψ(m, n) = 2
−
m
2
N−1
X
k=0
X
k
Ψ(2
−m
k − n), (2)
where Ψ(k) need not be a sampled version of Ψ(t).
It is computationally impossible to analyze a signal using all wavelet coeffi-
cients, so one may wonder if it is sufficient to pick a discrete subset of the upper
half plane to be able to reconstruct a signal from the corresponding wavelet
2488 K. K. Minu, M. C. Lineesh and C. Jessy John
coefficients. One such system is the affine system for some real parameters
a > 1, b > 0. The corresponding discrete subset of the half plane consists of
all the points {a
m
, na
m
b} with integers m, n ∈ Z. The corresponding baby
wavelets are now given as
Ψ
m,n
(t) = a
−m/2
Ψ(a
−m
t − nb). (3)
A sufficient condition for the reconstruction of any signal X of finite energy
by the formula
X(t) =
X
m∈Z
X
n∈Z
< X, Ψ
m,n
> .Ψ
m,n
(t) (4)
is that the functions {Ψ
m,n
: m, n ∈ Z} form a tight frame of L
2
(R).
5 Wavelet Networks
Representing a continuous function by a weighted sum of basis functions can
be made unique if the basis functions are orthonormal. It was proved that
[3] neural networks can be designed to represent such expansions with desired
degree of accuracy. Wavelets have many desired properties combined together
like compact support, orthogonality, localization in time and frequency and
fast algorithms. Neural Networks are used in function approximation, pattern
classification and in data mining but they could not characterize local fea-
tures like jumps in values well. The local features may be existing in time or
frequency. The improvement in their characterization will result in data com-
pression and subsequent modification of classification tools. Wavelet networks
are a class of neural networks that employ wavelets as activation functions[15].
5.1 Wavelet Neural Network (WNN) Model
for Prediction
The concept of time series forecasting by using wavelet is nothing but forecast-
ing by using the data which is preprocessed through the wavelet transform,
especially through DWT. By the presence of multiscale decomposition like
wavelet, the advantage is automatically separating the data components such
as trend component and irregular component in the data. There by forecasting
of stationary or nonstationary data.
Suppose we want to predict X
t+1
where the data {X
j
: j = 1, 2, 3, ..., t} are
given. The basic idea of WNN model is using preprocessed data that are
obtained through the wavelet decomposition of X
t
. Renaud et. al (2003) in-
troduce Multilayer Perceptron (MLP) NN architecture(Feed Foreward Neural
Network-FFNN) to process the wavelet coefficients. The FFNN architecture
Wavelet neural networks 2489
that is used for time series prediction consists of one hidden layer with P
neurons defined as;
ˆ
X
N+1
=
P
X
p=1
ˆ
b
p
g[
J
X
j=1
A
j
X
k=1
ˆa
j,k,p
W
j,N−2
j
(k−1)
+
A
J+1
X
k=1
a
J+1,k,p
v
j,N−2
j
(k−1)
], (5)
where j is the number of levels {j = 1, 2, 3, ..., J}, A
j
orders of MAR model
(k = 1, 2, 3, ..., A
j
); w
j,t
is the wavelet coefficient value, v
j,t
is the scale coef-
ficient value and a
j,k
is the MAR coefficient value. Here g is an activation
function in hidden layer of WNN.
6 Analysis of a Real World Nonlinear-
nonstationary Time Series
In this paper we have considered the number of terrorist attacks in the world
which is measured on monthly basis starting from February 1968 to January
2007. The plot of the data is given in figure 1.
Figure 1: plot of terrorist attacks data
The plot of the autocorrelations of the terrorist attack data is given in
figure 2. The plot shows that this is a nonstationary-nonlinear time series.