Munich Personal RePEc Archive
Latent Markov models: a review of a
general framework for the analysis of
longitudinal data with covariates
Bartolucci, Francesco and Farcomeni, Alessio and Pennoni,
Fulvia
13 April 2012
Online at https://mpra.ub.uni-muenchen.de/39023/
MPRA Paper No. 39023, posted 25 May 2012 13:45 UTC
Latent Markov models: a review of a general framework for the
analysis of longitudinal data with covariates
Francesco Bartolucci
∗
, Alessio Farcomeni
†
, Fulvia Pennoni
‡
April 13, 2012
Abstract
We provide a comprehensive overview of latent Markov (LM) models for the analysis of
longitudinal data. The main assumption behind these models is that the response variables are
conditionally independent given a latent process which follows a first-order Markov chain. We
first illustrate the more general version of the LM model which includes individual covariates.
We then illustrate several constrained versions of the general LM model, which make the model
more parsimonious and allow us to consider and test hypotheses of interest. These constraints
may be put on the conditional distribution of the response variables given the latent process
(measurement model) or on the distribution of the latent process (latent model). For the
general version of the model we also illustrate in detail maximum likelihood estimation through
the Expec t at ion -M ax i mi zat i on algorithm, which may be efficiently implemented by recursions
known in the hidden Markov literature. We discuss about the model identifiability and we outline
methods for obtaining standard errors for the parameter estimates. We also illustrate me thods
for selecting the numbe r of st at es and f or path prediction. Finally, we illustrate Bayesian
estimation method. Models and related inference are illustrated by the description of relevant
socio-economic applications available in the literature.
Keywords: EM algorithm, Bayesian framework, Forward-Backward recursions, Hidden Markov
models, Me asu re ment errors, Panel data, Unobserved heterogeneity
∗
Department of Economics, Finance and Statistics, University of Perugia, Via A. Pascoli, 20, 06123 Perugia, Italy,
email: bart@stat.unipg.it
†
Department of Public Health and Infectious Diseases, Sapienza - University of Rome, Piazzale Aldo Moro, 5,
00185 Roma, Italy, email: alessio.farcomeni@uniroma1.it
‡
Department of Statistics, University of Milano-Bicocca, Via Bicocca degli Arcimboldi 8, 20126 Milano, Italy,
email: fulvia.pennoni@unimib.it
1
1 Introduction
In many applications involving longitudinal data, the interest is often focused on the evolution of a
latent characteristic of a group of i nd i v id u als over time, which is measured by one or more occasion-
specific response variabl es . This characteristic may correspond, for instance, to the quali ty-of-life
of subjects suffering from a cer t ai n disease, which i s indirectly assessed on the basis of responses to
a set of suitably desi gne d items that are repeatedly administered during a certain period of time.
Many models are proposed in the statistical lit er at u re for the analysis of longitudinal data; the
choice mainly depends on the context of application. For a revie w see, among others, Fitzmaurice
et al. (2009). In this literature, latent Markov ( LM ) models, on which the present paper is focused,
have a special role. These mo d el s assume the existence of a latent process which affects the
distribution of the response variable. The main assumption behind this approach is t hat the
response variables are conditionally independent given this latent process, which follows a Markov
chain with a fini t e number of states. The basic idea rel at ed to thi s assumption, which is referred to
as local independence, is that the latent process fully e xp l ai ns the observable be havior of a subject
together with possibly available covariates. Therefore, in studying LM models, it is important to
distinguish between two components: the measurement model, i.e. the condit ional distribution of
the response variables given the latent process, and the latent model, i.e. the distribution of the
latent process.
From many points of view, longitudinal data are similar to t i me -se ri e s data. The main dif-
ference is that t i me series are usually made of many repeated measures referr ed to a single unit,
whereas only few repeated measures ar e typically available in a longitudinal dataset, but for many
statistical units. However, inferential approaches arising within the time series literature cannot
be directly extended to models for longitudinal data. In the context of time-series data analysis ,
for example, the asymptot i c properties of an estimat or are studied assuming that the numb er of
repeated measures grows to infinity. On the contrary, in the context of longitudinal data analysis,
asymptotic properties are studied assuming that the sample size ten ds to infinity, while t h e num-
ber of occasions of observation is held fixed . In the statistic al and econometric literatures, one of
the most interesting model for the analysis of time-series data is the Hidden Markov (HM) model
(Baum and Petrie, 1966). The model is based on the same assumptions and estimation methods
of the LM model, but the structur e of the data they aim to analyze is different. However, t he
terminology is not univocal and the name HM model is sometimes adop t ed even for models applied
to the analysis of longi tu d in al data.
2
Many relevant results in the HM literature have been developed in the 60’s and 70’s; one of the
most relevant paper is due to Baum et al. (1970). Then, many advances have been developed in
connection with engineering, informatics, and bioin for mat i cs appl i cat i on s; consider, for instance,
the papers by Levinson et al. (1983) and Ghahramani and J or d an (1997). For general reviews see
the monographs of MacDonald and Zucchini (1997), Koski (2001), and Zucchini and MacDonald
(2009), whereas for rev i ew s and recent advances about the estimation methods see Bickel et al.
(1998), Capp´e et al. (2005), and Andersson and Ryd´en (2009).
In the following, we refer to the LM model based on a first-order M arkov chain, non-homogeneous
transition probabilities, and covariates as the general LM model. This model is presented in the
case of multivariate data when we observe more response variables at each occasion. For this LM
model we discuss i n detail maximum likelihood estimation through the Expectation-Maximization
(EM) algorithm (Baum et al., 1970; Demp st e r et al., 1977). We also briefly outline th e Bayesian es-
timation method even though we acknowledge that other estimation methods are available (Archer
and Titterington, 2009; Capp´e et al., 1989; K¨unsch, 2005; Turner, 2008). For the implementa-
tion of the EM algorithm, we illustrate suitable r ec ur si on s which allow us to strongly reduce the
computational effort.
The paper also focuses on some constrained versions of the general LM model. Constraints
have the ai m of making the model more parsimonious, easier to interpret, and in correspondence
with certain hyp ot h es es that may be interesting to test. These constraints may be posed on the
measurement model, that is on the conditional di s t ri b ut i on of the response variables given the
latent process, or on the latent model, that is on the distribution of the latent process. Regarding
the form er , we discuss in detail parametrizations which makes the latent states interpretable in
terms of ability or propensity levels. Regard i ng the latter, we outline several simplifications of the
transition matrix, mostly based on constraints of equali ty between c er t ain elements of this mat r i x
and/or on the constraint that some elements are equal to 0.
One of the main problems is how to test for the above restrictions. For this aim, we make use
of the likelihood rat io (LR) stat is t i c. It is important to note that, when constraints concern the
transition matrix, the null asymptotic distribution has not necessarily an asymptotic chi-squared
distribution, but a distri b ut i on of chi-bar-squared typ e (Bart ol u cci , 2006). We also illustrate an
extension of the model to relax local independence. It is made in two ways: including among
the covariates the lagged response variable, and allowing the re sponse variables at the same t im e
occasion to be dependent even conditionally on the corresponding latent variable. We discuss model
identifiability and we r ev i ew methods for obtaining standard errors for the model parameters. We
3
illustrate the main approaches for selecting the number of latent states and we also discuss the
problem of path prediction through the Viterbi algor i t hm (Viterbi, 1967; Juang and Rab i ne r,
1991).
The paper i s organized as follows. In Section 2 we illustrate the main cases of application of the
LM model and we briefly outline the model history. In Section 3 we outline the general LM mo de l
and constrained versions of the model based on parsimonious and interpretable parameteriz ati on s.
In Section 4 we show how to ob t ai n the manifest distribu ti on of the response variables and the
posterior distribution of the latent states by the Baum-Welch recursions (Baum et al., 1970; Welch,
2003). Then, we discuss maximum likelihood estimation for the general LM model based on the
EM algori t h m in Section 5, where we also d eal with model identifiability and standard errors for
the parameter estimates. Section 6 il l us t rat e s methods for the selection of t h e number of states
and path prediction. Sect i on 7 briefly outlines Bayesian estimation as an alternative to m axi mum
likelihood estimati on. Section 8 illustrates different types of LM model through various examples
involving longitudinal categorical data, summarizing the res ul t s from other papers. The paper ends
with a section where we draw main conclusions and d i scu ss further developments of t he present
framework.
2 Model motivations and historical review
The use of the LM models finds justification in different types of anal y si s that one may b e interested
to perform. We illustrate in the following three main cases where it is sensible to apply LM models.
1. Accounting f or measurement errors. In such cases th e adopted LM Model is seen as an exten-
sion of a Markov chain model Anderson (1951, 1954), which represents a fundamental model
for stochastic processes. As an example, consider the case in which the response variables
correspond to different items and indicators reflecting the quality-of-life of an elderly subject.
The i t e ms may concern the activity of daily living, whereas the indicators derive from certain
clinical measures. In t he prese nt framework, the dependence struc t ure between these vari-
ables is simplified by introducing one or few latent variables. The different configurations of
latent variables are interpreted as different levels of the quality-of-life, an individual charac-
teristic which is only indirectly observable through the response variables. This interpretation
is enforced by t h e assumption of local independence.
2. Accounting for unobserved heterogeneity between subjects. The latent variable may have the
role of accounting for the unobs er ved heterogeneity between subjects. This aspect is in
4