Panel Data Analysis — Advantages and Challenges
Cheng Hsiao
∗
Department of Economics
University of Southern California
Los Angeles, CA 90089-0253
and
Wang Yanan
Institute for Studies in Economics
Xiamen University, China
April 19, 2006
ABSTRACT
We explain the proliferation of panel data studies in terms of (i) data availability, (ii)
the more heightened capacity for modeling the complexity of human behavior than a single
cross-section or time series data can possibly allow, and (iii) challenging methodology.
Advantages and issues of panel data modeling are also discussed.
Keywords: Panel data; Longitudinal data; Unobserved heterogeneity; Random effects;
Fixed effects
∗
I would like to thank Irene C. Hsiao for helpful discussion and editorial assistance and
Kannika Damrongplasit for drawing the figures. Some of the arguments presented here
also appear in Hsiao (2005, 2006).
1. Introduction
Panel data or longitudinal data typically refer to data containing time series obser-
vations of a number of individuals. Therefore, observations in panel data involve at least
two dimensions; a cross-sectional dimension, indicated by subscript i, and a time series
dimension, indicated by subscript t. However, panel data could have a more complicated
clustering or hierarchical structure. For instance, variable y may be the measurement of
the level of air pollution at station in city j of country i at time t (e.g. Antweiler (2001),
Davis (1999)). For ease of exposition, I shall confine my presentation to a balanced panel
involving N cross-sectional units, i =1,...,N,overT time periods, t =1,...,T.
There is a proliferation of panel data studies, be it methodological or empirical. In
1986, when Hsiao’s (1986) first edition of Panel Data Analysis was published, there were
29 studies listing the key words: “panel data or longitudinal data”, according to Social
Sciences Citation index. By 2004, there were 687 and by 2005, there were 773. The growth
of applied studies and the methodological development of new econometric tools of panel
data have been simply phenomenal since the seminal paper of Balestra and Nerlove (1966).
There are at least three factors contributing to the geometric growth of panel data
studies. (i) data availability, (ii) greater capacity for modeling the complexity of human
behavior than a single cross-section or time series data, and (iii) challenging methodology.
In what follows, we shall briefly elaborate each of these one by one. However, it is impos-
sible to do justice to the vast literature on panel data. For further reference, see Arellano
(2003), Baltagi (2001), Hsiao (2003), Matyas and Sevester (1996), and Nerlove (2002), etc.
2. Data Availability
The collection of panel data is obviously much more costly than the collection of cross-
sectional or time series data. However, panel data have become widely available in both
developed and developing countries.
The two most prominent panel data sets in the US are the National Longitudinal
Surveys of Labor Market Experience (NLS) and the University of Michigan’s Panel Study
1
of Income Dynamics (PSID). The NLS began in the mid 1960’s. It contains five separate
annual surveys covering distinct segments of the labor force with different spans: men
whose ages were 45 to 59 in 1966, young men 14 to 24 in 1966, women 30 to 44 in 1967,
young women 14 to 24 in 1968, and youth of both sexes 14 to 21 in 1979. In 1986, the NLS
expanded to include annual surveys of the children born to women who participated in
the National Longitudinal Survey of Youth 1979. The list of variables surveyed is running
into the thousands, with emphasis on the supply side of market.
The PSID began with collection of annual economic information from a representative
national sample of about 6,000 families and 15,000 individuals in 1968 and has continued
to the present. The data set contains over 5,000 variables (Becketti, Gould, Lillard and
Welch (1988)). In addition to the NLS and PSID data sets, there are many other panel
data sets that could be of interest to economists, see Juster (2000).
In Europe, many countries have their annual national or more frequent surveys such
as the Netherlands Socio-Economic Panel (SEP), the German Social Economics Panel
(GSOEP), the Luxembourg Social Panel (PSELL), the British Household Panel Survey
(BHS), etc. Starting in 1994, the National Data Collection Units (NDUS) of the Statistical
Office of the European Committees have been coordinating and linking existing national
panels with centrally designed multi-purpose annual longitudinal surveys. The European
Community Household Panel (ECHP) are published in Eurostat’s reference data base New
Cronos in three domains: health, housing, and income and living conditions.
Panel data have also become increasingly available in developing countries. In these
countries, there may not have been a long tradition of statistical collection. It is of special
importance to obtain original survey data to answer many significant and important ques-
tions. Many international agencies have sponsored and helped to design panel surveys. For
instance, the Dutch non-government organization (NGO), ICS, Africa, collaborated with
the Kenya Ministry of Health to carry out a Primary School Deworming Project (PDSP).
The project took place in Busia district, a poor and densely-settled farming region in
2
western Kenya. The 75 project schools include nearly all rural primary schools in this
area, with over 30,000 enrolled pupils between the ages of six to eighteen from 1998-2001.
Another example is the Development Research Institute of the Research Center for Rural
Development of the State Council of China, in collaboration with the World Bank, which
undertook an annual survey of 200 large Chinese township and village enterprises from
1984 to 1990.
3. Advan tages of Panel Data
Panel data, by blending the inter-individual differences and intra-individual dynamics
have several advantages over cross-sectional or time-series data:
(i) More accurate inference of model parameters. Panel data usually contain more
degrees of freedom and less multicollinearity than cross-sectional data which may
be viewed as a panel with T = 1, or time series data which is a panel with N =1,
hence improving the efficiency of econometric estimates (e.g. Hsiao, Mountain
and Ho-Illman (1995).
(ii) Greater capacity for capturing the complexity of human behavior than a single
cross-section or time series data. These include:
(ii.a) Constructing and testing more complicated behavioral hypotheses. For in-
stance, consider the example of Ben-Porath (1973) that a cross-sectional
sample of married women was found to have an average yearly labor-force
participation rate of 50 percent. These could be the outcome of random
draws from a homogeneous population or could be draws from heteroge-
neous populations in which 50% were from the population who always work
and 50% never work. If the sample was from the former, each woman would
be expected to spend half of her married life in the labor force and half out of
the labor force. The job turnover rate would be expected to be frequent and
3
