NBER WORKING PAPER SERIES
ROBUSTNESS OF PRODUCTIVITY ESTIMATES
Johannes Van Biesebroeck
Working Paper 10303
http://www.nber.org/papers/w10303
NATIONAL BUREAU OF ECONOMIC RESEARCH
1050 Massachusetts Avenue
Cambridge, MA 02138
February 2004
I would like to thank Mel Fuss, Robert Gagné, Marc Melitz, Ariel Pakes, Peter Reiss, Chad Syverson, Frank
Wolak, and participants at the NBER 2002 Summer Institute and SITE 2003 for comments. All remaining
errors are my own. Financial support from the Connaught Fund is gratefully acknowledged. The views
expressed herein are those of the authors and not necessarily those of the National Bureau of Economic
Research.
©2004 by Johannes Van Biesebroeck. All rights reserved. Short sections of text, not to exceed two
paragraphs, may be quoted without explicit permission provided that full credit, including © notice, is given
to the source.
Robustness of Productivity Estimates
Johannes Van Biesebroeck
NBER Working Paper No. 10303
February 2004
JEL No. D24, C13, C14, C15, C43
ABSTRACT
Researchers interested in estimating productivity can choose from an array of methodologies, each
with its strengths and weaknesses. Many methodologies are not very robust to measurement error
in inputs. This is particularly troublesome, because fundamentally the objective of productivity
measurement is to identify output differences that cannot be explained by input differences. Two
other sources of error are misspecifications in the deterministic portion of the production technology
and erroneous assumptions on the evolution of unobserved productivity. Techniques to control for
the endogeneity of productivity in the firm's input choice decision risk exacerbating these problems.
I compare the robustness of five widely used techniques: (a) index numbers, (b) data envelopment
analysis, and three parametric methods: (c) instrumental variables estimation, (d) stochastic
frontiers, and (e) semiparametric estimation. The sensitivity of each method to a variety of
measurement and specification errors is evaluated using Monte Carlo simulations.
Johannes Van Biesebroeck
Department of Economics
University of Toronto
150 St. George Street
Toronto, ON M5S 3G7
CANADA
and NBER
jovb@chass.utoronto.ca
1 Motivation
Accurate measurement is at the heart of productivity comparisons. Fundamentally, the
objective is to identify output differences that c annot be explained by input differences.
To perform this exercise, one needs to observe inputs and outputs accurately and control
for the input substitution that the production technology allows. Problems can arise from
misspecifications in the deterministic or stochastic portion of the production technology and
from measurement errors in the data.
Firms use different input combinations to produce one unit of output because their
technology differs, which I label productivity differences, or because they face different factor
price, which leads firms to pick different points on the production frontier.
1
The extent to
which one input can be substituted for another is determined by the shape and position of
the production function—or any other representation of technology—and is naturally not
observable. Methodologies to estimate productivity differ by the mix of statistical techniques
and economic assumptions they employ to control f or input substitution. Misspecific ations
in the deterministic part of the production function or in the statistical model underlying the
evolution of unobserved productivity will have repe rcussions on the productivity e stimates.
Mismeasurement can result, among other things, from unobserved quality or price
differences, aggregation problems, recall errors in surveys, or incompatibilities in reference
period for output and inputs. The effect on productivity estimates obviously depend on
the estimation method. For example, Griliches and Hausman (1986) argue that while first-
differencing is useful to control for unobserved firm-specific effects, identification based on
thinner slices of the data are more vulnerable to measurement errors. Solutions exist for
dealing with well-defined forms of measurement error, but they are rarely used in practice.
One of the goals in this paper is to verify how se nsitive different me thods for productivity
measurement are to different forms of measurement error.
I evaluate the robustness to misspecific ation and measurement errors for five popular
1
Some authors have argued that some of the output shortfall relative to the best practice frontier is the
result of inefficiency. I still classify such shortfall as productivity differences, to remain consistent with a
profit maximizing model of the firm. Lower output might be caused by differences in production technology,
unmeasured inputs, or quality differences in outputs. See Stigler (1976) for a more elaborate motivation.
2
methodologies. The first two methods, index numbers and data envelopment analysis, are
very flexible in the s pecification of technology, but do not allow for unobservables, making
the effect of measurement error completely unpredictable. The three parametric methods
calculate productivity from an estimated production function. In the simplest linear regres-
sion model, measurement error in the dependent variable has no effect on the consistency of
least squares estimates, while errors in the independent variables biases coefficient estimates
downwards. For most production function estimators, the effects are not so straightforward,
because more complicated estimators are devised to deal with the simultaneity of produc-
tivity and input choice. Moreover, the principal interest is in the residual of the production
function, which is always affected. I evaluate the robustness of both productivity level and
growth estimates using simulated data.
2
In the next section, I start with some background on productivity measurement and,
subsequently, I introduce the different methodologies. An attempt is made to present the
general idea of each methodology in a consistent framework and convey the distinctive fea-
tures as briefly as possible. Links to the literature for more extensive information and
discussion are provided. Section 3 describes the data generation process, starting from the
input choices of a profit maximizing representative firm. For each set of assumptions on the
evolution of productivity that have been considered in the literature, I solve analytically or
numerically for the optimal investment policy. In Section 4, the sensitivity of the different
estimation methodologies to variations of three elements of the data generating process is
evaluated. First, different assumptions are used to model the unobserved productivity term.
Second, measurement error of varying siz e is added to output and inputs. Third, the returns
to scale of the production technology is varied. Lessons to take away from these exercises
are summarized at the end.
2
For a related study that uses manufacturing data from Colombia to compare the different methodologies,
see Van Biesebroeck (2003b).
3
2 Measuring Productivity
In plain English, one firm is more productive than another if it is able to produce the same
outputs with less inputs or if it produces more outputs from the same amount of inputs.
Similarly, a firm has experienced positive productivity growth if outputs have increased more
than inputs or inputs have decreased more than outputs. The comparison becomes more
interesting if one firm (or the same firm in one of the comparison periods) uses more of one
input, while the other relies more on a second input. In that case, it becomes necessary
to specify a transformation function that links inputs to outputs. Since a firm’s input
substitution possibilities are determined by the technology it employs, each productivity
measure is only defined with respect to that specific production technology.
Measuring productivity necessarily involves decomposing differences in the input-
output combinations into shifts along a production frontier and shifts of the frontier itself.
In Figure 1, two production plans, P
0
and P
1
, are compared in input space and the frontier
is represented by the unit isoquant. Part of the difference, from P
0
to 1, is a shift along
the frontier, exploiting the input substitution the technology allows . The remainder of the
difference, from 1 to P
1
, is an actual shift of the frontier, which is counted as technical change
or productivity growth. In this example, an intuitive measure of P
0
’s productivity relative
to P
1
is
0P
1
01
.
If the shape of the unit isoquant in Figure 1 is not known, it can be estimated paramet-
rically if one is willing to make functional form assumptions. Simultaneity of productivity
and input choice is the main econometric issue. I discuss three different estimators that
control for it in Section 2.3.
Another approach is to rely on index number theory, which is discussed in Section
2.2. If the first order conditions for input choices hold, the factor price ratio will equal the
slope of the input isoquant, which determines input substitution possibilities. Taking the
average of the ratio for both production plans that are compared, it is possible to control
for input differences without having to estimate anything. Figure 2 compares the same two
production plans as before. The reference production plan (P
0
) uses more labor (less capital)
which will be accounted for in proportion to the average labor share (capital s hare) in costs.
4
