Theory, Policy, Institutions: Papers from the
Carnegie-Rochester Conference
Series
on
Public Policy
Karl Brunner and Alan Meltzer (eds.)
©Elsevier Science
Publishers B.V. (North-Holland), 1983
ECONOMETRIC POLICY EVALUATION: A CRITIQUE
Robert E. Lucas, Jr.
1.
Introduction
257
The fact that nominal prices and wages tend
to
rise more rapidly
at
the peak
of
the business cycle than they
do
in the trough has been well recognized from the
time when the cycle was first perceived as a distinct phenomenon. The inference
that permanent inflation will therefore induce a permanent economic high
is
no
doubt
equally ancient, yet it
is
only recently
that
this notion has undergone the
mysterious transformation from obvious fallacy to cornerstone
of
the theory
of
economic policy.
This transformation did not arise from new developments in economic theo-
ry.
On
the
contrary, as soon as Phelps and others made the first serious attempts
to rationalize the apparent trade-off in modern theoretical terms, the zero-degree
homogeneity
of
demand and supply functions was re-discovered in this new con-
text (as Friedman predicted it would be) and re-named the "natural rate hypothe-
sis"}
It
arose, instead, from the younger tradition
of
the econometric forecasting
models, and from the commitment on
the
part
of
a large fraction
of
economists
to
the
use
of
these models for quantitative policy evaluation. These models have
implied the existence
of
long-run unemployment-inflation trade-offs ever since
the
"wage-price sectors" were first incorporated and they promise
to
do
so in the
future although
the
"terms"
of
the
trade-off continue to shift.2
This clear-cut conflict between two rightly respected traditions - theoreti-
cal and econometric - caught those
of
us who viewed the two
as
harmoniously
complementary quite by surprise. At first, it seemed that the conflict might be
resolved by somewhat fancier econometric footwork.
On the theoretical level,
one hears talk
of
a "disequilibrium dynamics" which will somehow make money
illusion respectable while going beyond
the
sterility
of
~
= k(p_pe). Without un-
derestimating the ingenuity
of
either econometricians or theorists, it seems to me
appropriate to entertain the possibility
that
reconciliation along
both
of
these
lines will fail, and
that
one
of
these traditions
is
fundamentally in error.
The thesis
of
this essay
is
that
it
is
the
econometric tradition, or more pre-
1See Phelps et aI. [311,Phelps'earlier [301 and Friedman
(13).
2
The
earliest-;';;e-price sector embodying the
"trade-off'
is
(as far
as
I know) in the 1955 version
of
the
Klein.Goldberger model
(19).
It
has persisted, with minimal conceptual change,
into
all current generation
forecasting models. The subsequent shift
of
the
"trade-off'
relationship to center stage in policy discussions
appears due primarily
to
Phillips (32) and Samuelson and Solow [331.
Reprinted from The Phillips Curve and Labor Markets, Carnegie-Rochester Conference Series
on Public Policy, Volume 1 (1976), pp. 19-46.
258
R.E.
Lucas.
Jr.
cisely, the
"theory
.of
economic policy" based on this tradition, which
is
in need
of
major revision. More particularly, I shall argue that the features which lead to
success in short-term forecasting are unrelated to quantitative policy evaluation,
that the major econometric models are (well) designed to perform the former
task only, and that simulations using these models can, in principle,
provide.!!Q
useful information
as
to
the actual consequences
of
alternative economic policies.
These contentions will be based
not
on deviations between estimated and
"true"
structure prior
to
a policy change
but
on the deviations between the prior
"true"
structure and the
"true"
structure prevailing afterwards.
Before turning to details, I should like
to
advance two disclaimers. First,as
is
true with any technically difficult and novel area
of
science, econometric model
building
is
subject
to
a great deal
of
ill-informed and casual criticism. Thus mod-
els are condemned
as
being
"too
big" (with equal insight, I suppose one could
fault smaller models for being
"too
little"),
too
messy,
too
simplistic
(that
is,
not
messy enough), and, the ultimate blow, inferior to "naive" models. Surely the in-
creasing sophistication
of
the "naive" alternatives to the major forecasting models
is
the highest
of
tributes to the remarkable success
of
the latter. I hope I can suc-
ceed in disassociating the criticism which follows from any denial
of
the very im-
portant
advances in forecasting ability recorded by the econometric models, and
of
the promise they offer for advancement
of
comparable importance in the fu-
ture.
One may well define a critique
as
a paper which does
not
fully engage the
vanity
of
its author. In this spirit,
let
me offer a second disclaimer. There
is
little
in this essay which
is
not
implicit (and perhaps to more discerning readers, expli-
cit) in Friedman
[11],
Muth [29] and, still earlier, in Knight
[21].
For
that mat-
ter, the criticisms I shall raise against currently popular applications
of
econome-
tric theory have, for the most part, been anticipated
by
the major original contri-
butors to that theory.3 Nevertheless, the case for sustained inflation, based en-
tirely on econometric simulations,
is
attended now with a seriousness
it
has
not
commanded for many decades_
It
may, therefore, be worthwhile
to
attempt
to
trace this case back
to
its foundation, and then to examine again the scientific ba-
sis
of
this foundation itself.
2. The Theory
of
Economic Policy
Virtually all quantitative macro-economic policy discussions today are con-
ducted within a theoretical framework which I shall call
"the
theory
of
economic
3
See
in particular Marschak's discussion in
[lSi
(helpfuUy recaUed
to
me by T. D.
WaUace)
and Tinbergen's
in [361. especiaUy his discussion
of
"qualitative policy" in ch.
5,
pp. 149-185.
Econometric policy evaluation
259
policy",(following Tinbergen
[35]).
The essentials
of
this framework are so wide-
ly known and subscribed
to
that
it may be superfluous to devote space to their re-
view.
On the
other
hand, since the main theme
of
this paper
is
the
inadequacy
of
this framework,
it
is
probably best to have an explicit version before us.
One describes the economy in a time period t by a vector y t
of
state varia-
bles, a vector x
t
of
exogeneous forcing variables, and a vector
ft
of
independent
(through time), identically distributed random shocks. The motion
of
the econo-
my
is
determined
by
a difference
equation
the distribution
of
ft,
and a description
of
the temporal behavior
of
the forcing
variables, x
t
.
The function f
is
taken to be
fIXed
but
not
directly known; the
task
of
empiricists
is
then to estimate
f.
For
practical purposes, one usually
thinks
of
estimating the values
of
a fixed parameter vector
e,
with
f(y,x,f)
==
F(y,x,e,f)
and F being specified in advance.
Mathematically, the sequence {Xt}
of
forcing vectors
is
regarded
as
being
"arbitrary"
(that
is,
it
is
not
characterized stochastically). Since the past
Xt
val-
ues are observed, this causes no difficulty in estimating
e,
and in fact simplifies
the theoretical estimation problem slightly.
For
forecasting, one
is
obliged
to
in-
sert forecasted
Xt
values
into
F.
With knowledge
of
the function F and
e,
policy evaluation
is
a straight-
forward matter. A policy
is
viewed as a specification
of
present and future values
of
some components
of
{Xt}. With
the
other
components somehow specified,
the stochastic behavior
of
{y
t,Xt,f t } from the present on
is
specified, and func-
tionaIs defined on this sequence are well-defined random variables, whose mo-
ments may be calculated theoretically
or
obtained
by
numerical simulation.
Sometimes, for example, one wishes
to
examine the mean value
of
a hypothetical
"social objective function", such as
~
i3
t
u(y
t,Xt,ft)
t = 0
under alternative policies. More usually, one
is
interested in the "operating char-
acteristics"
of
the system
under
alternative policies. Thus, in this standard con-
text, a
"long-run Phillips curve" is simply a plot
of
average inflation - unemploy-
260
R.E. Lucas,
Jr.
ment pairs under a range
of
hypothetical policies.
4
Since one cannot treat e
as
known in practice, the actual problem
of
policy evaluation
is
somewhat more complicated. The fact
that
e
is
esti-
mated from past sample values affects the above moment calculations for small
samples;
it
also makes policies which promise to sharpen estimates
of
e relatively
more attractive. These considerations complicate without, I think, essentially
al-
tering the theory
of
economic policy
as
sketched above.
Two features
of
this theoretical framework deserve special comment. The
first
is
the uneasy relationship between this theory
of
economic policy and tradi-
tional economic theory. The components
of
the vector-valued function
Fare
behavioral relationships - demand functions; the role
of
theory may thus be
viewed
as
suggesting forms for F, or in Samuelson's terms, distributing zeros
throughout the Jacobian
of
F. This role for theory
is
decidedly secondary: mi-
croeconomics shows surprising power
to
rationalize individual econometric rela-
tionships in a variety
of
ways. More significantly, this micro-economic role for
theory abdicates the task
of
describing the aggregate behavior
of
the system en-
tirely to the econometrician. Theorists suggest forms for consumption, invest-
ment, price and wage setting functions separately; these suggestions,
if
useful, in-
fluence individual components
of
F. The aggregate behavior
of
the system then
is
whatever it
is.
S
Surely this point
of
view (though I
doubt
if
many would now
endorse
it
in so bald a form) accounts for the demise
of
traditional "business cy-
cle theory" and the widespread acceptance
of
a Phillips
"trade-off'
in
the absence
of'!!!!y aggregative theoretical model embodying such a relationship.
Secondly, one must emphasize the intimate link between short-term
fore-
casting and long-term simulations within this standard framework. The variance
of
short-term forecasts tends
to
zero with the variance
of
et;
as
the latter becomes
small, so also does the variance
of
estimated behavior
of
{y t } conditional on hy-
pothetical policies { X
t
}.
Thus forecasting accuracy in the short-run implies relia-
bility
of
long-term policy evaluation.
3. Adaptive Forecasting
There are many signs
that
practicing econometricians pay little more than
lip-service to the theory outlined in the preceding section. The most striking
is
the indifference
of
econometric forecasters to data series prior
to
1947. Within
the theory
of
economic policy, more observations always sharpen parameter esti-
4
See
, for example, de
Menil
and Enzler
(6),
Hirsch (16) and Hymans
(17).
SThe ill·fated Brookings model project was probably the Ultimate expression
of
this view.
Econometric policy evaluation
261
mates and forecasts, and observations on
"extreme"
Xt
values particularly so;
yet
even the readily available annual series from 1929-1946 are rarely used as a
check on the post-war fits.
A second sign
is
the frequent and frequently
important
refitting
of
econome-
tric relationships. The revisions
of
the wage-price sector now in progress are a
good example.
6
The continuously improving precision
of
the estimates
of
e
within the fixed structure
F,
predicted
by
the theory, does
not
seem
to
be occur-
ring in practice.
Finally, and most suggestively,
is
the practice
of
using patterns in recent re-
siduals
to
revise intercept estimates for forecasting purposes.
For
example,
if
a
"run"
of
positive residuals (predicted less actual) arises in an equation in recent
periods, one revises
the
estimated intercept downward
by
their average amount.
This practice accounts, for example,
for
the
superiority
of
the actual Wharton
forecasts
as
compared
to
forecasts based on the published version
of
the
model.
7
It
should be emphasized
that
recounting these discrepancies between theory
and practice is
not
to
be taken
as
criticism
of
econometric forecasters. Certainly
if
new observations are
better
accounted for by new
or
modified equations,
it
would be foolish to continue to forecast using the old relationships.
The
point
is
simply
that,
econometrics textbooks
not
withstanding, current forecasting prac-
tice
is
not
conducted within the framework
of
the theory
of
economic policy, and
the unquestioned success
of
the forecasters should
not
be construed as evidence
for the soundness
or
reliability
of
the structure proposed in
that
theory.
An alternative structure to
that
underlying the theory
of
economic policy
has recently been proposed (in [3] and
[5])
by
Cooley and Prescott. The struc-
ture
is
of
interest in the present
context,
since optimal forecasting within
it
shares
many features with current forecasting practice
as
just
described. Instead
of
treating the parameter vector e as fixed, Cooley and Prescott view
it
as
a random
variable following the random walk
where
h)t}
is a sequence
of
independent, identically distributed random variables.
Maximum likelihood forecasting
under
this alternative framework ("adap-
tive regression") resembles "exponential smoothing" on
the
observations, with
observations in the distant past receiving a small
"weight"
- very much
as
in
6
See
, for example, Gonion
[14).
7 A good account
of
this and
other
aspects
of
foreca<ting in
theOlY
and practice is provided by Klein
(20).
A
fuUer
treatment
is
available in Evans and Klein
[9).