DO ECONOMIC RESTRICTIONS IMPROVE FORECASTS?

Authors: Beth Murphy, Research Assistant, Department of Agricultural

and Resource Economics, North Carolina State University.

Bailey Norwood, Assistant Professor, Department of Agricultural

Economics, Oklahoma State University.

Michael Wohlgenant, William Neal Reynolds Distinguished Professor,

Department of Agricultural and Resource Economics, North Carolina

State University.

Paper prepared for presentation at the American Agricultural Economics Association Annual

Meeting, Montreal, Canada, July 27-30, 2003

Copyright 2003 by [authors(s)]. All rights reserved. Readers may make verbatim copies of this

document for non-commercial purposes by any means, provided that this copyright notice

appears on all such copies.

The authors would like to thank Terry Kastens for providing the data, computer code, and notes

necessary to replicate his results.

1

DO ECONOMIC RESTRICTIONS IMPROVE FORECASTS?

Using several popular demand systems in conjunction with food consumption data,

Kastens and Brester (KB) show that theory-constrained demand systems forecast better out-of-

sample (hereafter forecast) than their unrestrained counterparts. While at first this seems to

provide some justification for imposing theoretical constraints, it does not address the question of

whether the forecast benefit derives from economic theory or higher degrees of freedom.

Parameter restrictions serve to enhance degrees of freedom regardless of whether the

restrictions are derived from theory or not. Because models with greater degrees of freedom

forecast better, in this paper we ask whether the theory-constrained models in KB forecast better

because the restrictions are “true” or because their degrees of freedom are higher.

1

We wish to

separate the contribution of forecast improvements due to economic theory from that of higher

degrees of freedom.

We use the data from the KB study to re-estimate their models with arbitrary restrictions.

These arbitrary restrictions are not derived from theory, but they increase the degrees of freedom

by an identical amount as the economic restrictions. Results indicate that arbitrary restrictions,

due to more degrees of freedom, do improve forecasts relative to no restrictions. However,

economic restrictions improve forecasts even more, suggesting that there is valuable information

contained in economic theory, and that economic theory has an important role in forecasting.

THE VALUE OF PARAMETER RESTRICTIONS

It may seem strange that theoretical restrictions would be rejected in-sample, and then

reduce forecast errors out-of-sample.

2

Why would theoretical restrictions appear informative out-

of-sample but not in-sample?

3

One reason, based on the concept of sample and non-sample

information, is that economic restrictions improve forecasts because economic theory is

informative. The other explanation, based on degrees of freedom issues, is that any restriction

might improve forecasts, regardless of whether the restriction is true or not.

2

Sample information refers to a set of observations. Theoretical restrictions are a form of

non-sample information. They represent information researchers believe to be true but may not

be reflected adequately in a random sample. The three most popular restrictions; symmetry,

homogeneity, and adding-up are derived from theory of the representative consumer. Their

derivation rests on several assumptions that may be too restrictive. Assumptions commonly made

are that all consumers possess and maximize the same utility function, the parameters of that

function are time-invariant, all consumers face identical real prices, and either all specified goods

must be exhaustive or a subset must be separable (Deaton and Mullbauer).

But restrictions derived from economic theory need not hold perfectly to have value.

Economic restrictions convey information even if none of the above assumptions hold. If beef is

a strong substitute for pork, pork should be a strong substitute for beef. The symmetry condition

ensures this is the case.

4

The point is that theoretical restrictions may convey much information

we know about consumers, even if their parametric representation is not perfectly accurate.

Suppose we wish to estimate a parameter vector β for use in forecasting, and the

prediction errors (either in- or out-of-sample) are an increasing function of the distance between

the true vector β and its estimate

β . The more information contained in β , the smaller this

distance. Information in

is a function of sample and non-sample information. Let the

unrestricted estimate be denoted

and its theory-constrained counterpart be , where only

contains sample information and

β contains sample and non-sample input. When predicting in-

sample observations of the dependent variable, it is possible that sample information may

dominate the non-sample information (the information in theoretical restrictions). Though the

restrictions do reflect reality to some degree, their parametric representation is not exactly true,

and allowing the estimation routine to search unrestricted over all possible value for β results in

significantly smaller [in-sample] prediction errors than if constrained by theory.

ˆ

U

β

ˆ

R

ˆ

ˆ

β

ˆ

R

β

ˆ

U

β

ˆ

3

Now, let us turn to the case where β and are used for forecasting out-of-sample.

Specifically, we focus on the case where observations from earlier dates are used to forecast

future observations. It is likely that the true parameter vector β changes over time due to

changing consumer preferences, model misspecification, and other complexities involved in

econometrics, in ways difficult to capture even with the most advanced random coefficient

estimation techniques. If this is true, then sample information from previous time periods are of

less use in explaining future observations as they were in explaining in-sample observations. But

the value of non-sample information via theoretical restrictions stays the same because theory is

not time dependent. The amount of information in theoretical restrictions, relative to the

information contained in the in-sample observations, is now greater, and the restricted estimates’

forecasting ability, relative to unrestricted estimates, begins to improve.

U

ˆ

R

β

ˆ

Some evidence for this is given in Table 1 using data from KB and their form of the

AIDS model. This table shows the ratio of forecast errors from an unrestricted AIDS model to an

AIDS model with symmetry and homogeneity imposed. With only a one-year-ahead forecast

horizon, the restricted model performed better in some cases and worse in others. Once this

horizon increases, the restricted form has lower errors for all food groups. As the forecast

horizon increases, the theoretically constrained model forecasts better. This may be due to the

economic content of the restrictions, i.e., that the restrictions are theory-based and the theory is

sound.

Restrictions do not have to be based on theory, empirical results, or even make sense to

improve forecasts. Restrictions may improve forecasts simply because they increase the degrees

of freedom (Brieman). As Sawa notes, even if one model is a closer approximation to the true

model analytically, in small samples, models with more degrees of freedom may better represent

the true data generating process. Consider again the data and AIDS model used by KB. In Table

1, an unrestricted AIDS model is compared to a parsimonious AIDS model, where the value of all

4

parameters except for own-price and intercept terms are set to zero. At a one-year horizon, the

unrestrained AIDS model has lower forecast errors for four out of six goods, but at a 1-11 year

horizon, the parsimonious AIDS model has better forecasts for four out of six goods. At longer

forecast horizons, forecast improvements can be obtained simply by increasing the degrees of

freedom. This finding is not isolated; it is generally accepted that models with more degrees of

freedom tend to forecast better.

Consider again one forecast series from the parameter vector

β and one from the vector

. In this case, it is assumed that β is estimated using restrictions not based on theory, but

since restrictions are imposed, the degrees of freedom are higher for

β than . The mean-

squared error of

β from its true value β is the variance of the estimator plus the squared bias,

i.e., . Forecasts from β will be more

accurate than those from

β if . If the restrictions are not true, it

is certainly the case that

. However, since degrees of freedom are higher for

the restricted estimate, it may be that

V

U

ˆ

R

ˆ

R

β

ˆ

R

ˆ

[]

2

β−

+

BIAS

U

β

ˆ

R

ˆ

V=

( ) () ()

2

RRRR

2

R

BIASVβ

ˆ

Eβ

ˆ

ββ

ˆ

E +=+−

U

ˆ

U

2

RR

VBIASV +<

2

U

2

R

BIAS >

UR

V

R

ˆ

2

U

BIAS

<

, such that the mean-squared error for β is lower

than

β , thus producing better forecasts. A case can be made for V

R

being lower than V

U

. With

more degrees of freedom, the restricted parameter estimates are derived from more observations;

thus, their variability in repeated samples should be smaller (Breiman).

R

ˆ

U

ˆ

The KB study found that models with economic restrictions forecast better than their

unrestrained counterparts. We have just explained how this could occur. First, the economic

theory used to derive those restrictions might be valuable non-sample information, i.e., the theory

might be correct. Second, even if the theory is not correct, restrictions serve to increase degrees

of freedom, and more degrees of freedom could result in more accurate forecasts. Which

explanation is correct? This is an important question to address, because the answer will guide

5