Consider the ridge estimate (λ) for β in the model unknown, (λ) = (X T X + nλI)−1 X T y. We study the method of generalized cross-validation (GCV) for choosing a good value for λ from the data. The estimate is the minimizer of V(λ) given by where A(λ) = X(X T X + nλI)−1 X T . This estimate is a rotation-invariant version of Allen's PRESS, or ordinary cross-validation. This estimate behaves like a risk improvement estimator, but does not require an estimate of σ2, so can be used when n − p is small, or even if p ≥ 2 n in certain cases. The GCV method can also be used in subset selection and singular value truncation methods for regression, and even to choose from among mixtures of these methods.

/pdf/generalized-cross-validation-as-a-method-for-choosing-a-good-ni6kr99a0h.pdf

Generalized Cross-Validation as a Method for Choosing a Good Ridge Parameter

A Biometrics Invited Paper. The Analysis and Selection of Variables in Linear Regression

Cross-validation:a review 2

Biased estimators of the coefficients in the linear regression model have been the subject of considerable discussion in the recent, literature. The purpose of this paper is to provide a unified approach to the study of biased estimators in an effort to determine their relative merits. The class of estimators includes the simple and the generalized ridge estimators proposed by Hoerl and Kennard [9], the principal component estimator with extensions such as that, proposed by Marquardt [19] and the shrunken estimator proposed by Stein [23]. The problem of estimating the biasing parameters is considered and illustrated with two examples.

A Class of Biased Estimators in Linear Regression

A Survey of Ridge Regression and Related Techniques for Improvements over Ordinary Least Squares

In this paper we explore the similarity between the minimum mean square error estimator and Hoer1 and Kennard's ridge regression.

R. W. Farebrother

Papers

The Minimum Mean Square Error Linear Estimator and Ridge Regression