Showing papers on "Proper linear model published in 1984"
TL;DR: The class of generalized additive models is introduced, which replaces the linear form E fjXj by a sum of smooth functions E sj(Xj), and has the advantage of being completely auto- matic, i.e., no "detective work" is needed on the part of the statistician.
Abstract: Likelihood-based regression models such as the normal linear regression model and the linear logistic model, assume a linear (or some other parametric) form for the covariates $X_1, X_2, \cdots, X_p$. We introduce the class of generalized additive models which replaces the linear form $\sum \beta_jX_j$ by a sum of smooth functions $\sum s_j(X_j)$. The $s_j(\cdot)$'s are unspecified functions that are estimated using a scatterplot smoother, in an iterative procedure we call the local scoring algorithm. The technique is applicable to any likelihood-based regression model: the class of generalized linear models contains many of these. In this class the linear predictor $\eta = \Sigma \beta_jX_j$ is replaced by the additive predictor $\Sigma s_j(X_j)$; hence, the name generalized additive models. We illustrate the technique with binary response and survival data. In both cases, the method proves to be useful in uncovering nonlinear covariate effects. It has the advantage of being completely automatic, i.e., no "detective work" is needed on the part of the statistician. As a theoretical underpinning, the technique is viewed as an empirical method of maximizing the expected log likelihood, or equivalently, of minimizing the Kullback-Leibler distance to the true model.
2,708 citations
TL;DR: A general index of predictive discrimination is used to measure the ability of a model developed on training samples of varying sizes to predict survival in an independent test sample of patients suspected of having coronary artery disease.
Abstract: Regression models such as the Cox proportional hazards model have had increasing use in modelling and estimating the prognosis of patients with a variety of diseases. Many applications involve a large number of variables to be modelled using a relatively small patient sample. Problems of overfitting and of identifying important covariates are exacerbated in analysing prognosis because the accuracy of a model is more a function of the number of events than of the sample size.
We used a general index of predictive discrimination to measure the ability of a model developed on training samples of varying sizes to predict survival in an independent test sample of patients suspected of having coronary artery disease. We compared three methods of model fitting: (1) standard ‘step-up’ variable selection, (2) incomplete principal components regression, and (3) Cox model regression after developing clinical indices from variable clusters. We found regression using principal components to offer superior predictions in the test sample, whereas regression using indices offers easily interpretable models nearly as good as the principal components models. Standard variable selection has a number of deficiencies.
1,657 citations
01 Jan 1984
TL;DR: A class of methods for robust regression is developed, based on estimators of scale, that are introduced because of their invulnerability to large fractions of contaminated data and are proposed to be called “S-estimators”.
Abstract: There are at least two reasons why robust regression techniques are useful tools in robust time series analysis. First of all, one often wants to estimate autoregressive parameters in a robust way, and secondly, one sometimes has to fit a linear or nonlinear trend to a time series. In this paper we shall develop a class of methods for robust regression, and briefly comment on their use in time series. These new estimators are introduced because of their invulnerability to large fractions of contaminated data. We propose to call them “S-estimators” because they are based on estimators of scale.
924 citations
482 citations
447 citations
TL;DR: In this paper, many model selection rules proposed from various viewpoints are available for normal linear regression analysis and the information criteria AIC, FPE, PSS, BIC, and BIC are explicitly obtained.
Abstract: In normal linear regression analysis, many model selection rules proposed from various viewpoints are available. For the information criteria AIC, FPE, $C_p$, PSS and BIC, the asymptotic distribution of the selected model and the asymptotic quadratic risk based on each criterion are explicitly obtained.
346 citations
TL;DR: Modifications and extensions of linear model displays lead to three methods for diagnostic checking of logistic regression models, which are illustrated through the analyses of simulated and real data.
Abstract: In ordinary linear regression, graphical diagnostic displays can be very useful for detecting and examining anomalous features in the fit of a model to data. For logistic regression models, the discreteness of binary data makes it difficult to interpret such displays. Modifications and extensions of linear model displays lead to three methods for diagnostic checking of logistic regression models. Local mean deviance plots are useful for detecting overall lack of fit. Empirical probability plots help point out isolated departures from the fitted model. Partial residual plots, when smoothed to show underlying structure, help identify specific causes of lack of fit. These methods are illustrated through the analyses of simulated and real data.
306 citations
TL;DR: In this article, the problem of predicting future responses of an individual based on the random effects model is also considered, and the prediction mean squared errors of various predictors are derived and compared for the practical situation where all parameters of the model are unknown and estimates must be used.
Abstract: Estimation of an individual's regression coefficients in a multivariate random effects linear model is considered. The problem of prediction of future responses of an individual based on the random effects model is also considered, and the prediction mean squared errors of various predictors are derived and compared for the practical situation where all parameters of the model are unknown and estimates must be used.
116 citations
TL;DR: In this article, the authors discuss that the standard linear regression model has been an attractive model to use in econometrics, and that if econometricians can uncover stable economic relations that satisfy at least approximately the assumptions of this model, they deserve the credit and the convenience of using it.
Abstract: Publisher Summary This chapter discusses that the standard linear regression model has been an attractive model to use in econometrics. If econometricians can uncover stable economic relations that satisfy at least approximately the assumptions of this model, they deserve the credit and the convenience of using it. Sometimes, however, econometricians are not lucky or ingenious enough to specify a stable regression relationship, and the relationship being studied gradually changes. Under such circumstances, an option is to specify a linear regression model with stochastically evolving coefficients. The chapter also reviews that for the purpose of parameter estimation, this model takes into account the possibility that the coefficients may be time dependent and provides estimates of these coefficients at different points of time. For the purpose of forecasting, this model has an advantage over the standard regression model in utilizing the estimates of the most up-to-date coefficients. From the viewpoint of hypothesis testing, this model serves as a viable alternative to the standard regression model for the purpose of checking the constancy of the coefficients of the latter model.
111 citations
102 citations
TL;DR: In this paper, weak consistency of the regression parameter estimators under regularity conditions which avoid restrictions on the censoring patterns is considered. But this method is not suitable for linear regression models with residual distribution and right-censored response variables.
Abstract: Buckley and James (1979) proposed an estimation method for linear regression models with unspecified residual distribution and right-censored response variables. In this paper we consider weak consistency of the regression parameter estimators under regularity conditions which avoid restrictions on the censoring patterns.
TL;DR: It is found that overall, pre-testing is preferable to pure OLS regression techniques and generally compares favourably with the strategy of always correcting for possible autocorrelation.
TL;DR: Smoothness priors as mentioned in this paper represent prior information that an unknown function does not change slope quickly and hence that the function describes a simple curve (e.g., Wahba 1978). But smoothing priors for the multiple nonlinear regression model are developed in such a way that estimates and standard errors can be obtained as a natural and conceptually straightforward extension of linear multiple regression estimation with the addition of dummy variables and dummy observations.
Abstract: Smoothness priors represent prior information that an unknown function does not change slope quickly and hence that the function describes a simple curve (e.g., Wahba 1978). In this article such priors for the multiple nonlinear regression model are developed in such a way that estimates and “standard errors” can be obtained as a natural and conceptually straightforward extension of linear multiple-regression estimation with the addition of dummy variables and dummy observations. Relations to spline and polynomial interpolation are described. An illustrative example of cost-function estimation is provided.
TL;DR: The literature pertaining to splines in regression analysis is reviewed in this paper, where the concepts of fixed and variable knot spline regression are developed and corresponding inferential procedures are considered.
Abstract: The literature pertaining to splines in regression analysis is reviewed. Spline regression is motivated as a simple extension of the basic polynomial regression model. Using this framework, the concepts of fixed and variable knot spline regression are developed and corresponding inferential procedures are considered. Smoothing splines are also seen to be an extension of polynomial regression and various optimality properties, as well as inferential and diagnostic methods, for these types of splines are discussed.
TL;DR: In this paper, the performance of biased estimators in the linear regression model when the assumption of homoscedasticity is not satisfied is investigated and conditions are derived which show when OLS and GLS are dominated by shrinkage estimator with respect to various mean square error criteria.
TL;DR: In this article, a theory for finding designs in estimating a linear functional of a regression function is developed for classes of regression functions which are infinite dimensional and the estimates are restricted to be linear and the design (and estimate) sought is minimax for mean square error.
Abstract: Theory for finding designs in estimating a linear functional of a regression function is developed for classes of regression functions which are infinite dimensional. These classes can be viewed as representing possible departures from an "ideal" simple model and thus describe a model robust setting. The estimates are restricted to be linear and the design (and estimate) sought is minimax for mean square error. The structure of the design is obtained in a variety of cases; some asymptotic theory is given when the functionals are integrals. As to be expected, optimal designs depend critically on the particular functional to be estimated. The associated estimate is generally not a least squares estimate but we note some examples where a least squares estimate, in conjunction with a good design, is adequate.
TL;DR: In this paper, the authors derived a best unbiased estimator and a minimum MSE estimator under the assumption of a normal distribution, and compared the bias and the MSE of these estimators.
Abstract: If a linear regression model is used for prediction, the mean squared error of prediction (MSEP) measures the performance of the model. The MSEP is a function of unknown parameters and good estimates of it are of interest. This article derives a best unbiased estimator and a minimum MSE estimator under the assumption of a normal distribution. It compares the bias and the MSE of these estimators and some others. Similar results are presented for the case in which the model is used to estimate values of the response function.
TL;DR: This article reviewed statistical tests and procedures which aid the experimenter in deterrmining lack of fit or functional misspecification associated with the deterministic portion of a proposed linear regression model.
Abstract: Assessment of the adequacy of a proposed linear regression model is necessarily subjective. However, the following three criteria may warrant investigation whether the distributional assumptions for the stochastic portion of the model are satisfied, whether the predictive capability of the model is satisfactory, and whether the deterministic portion of the model is adejuate in a statistical sense. The first two criteria have been reviewed in the literature to some extent. This paper reviews statistical tests and procedures which aid the experimenter in deterrmining lack of fit or functional misspecification associated with the deterministic portion of a proposed linear regression model.
TL;DR: A comprehensive list of articles on least absolute value (LAV) estimation as applied to linear and non-linear regression models and in systems of equations can be found in this paper, where references to the LAV method as applied in approximation theory are also included.
Abstract: This paper presents a comprehensive listing of articles on least absolute value (LAV) estimation as applied to linear and non-linear regression models and in systems of equations. References to the LAV method as applied in approximation theory are also included. Annotations describing the content of each article follow each reference.
TL;DR: In this paper, the authors discuss the relationship between the multiple correlation coefficient and the correlations among the variables in a bivariate regression and point out a common error concerning the variation explained by regression variables.
Abstract: We discuss the relationship between the multiple correlation coefficient and the correlations among the variables in a bivariate regression. We point out a common error concerning the variation explained by regression variables. A geometric interpretation is given and examples to illustrate the error are provided. example, the proportion of variation in y explained by x1 andx2 together can be greater than the sum of the proportions explained by x1 alone and x2 alone. It would appear that this situation may be misunderstood; in fact, two recent textbooks on regression actually state that this cannot arise. We present results, diagrams and examples which show that it does arise and that it may be quite common. We will call such an occurrence enhancement. The fitted regression coefficients can also exhibit bizarre behaviour. If we compare the fitted coefficient of, say, x1 in the regression of y on x1 with the fitted coefficient of x1 in the bivariate regression then these coefficients may differ by orders of magnitude or be of opposite sign. Such behaviour is sometimes called suppression. We show that there is a close connection between enhancement and suppression. We give examples taken both from bivariate and multiple regression to illustrate these phenomena.
TL;DR: In this article, local sensitivity analysis is defined as matrix derivatives of the GLS estimator with respect to changes in the weight matrix ∑-1, where ∑ = var(e).
Abstract: For the linear regression model y = Xβ + e the local sensitivities of estimates for β are investigated with respect to a general error structure for the residuals e. In particular, we define local sensitivity analysis as matrix derivatives of the general least squares (GLS) estimator with respect to changes in the weight matrix ∑-1, where ∑ = var(e). The results are extensions of derivative formulas given in Belsley, Kuh, and Welsch (1980) for ordinary least squares (OLS) regressions. An example is given showing how the new derivatives can be used for regression diagnostics in regression models with autocorrelated errors.
TL;DR: In this article, a weighted resampling method analogous to the weighted jackknife developed by Hinkley (1977) is proposed for regression models, which is used for functions of a linear model.
TL;DR: This article compares eight ways to form regression models by forming them with old data and then validating them with fresh data to study the quality of prediction.
Abstract: The best way to validate the predictive ability of a statistical model is to apply it to new data. This article compares eight ways to form regression models by forming them with old data and then validating them with fresh data. One goal here is to study which methods will work as a function of the type of data. To some extent one can tell which methods will work well by looking at the data. Another goal is to study the quality of prediction when the regression is applied to new data. Prediction quality is determined in large part by the distance of the new data in relation to the old.
TL;DR: In this article, a statistical model is proposed that describes the determination of an educational outcome variable as a nonlinear function of explanatory variables defined at different levels of a survey data hierarchy, say students and classes.
Abstract: A statistical model is proposed that describes the determination of an educational outcome variable as a nonlinear function of explanatory variables defined at different levels of a survey data hierarchy, say students and classes. The model hypothesizes that the student-level explanatory variables form a composite such that the intercept and slope in the regression of the outcome on the composite vary across classes systematically as functions of class-level variables and aggregates. A method is described for estimating the parameters of the model using robust techniques. The theoretical and practical derivation of the model is discussed, and an example is given.
TL;DR: In this article, the authors investigate the multicollinearity among the property characteristics (regressor variables) and examine the stability of the estimated regression coefficients over time using ridge regression techniques.
Abstract: The use of multiple regression analysis as a tool of real estate valuation has received considerable attention in recent years. The primary objectives of this study are to investigate the multicollinearity among the property characteristics (regressor variables) and examine the stability of the estimated regression coefficients over time. Ridge regression techniques are used to partially adjust for the presence of collinearity. The results indicate that the ridge regression model provides a consistent set of properly signed, statistically significant regression coefficients throughout the sample period. Furthermore, ridge regression techniques are shown to have certain advantages over those of ordinary least squares for establishing logical and consistent values for specific property characteristics.
TL;DR: In this article, an alternative technique for constructing a prediction function for the normal linear regression model based on the concept of maximum likelihood is proposed, and the form of this prediction function is evaluated and normalized to produce a multivariate Student's t-density.
Abstract: An alternative technique to current methods for constructing a prediction function for the normal linear regression model is proposed based on the concept of maximum likelihood. The form of this prediction function is evaluated and normalized to produce a multivariate Student's t-density. Consistency properties are established under regularity conditions, and an empirical comparison, based on the Kullback-Leibler information divergence, is made with some other prediction functions.
Book•
01 Jan 1984
TL;DR: In this paper, the authors present a model extension for linear regression with one Categorical Independent Variable Linear regression with two independent variables and interaction analysis of Covariance Analysis Considerations and model extensions.
Abstract: Introduction Basic Statistical Concepts Matrix Algebra Multiple Regression Annalysis Linear Regression with One Categorical Independent Variable Linear Regression with Two Categorical Independent Variables Regression Models with Interaction Analysis of Covariance Analysis Considerations and Model Extensions