Showing papers on "Proper linear model published in 1988"

Sensitivity analysis in linear regression

Abstract: The Prediction Matrix Role of Variables in a Regression Equation Effects of an Observation on a Regression Equation Assessing the Influence of Multiple Observations Joint Impact of a Variable and an Observation Assessing the Effect of Errors of Measurements A Study of Model Sensitivity by the Generalized Linear Model Approach Computational Considerations. Appendix: Summary of Vector and Matrix Norms, Proofs of Three Theorems.

A regression model for time series of counts

Abstract: SUMMARY This paper discusses a model for regression analysis with a time series of counts. Correlation is assumed to arise from an unobservable process added to the linear predictor in a log linear model. An estimating equation approach used for parameter estimation leads to an iterative weighted and filtered least-squares algorithm. Asymptotic properties for the regression coefficients are presented. We illustrate the technique with an analysis of trends in U.S. polio incidence since 1970.

Applied Statistics: An Analysis of Variance and Regression.

Abstract: Preface.1. Data Screening.1.1 Variables and Their Classification.1.2 Describing the Data.1.2.1 Errors in the Data.1.2.2 Descriptive Statistics.1.2.3 Graphical Summarization.1.3 Departures from Assumptions.1.3.1 The Normal Distribution.1.3.2 The Normality Assumption.1.3.3 Transformations.1.3.4 Independence.1.4 Summary.Problems.References.2. One-Way Analysis of Variance Design.2.1 One-Way Analysis of Variance with Fixed Effects.2.1.1 Example.2.1.2 The One-Way Analysis of Variance Model with Fixed Effects.2.1.3 Null Hypothesis: Test for Equality of Population Means.2.1.4 Estimation of Model Terms.2.1.5 Breakdown of the Basic Sum of Squares.2.1.6 Analysis of Variance Table.2.1.7 The F Test.2.1.8 Analysis of Variance with Unequal Sample Sizes.2.2 One-Way Analysis of Variance with Random Effects.2.2.1 Data Example.2..2.2 The One-Way Analysis of Variance Model with Random Effects.2.2.3 Null Hypothesis: Test for Zero Variance of Population Means.2.2.4 Estimation of Model Terms.2.2.5 The F Test.2.3 Designing an Observational Study or Experiment.2.3.1 Randomization for Experimental Studies.2.3.2 Sample Size and Power.2.4 Checking if the Data Fit the One-Way ANOVA Model.2.4.1 Normality.2.4.2 Equality of Population Variances.2.4.3 Independence.2.4.4 Robustness.2.4.5 Missing Data.2.5 What to Do if the Data Do Not Fit the Model.2.5.1 Making Transformations.2.5.2 Using Nonparametric Methods.2.5.3 Using Alternative ANOVAs.2.6 Presentation and Interpretation of Results.2.7 Summary.Problems.References.3. Estimation and Simultaneous Inference.3.1 Estimation for Single Population Means.3.1.1 Parameter Estimation.3.1.2 Confidence Intervals.3.2 Estimation for Linear Combinations of Population Means.3.2.1 Differences of Two Population Means.3.2.2 General Contrasts for Two or More Means.3.2.3 General Contrasts for Trends.3.3 Simultaneous Statistical Inference.3.1.1 Straightforward Approach to Inference.3.3.2 Motivation for Multiple Comparison Procedures and Terminology.3.3.3 The Bonferroni Multiple Comparison Method.3.3.4 The Tukey Multiple Comparison Method.3.3.5 The Scheffe Multiple Comparison Method.3.4 Inference for Variance Components.3.5 Presentation and Interpretation of Results.3.6 Summary.Problems.References.4. Hierarchical or Nested Design.4.1 Example.4.2 The Model.4.3 Analysis of Variance Table and F Tests.4.3.1 Analysis of Variance Table.4.3.2 F Tests.4.3.3 Pooling.4.4 Estimation of Parameters.4.4.1 Comparison with the One-Way ANOVA Model of Chapter 2.4.5 Inferences with Unequal Sample Sizes.4.5.1 Hypothesis Testing.4.5.2 Estimation.4.6 Checking If the Data Fit the Model.4.7 What to Do If the Data Don't Fit the Model.4.8 Designing a Study.4.8.1 Relative Efficiency.4.9 Summary.Problems.References.5. Two Crossed Factors: Fixed Effects and Equal Sample Sizes.5.1 Example.5.2 The Model.5.3 Interpretation of Models and Interaction.5.4 Analysis of Variance and F Tests.5.5 Estimates of Parameters and Confidence Intervals.5.6 Designing a Study.5.7 Presentation and Interpretation of Results.5.8 Summary.Problems.References.6 Randomized Complete Block Design.6.1 Example.6.2 The Randomized Complete Block Design.6.3 The Model.6.4 Analysis of Variance Table and F Tests.6.5 Estimation of Parameters and Confidence Intervals.6.6 Checking If the Data Fit the Model.6.7 What to Do if the Data Don't Fit the Model.6.7.1 Friedman's Rank Sum Test.6.7.2 Missing Data.6.8 Designing a Randomized Complete Block Study.6.8.1 Experimental Studies.6.8.2 Observational Studies.6.9 Model Extensions.6.10 Summary.Problems.References.7. Two Crossed Factors: Fixed Effects and Unequal Sample Sizes.7.1 Example.7.2 The Model.7.3 Analysis of Variance and F Tests.7.4 Estimation of Parameters and Confidence Intervals.7.4.1 Means and Adjusted Means.7.4.2 Standard Errors and Confidence Intervals.7.5 Checking If the Data Fit the Two-Way Model.7.6 What To Do If the Data Don't Fit the Model.7.7 Summary.Problems.References.8. Crossed Factors: Mixed Models.8.1 Example.8.2 The Mixed Model.8.3 Estimation of Fixed Effects.8.4 Analysis of Variance.8.5 Estimation of Variance Components.8.6 Hypothesis Testing.8.7 Confidence Intervals for Means and Variance Components.8.7.1 Confidence Intervals for Population Means.8.7.2 Confidence Intervals for Variance Components.8.8 Comments on Available Software.8.9 Extensions of the Mixed Model.8.9.1 Unequal Sample Sizes.8.9.2 Fixed, Random, or Mixed Effects.8.9.3 Crossed versus Nested Factors.8.9.4 Dependence of Random Effects.8.10 Summary.Problems.References.9. Repeated Measures Designs.9.1 Repeated Measures for a Single Population.9.1.1 Example.9.1.2 The Model.9.1.3 Hypothesis Testing: No Time Effect.9.1.4 Simultaneous Inference.9.1.5 Orthogonal Contrasts.9.1.6 F Tests for Trends over Time.9.2 Repeated Measures with Several Populations.9.2.1 Example.9.2.2 Model.9.2.3 Analysis of Variance Table and F Tests.9.3 Checking if the Data Fit the Repeated Measures Model.9.4 What to Do if the Data Don't Fit the Model.9.5 General Comments on Repeated Measures Analyses.9.6 Summary.Problems.References.10. Linear Regression: Fixed X Model.10.1 Example.10.2 Fitting a Straight Line.10.3 The Fixed X Model.10.4 Estimation of Model Parameters and Standard Errors.10.4.1 Point Estimates.10.4.2 Estimates of Standard Errors.10.5 Inferences for Model Parameters: Confidence Intervals.10.6 Inference for Model Parameters: Hypothesis Testing.10.6.1 t Tests for Intercept and Slope.10.6.2 Division of the Basic Sum of Squares.10.6.3 Analysis of Variance Table and F Test.10.7 Checking if the Data Fit the Regression Model.10.7.1 Outliers.10.7.2 Checking for Linearity.10.7.3 Checking for Equality of Variances.10.7.4 Checking for Normality.10.7.5 Summary of Screening Procedures.10.8 What to Do if the Data Don't Fit the Model.10.9 Practical Issues in Designing a Regression Study.10.9.1 Is Fixed X Regression an Appropriate Technique?10.9.2 What Values of X Should Be Selected?10.9.3 Sample Size Calculations.10.10 Comparison with One-Way ANOVA.10.11 Summary.Problems.References.11. Linear Regression: Random X Model and Correlation.11.1 Example.11.1.1 Sampling and Summary Statistics.11.2 Summarizing the Relationship Between X and Y.11.3 Inferences for the Regression of Y and X.11.3.1 Comparison of Fixed X and Random X Sampling.11.4 The Bivariate Normal Model.11.4.1 The Bivariate Normal Distribution.11.4.2 The Correlation Coefficient.11.4.3 The Correlation Coefficient: Confidence Intervals and Tests.11.5 Checking if the Data Fit the Random X Regression Model.11.5.1 Checking for High-Leverage, Outlying, and Influential Observations.11.6 What to Do if the Data Don't Fit the Random X Model.11.6.1 Nonparametric Alternatives to Simple Linear Regression.11.6.2 Nonparametric Alternatives to the Pearson Correlation.11.7 Summary.Problem.References.12. Multiple Regression.12.1 Example.12.2 The Sample Regression Plane.12.3 The Multiple Regression Model.12.4 Parameters Standard Errors, and Confidence Intervals.12.4.1 Prediction of E(Y\\X1,...,Xk).12.4.2 Standardized Regression Coefficients.12.5 Hypothesis Testing.12.5.1 Test That All Partial Regression Coefficients Are 0.12.5.2 Tests that One Partial Regression Coefficient is 0.12.6 Checking If the Data Fit the Multiple Regression Model.12.6.1 Checking for Outlying, High Leverage and Influential Points.12.6.2 Checking for Linearity.12.6.3 Checking for Equality of Variances.12.6.4 Checking for Normality of Errors.12.6.5 Other Potential Problems.12.7 What to Do If the Data Don't Fit the Model.12.8 Summary.Problems.References.13. Multiple and Partial Correlation.13.1 Example.13.2 The Sample Multiple Correlation Coefficient.13.3 The Sample Partial Correlation Coefficient.13.4 The Joint Distribution Model.13.4.1 The Population Multiple Correlation Coefficient.13.4.2 The Population Partial Correlation Coefficient.13.5 Inferences for the Multiple Correlation Coefficient.13.6 Inferences for Partial Correlation Coefficients.13.6.1 Confidence Intervals for Partial Correlation Coefficients.13.6.2 Hypothesis Tests for Partial Correlation Coefficients.13.7 Checking If the Data Fit the Joint Normal Model.13.8 What to Do If the Data Don't Fit the Model.13.9 Summary.Problems.References.14. Miscellaneous Topics in Regression.14.1 Models with Dummy Variables.14.2 Models with Interaction Terms.14.3 Models with Polynomial Terms.14.3.1 Polynomial Model.14.4 Variable Selection.14.4.1 Criteria for Evaluating and Comparing Models.14.4.2 Methods for Variable Selection.14.4.3 General Comments on Variable Selection.14.5 Summary.Problems.References.15. Analysis of Covariance.15.1 Example.15.2 The ANCOVA Model.15.3 Estimation of Model Parameters.15.4 Hypothesis Tests.15.5 Adjusted Means.15.5.1 Estimation of Adjusted Means and Standard Errors.15.5.2 Confidence Intervals for Adjusted Means.15.6 Checking If the Data Fit the ANCOVA Model.15.7 What to Do if the Data Don't Fit the Model.15.8 ANCOVA in Observational Studies.15.9 What Makes a Good Covariate.15.10 Measurement Error.15.11 ANCOVA versus Other Methods of Adjustment.15.12 Comments on Statistical Software.15.13 Summary.Problems.References.16. Summaries, Extensions, and Communication.16.1 Summaries and Extensions of Models.16.2 Communication of Statistics in the Context of Research Project.References.Appendix A.A.1 Expected Values and Parameters.A.2 Linear Combinations of Variables and Their Parameters.A.3 Balanced One-Way ANOVA, Expected Mean Squares.A.3.1 To Show EMS(MSa) = sigma2 + n SIGMAai= 1 alpha2i /(a - 1).A.3.2 To Show EMS(MSr) = sigma2.A.4 Balanced One-Way ANOVA, Random Effects.A.5 Balanced Nested Model.A.6 Mixed Model.A.6.1 Variances and Covariances of Yijk.A.6.2 Variance of Yi.A.6.3 Variance of Yi. - Yi'..A.7 Simple Linear Regression-Derivation of Least Squares Estimators.A.8 Derivation of Variance Estimates from Simple Linear Regression.Appendix B.Index.

Selection of optimal regression models via cross-validation

Abstract: A general problem arising in the development of regression models is the selection of the optimal model. Whenever a feature selection procedure, such as step forward, backward elimination, best subset or all possible combinations, or when a data compression approach, such as principal components or partial least-squares regression, is used, the question of how many regression terms to include in the final model must be addressed. This work describes the evaluation of four different criteria for selection of the optimal predictive regression model using cross-validation. The results obtained in this work illustrate the problems which can arise in the analysis of small or inadequately sampled data sets. The common approach, selecting the model which yields the absolute minimum in the predictive residual error sum of squares (PRESS), was found to have particularly poor statistical properties. A very simple change to a criterion based on the first local minimum in PRESS will provide a significant improvement in the cross-validation result. A criterion based on testing the significance of incremental changes in PRESS with an F-test may provide more robust performance than the local minimum in PRESS method.

Logistic Regression: Description, Examples, and Comparisons.

Abstract: Family studies have seen a dramatic increase in the use of statistical tools for the analysis of nominal-level variables. Such models are categorized as log-linear models often known as logit models or logistic-regression models. Despite logistic regressions growing popularity there is still confusion about the nature and proper use in family studies. The authors present a nontechnical discussion of logistic regression with illustrations and comparisons to better-known procedures such as percentaging tables and ordinary least squares regression. They contend that logistic regression can be a powerful statistical procedure when used appropriately. Nominal-level dependent variables are common in family research and logistic-regression models appropriately model the impact of predictor variables on these outcomes. With the proliferation of computer software for estimating logistic-regression models use of logistic regression is likely to increase. Though some time and attention is required to master it the advantages of logistic regression make the effort worthwhile.

Variable Addition and Lagrange Multiplier Tests for Linear and Logarithmic Regression Models

Abstract: The purpose of this paper is to examine the properties of various tests of linear and logarithmic (or log-linear) regression models. The test procedures may be categorized as follows: (1) tests that exploit the fact that the two models are intrinsically non-nested; (2) tests based on the Box-Cox data transformation; and (3) diagnostic tests of functional form misspecification against an unspecified alternative. The small-sample properties of several tests are investigated through a Monte Carlo experiment, as is their robustness to non-normality of the errors. Copyright 1988 by MIT Press.

A Note on Asymmetry and Robustness in Linear Regression

Abstract: We discuss the assumption of symmetry in robust linear regression. It is important to distinguish between the intercept term and the slope parameters. Ordinary robust regression requires no assumption of symmetry when interest lies in slope parameters; computer programs, confidence intervals, standard errors, and so forth do not change because the errors are asymmetric. The situation is radically different for bounded-influence estimators. With the exception of the Mallows class, these estimators are inconsistent for slope when the errors are asymmetric.

A Nonparametric Approach to the Truncated Regression Problem

Abstract: A description is given of a new method of estimating the regression parameters in the linear regression model from data where the dependent variable is subject to truncation. The residual distribution is allowed to be unspecified. The method is iterative and involves estimation of the residual distribution under the truncated sampling scheme. The technique can be interpreted as an iterative bias adjustment of the observations in order to correct the regression relationship in the sampled population to match that of the model. A simulation study compares the performance of various estimators, including one suggested by Bhattacharya, Chernoff, and Yang (1983). This truncation regression problem arises in many contexts of scientific and social research. In economics Tobin (1958) analyzed household expenditure on durable goods using a regression model that took account of the fact that the expenditure is always nonnegative. A more general situation was studied by Hausman and Wise (1976, 1977) in conn...

Characterizing the Domain of a Regression Model

Abstract: Several frequently used estimates of the domain of a regression model are discussed and compared with the convex hull of the predictor-variable values. A simple procedure for identifying new prediction values as interpolation or extrapolation points is presented using the convex hull as the model domain estimate and is illustrated by examples.

More about transformations and influential cases in regression

Abstract: We extend the diagnostic method of Cook and Wang (1983) for assessing case influence to regression models in which both response and linear predictor are transformed. We also suggest a possible improvement of Cook and Wang's method.

A routine for converting regression algorithms into corresponding orthogonal regression algorithms

Abstract: The routine converts any standard regression algorithm (that calculates both the coefficients and residuals) into a corresponding orthogonal regression algorithm. Thus, a standard, or robust, or L1 regression algorithm is converted into the corresponding standard, or robust, or L1orthogonal algorithm. Such orthogonal procedures are important for three basic reasons. First, they solve the classical errors-in-variables (EV) regression problem. Standard L2 orthogonal regression, obtained by converting ordinary least squares regression, is the maximum likelihood solution of the EV problem under Gaussian assumptions. However, this L2 solution is known to be unstable under even slight deviations from the model. Thus this routine's ability to create robust orthogonal regression algorithms from robust ordinary regression algorithms will also be very useful in practice. Second, orthogonal regression is intimately related to principal components procedures. Therefore, this routine can also be used to create corresponding L1, robust, etc., principal components algorithms. And third, orthogonal regression treats the x and y variables symmetrically. This is very important in many science and engineering modeling problems. Monte Carlo studies, which test the effectiveness of the routine under a variety of types of data, are given.

Regression diagnostics to detect nonrandom missingness in linear regression

Abstract: Missing data is a common problem in regression analysis. The usual estimation strategies require that the data values be missing completely at random (MCAR); if this is not the case, estimates can be severely biased. In this article it is shown that tests can be constructed based on common regression diagnostics to detect non-MCAR behavior. The construction of these tests and their properties when data are missing in one explanatory variable are detailed. Computer simulations indicate good power to detect various non-MCAR processes. Three examples are presented. Extensions to missing data in more than one explanatory variable and to arbitrary regression models are discussed.

Simultaneous Tolerance Intervals for the Linear Regression Model

Abstract: Statistical tolerance intervals are developed for the normal regression model. These intervals are constructed to guarantee at least P content for all possible values of the predictor variates. The confidence-set approach suggested by Wilson (1967) is used. Wilson used an ellipsoidal confidence set for the regression coefficients, β, and standard deviation of the residuals, σ, which imposes an unnecessary lower bound on σ. By modifying the ellipsoidal confidence set to remove the lower bound imposed on σ, we obtain narrower tolerance intervals. Another confidence set formed from the product set of confidence sets for β and σ is used to construct tolerance intervals. The tolerance intervals of the two new procedures are compared with those of the Wilson method for a simple linear regression example. The tolerance intervals based on the product confidence set are found to be efficient and easy to compute compared with those constructed from the ellipsoidal and the modified ellipsoidal confidence sets.

Asymptotic properties of linear regression estimators under a fixed censorship model

Abstract: Summary Modifications to the usual least squares normal equations have been proposed by Buckley and James (1979) when using distribution-free linear regression modelling under fixed right-censorship of the response. In this paper we consider large-sample distributional properties of the modified normal equation for the slope parameter and of estimators which ‘satisfy’ this equation.

A Note on the Existence of Maximum Likelihood Estimates in Linear Regression Models Using Interval-censored Data

Abstract: SUMMARY For a class of linear regression models with interval-censored data, including the exponential regression model as a special case, a necessary and sufficient condition is given for the existence of the maximum likelihood estimator (MLE). The condition is especially simple to check for simple linear regression. This result is useful for comparing the small sample properties of MLEs from different special cases of interval-censored data.

The linear regression model : model structure selection and biased estimators

Abstract: • A submitted manuscript is the version of the article upon submission and before peer-review. There can be important differences between the submitted version and the official published version of record. People interested in the research are advised to contact the author for the final version of the publication, or visit the DOI to the publisher's website. • The final author version and the galley proof are versions of the publication after peer review. • The final published version features the final layout of the paper including the volume, issue and page numbers.

Linear modelling with clustered observations : an illustrative example of earnings in the engineering industry.

Abstract: Conventional least-squares regression can lead to misleading results if the data have a hierarchical structure. An appropriate linear model is presented for such data. The model has conventional regression, variance-component, and random coefficient models as special cases and may be calibrated by use of recently available software. The effectiveness of the model is demonstrated by an analysis of earnings in the engineering industry. Particular attention is given to the problems of interpreting the parameter estimates and residuals.

Multiple linear regression

Abstract: Central Retinal Artery Equivalent Central Retinal Vein Equivalent Categorical Variables Mean Δ† 95% CI‡ P-value Mean Δ† 95% CI‡ P-value Race <0.001 <0.001 Black vs White 9.7 5.5, 14.0 12.3 6.2, 18.5 Black vs other 0.7 -4.3, 5.7 5.8 -1.3, 13.0 Hyperlipidemia (yes vs no) -8.6 -15.0, -2.3 0.008 Sex (Male vs female) -2.7 -7.3, 1.9 0.25 -1.0 -7.5, 5.6 0.77 Continuous Variables Slope† 95% CI‡ P-value Slope† 95% CI‡ P-value Age (/year) -0.35 -0.55, -0.15 0.001 -0.28 -0.57, 0.01 0.05