Statistics for Spatial Data.

Convergence of Probability Measures. By P. Billingsley. Chichester, Sussex, Wiley, 1968. xii, 253 p. 9 1/4“. 117s.

Convergence of Probability Measures

1. Introduction and Overview. 2. Detecting Influential Observations and Outliers. 3. Detecting and Assessing Collinearity. 4. Applications and Remedies. 5. Research Issues and Directions for Extensions. Bibliography. Author Index. Subject Index.

Regression Diagnostics: Identifying Influential Data and Sources of Collinearity

Summary. Recent work by Reiss and Ogden provides a theoretical basis for sometimes preferring restricted maximum likelihood (REML) to generalized cross-validation (GCV) for smoothing parameter selection in semiparametric regression. However, existing REML or marginal likelihood (ML) based methods for semiparametric generalized linear models (GLMs) use iterative REML or ML estimation of the smoothing parameters of working linear approximations to the GLM. Such indirect schemes need not converge and fail to do so in a non-negligible proportion of practical analyses. By contrast, very reliable prediction error criteria smoothing parameter selection methods are available, based on direct optimization of GCV, or related criteria, for the GLM itself. Since such methods directly optimize properly defined functions of the smoothing parameters, they have much more reliable convergence properties. The paper develops the first such method for REML or ML estimation of smoothing parameters. A Laplace approximation is used to obtain an approximate REML or ML for any GLM, which is suitable for efficient direct optimization. This REML or ML criterion requires that Newton–Raphson iteration, rather than Fisher scoring, be used for GLM fitting, and a computationally stable approach to this is proposed. The REML or ML criterion itself is optimized by a Newton method, with the derivatives required obtained by a mixture of implicit differentiation and direct methods. The method will cope with numerical rank deficiency in the fitted model and in fact provides a slight improvement in numerical robustness on the earlier method of Wood for prediction error criteria based smoothness selection. Simulation results suggest that the new REML and ML methods offer some improvement in mean-square error performance relative to GCV or Akaike's information criterion in most cases, without the small number of severe undersmoothing failures to which Akaike's information criterion and GCV are prone. This is achieved at the same computational cost as GCV or Akaike's information criterion. The new approach also eliminates the convergence failures of previous REML- or ML-based approaches for penalized GLMs and usually has lower computational cost than these alternatives. Example applications are presented in adaptive smoothing, scalar on function regression and generalized additive model selection.

/pdf/fast-stable-restricted-maximum-likelihood-and-marginal-4rm5jbnyqs.pdf

Fast stable restricted maximum likelihood and marginal likelihood estimation of semiparametric generalized linear models

On robust estimation of the location parameter

Nonparametric regression techniques are often sensitive to the presence of correlation in the errors. The practical consequences of this sensitivityare explained, including the breakdown of several popu- lar data-driven smoothing parameter selection methods. We review the existing literature in kernel regression, smoothing splines and wavelet regression under correlation, both for short-range and long-range depen- dence. Extensions to random design, higher dimensional models and adaptive estimation are discussed. some of the difficulties associated with the presence of correlation in nonparametric regression, (2) to provide an overview of the nonparameteric regres- sion literature that deals with the correlated errors case and (3) to discuss some new developments in this area. Much of the literature in nonparametric regression relies on asymptotic arguments to clarify the probabilistic behavior of the proposed meth- ods. The same approach will be used here, but we attempt to provide intuition into the results as well. In this article, we will be looking at the following statistical model:

/pdf/nonparametric-regression-with-correlated-errors-30i7mrqi4j.pdf

Nonparametric Regression with Correlated Errors

While the additive model is a popular nonparametric regression method, many of its theoretical properties are not well understood, especially when the backfitting algorithm is used for computation of the estimators. This article explores those properties when the additive model is fitted by local polynomial regression. Sufficient conditions guaranteeing the asymptotic existence of unique estimators for the bivariate additive model are given. Asymptotic approximations to the bias and the variance of a homoscedastic bivariate additive model with local polynomial terms of odd and even degree are computed. This model is shown to have the same rate of convergence as that of univariate local polynomial regression.

/pdf/fitting-a-bivariate-additive-model-by-local-polynomial-248y106omg.pdf

Fitting a Bivariate Additive Model by Local Polynomial Regression

We study the class of penalized spline estimators, which enjoy similarities to both regression splines, without penalty and with fewer knots than data points, and smoothing splines, with knots equal to the data points and a penalty controlling the roughness of the fit. Depending on the number of knots, sample size and penalty, we show that the theoretical properties of penalized regression spline estimators are either similar to those of regression splines or to those of smoothing splines, with a clear breakpoint distinguishing the cases. We prove that using fewer knots results in better asymptotic rates than when using a large number of knots. We obtain expressions for bias and variance and asymptotic rates for the number of knots and penalty parameter.

/pdf/asymptotic-properties-of-penalized-spline-estimators-431x3vvdd5.pdf

Asymptotic properties of penalized spline estimators

Estimation of finite population totals in the presence of auxiliary information is considered. A class of estimators based on local polynomial regression is proposed. Like generalized regression estimators, these estimators are weighted linear combinations of study variables, in which the weights are calibrated to known control totals, but the assumptions on the superpopulation model are considerably weaker. The estimators are shown to be asymptotically design-unbiased and consistent under mild assumptions. A variance approximation based on Taylor linearization is suggested and shown to be consistent for the design mean squared error of the estimators. The estimators are robust in the sense of asymptotically attaining the Godambe–Joshi lower bound to the anticipated variance. Simulation experiments indicate that the estimators are more efficient than regression estimators when the model regression function is incorrectly specified, while being approximately as efficient when the parametric specification is correct.

/pdf/local-polynomial-regresssion-estimators-in-survey-sampling-2rmq9jn915.pdf

Jean D. Opsomer

Papers

Nonparametric Regression with Correlated Errors

Fitting a Bivariate Additive Model by Local Polynomial Regression

Fitting a Bivariate Additive Model by Local Polynomial Regression

Asymptotic properties of penalized spline estimators

Local polynomial regresssion estimators in survey sampling