Showing papers in "Journal of The Royal Statistical Society Series B-statistical Methodology in 1998"

Smoothing parameter selection in nonparametric regression using an improved Akaike information criterion

Abstract: Summary. Many different methods have been proposed to construct nonparametric estimates of a smooth regression function, including local polynomial, (convolution) kernel and smoothing spline estimators. Each of these estimators uses a smoothing parameter to control the amount of smoothing performed on a given data set. In this paper an improved version of a criterion based on the Akaike information criterion (AIC), termed AICc, is derived and examined as a way to choose the smoothing parameter. Unlike plug-in methods, AICc can be used to choose smoothing parameters for any linear smoother, including local quadratic and smoothing spline estimators. The use of AICc avoids the large variability and tendency to undersmooth (compared with the actual minimizer of average squared error) seen when other 'classical' approaches (such as generalized cross-validation or the AIC) are used to choose the smoothing parameter. Monte Carlo simulations demonstrate that the AICc-based smoothing parameter is competitive with a plug-in method (assuming that one exists) when the plug-in method works well but also performs well when the plug-in approach fails or is unavailable.

Optimal scaling of discrete approximations to Langevin diffusions

Abstract: We consider the optimal scaling problem for proposal distributions in Hastings–Metropolis algorithms derived from Langevin diffusions. We prove an asymptotic diffusion limit theorem and show that the relative efficiency of the algorithm can be characterized by its overall acceptance rate, independently of the target distribution. The asymptotically optimal acceptance rate is 0.574. We show that, as a function of dimension n, the complexity of the algorithm is O(n1/3), which compares favourably with the O(n) complexity of random walk Metropolis algorithms. We illustrate this comparison with some example simulations.

Wavelet thresholding via a Bayesian approach

Abstract: We discuss a Bayesian formalism which gives rise to a type of wavelet threshold estimation in nonparametric regression. A prior distribution is imposed on the wavelet coefficients of the unknown response function, designed to capture the sparseness of wavelet expansion that is common to most applications. For the prior specified, the posterior median yields a thresholding procedure. Our prior model for the underlying function can be adjusted to give functions falling in any specific Besov space. We establish a relationship between the hyperparameters of the prior model and the parameters of those Besov spaces within which realizations from the prior will fall. Such a relationship gives insight into the meaning of the Besov space parameters. Moreover, the relationship established makes it possible in principle to incorporate prior knowledge about the function's regularity properties into the prior model for its wavelet coefficients. However, prior knowledge about a function's regularity properties might be difficult to elicit; with this in mind, we propose a standard choice of prior hyperparameters that works well in our examples. Several simulated examples are used to illustrate our method, and comparisons are made with other thresholding methods. We also present an application to a data set that was collected in an anaesthesiological study.

Weighting for unequal selection probabilities in multilevel models

Abstract: When multilevel models are estimated from survey data derived using multistage sampling, unequal selection probabilities at any stage of sampling may induce bias in standard estimators, unless the sources of the unequal probabilities are fully controlled for in the covariates. This paper proposes alternative ways of weighting the estimation of a two-level model by using the reciprocals of the selection probabilities at each stage of sampling. Consistent estimators are obtained when both the sample number of level 2 units and the sample number of level 1 units within sampled level 2 units increase. Scaling of the weights is proposed to improve the properties of the estimators and to simplify computation. Variance estimators are also proposed. In a limited simulation study the scaled weighted estimators are found to perform well, although non-negligible bias starts to arise for informative designs when the sample number of level 1 units becomes small. The variance estimators perform extremely well. The procedures are illustrated using data from the survey of psychiatric morbidity.

Automatic Bayesian curve fitting

Abstract: Summary. A method of estimating a variety of curves by a sequence of piecewise polynomials is proposed, motivated by a Bayesian model and an appropriate summary of the resulting posterior distribution. A joint distribution is set up over both the number and the position of the knots defining the piecewise polynomials. Throughout we use reversible jump Markov chain Monte Carlo methods to compute the posteriors. The methodology has been successful in giving good estimates for 'smooth' functions (i.e, continuous and differentiable) as well as functions which are not differentiable, and perhaps not even continuous, at a finite number of points. The methodology is extended to deal with generalized additive models.

Multivariate Bayesian variable selection and prediction

Abstract: The multivariate regression model is considered with p regressors. A latent vector with p binary entries serves to identify one of two types of regression coefficients: those close to 0 and those not. Specializing our general distributional setting to the linear model with Gaussian errors and using natural conjugate prior distributions, we derive the marginal posterior distribution of the binary latent vector. Fast algorithms aid its direct computation, and in high dimensions these are supplemented by a Markov chain Monte Carlo approach to sampling from the known posterior distribution. Problems with hundreds of regressor variables become quite feasible. We give a simple method of assigning the hyperparameters of the prior distribution. The posterior predictive distribution is derived and the approach illustrated on compositional analysis of data involving three sugars with 160 near infra-red absorbances as regressors.

Estimating smooth monotone functions

Abstract: Many situations call for a smooth strictly monotone function f of arbitrary flexibility. The family of functions defined by the differential equation D 2 f = w Df, where w is an unconstrained coefficient function, comprises the strictly monotone twice differentiable functions. The solution to this equation is f = C 0 + C 1 D -1 {exp(D -1 w)}, where C 0 and C 1 are arbitrary constants and D -1 is the partial integration operator. A basis for expanding w is suggested that permits explicit integration in the expression of f. In fitting data, it is also useful to regularize f by penalizing the integral of w 2 since this is a measure of the relative curvature in f. Applications are discussed to monotone nonparametric regression, to the transformation of the dependent variable in non-linear regression and to density estimation.

Local maximum likelihood estimation and inference

Abstract: Local maximum likelihood estimation is a nonparametric counterpart of the widely used parametric maximum likelihood technique. It extends the scope of the parametric maximum likelihood method to a much wider class of parametric spaces. Associated with this nonparametric estimation scheme is the issue of bandwidth selection and bias and variance assessment. This paper provides a unified approach to selecting a bandwidth and constructing confidence intervals in local maximum likelihood estimation. The approach is then applied to least squares nonparametric regression and to nonparametric logistic regression. Our experiences in these two settings show that the general idea outlined here is powerful and encouraging.

Mixed effects smoothing spline analysis of variance

Abstract: We propose a general family of nonparametric mixed effects models. Smoothing splines are used to model the fixed effects and are estimated by maximizing the penalized likelihood function. The random effects are generic and are modelled parametrically by assuming that the covariance function depends on a parsimonious set of parameters. These parameters and the smoothing parameter are estimated simultaneously by the generalized maximum likelihood method. We derive a connection between a nonparametric mixed effects model and a linear mixed effects model. This connection suggests a way of fitting a nonparametric mixed effects model by using existing programs. The classical two-way mixed models and growth curve models are used as examples to demonstrate how to use smoothing spline analysis-of-variance decompositions to build nonparametric mixed effects models. Similarly to the classical analysis of variance, components of these nonparametric mixed effects models can be interpreted as main effects and interactions. The penalized likelihood estimates of the fixed effects in a two-way mixed model are extensions of James-Stein shrinkage estimates to correlated observations. In an example three nested nonparametric mixed effects models are fitted to a longitudinal data set.

Estimating the variance in nonparametric regression—what is a reasonable choice?

Abstract: The exact mean-squared error (MSE) of estimators of the variance in nonparametric regression based on quadratic forms is investigated. In particular, two classes of estimators are compared: Hall, Kay and Titterington's optimal difference-based estimators and a class of ordinary difference-based estimators which generalize methods proposed by Rice and Gasser, Sroka and Jennen-Steinmetz. For small sample sizes the MSE of the first estimator is essentially increased by the magnitude of the integrated first two squared derivatives of the regression function. It is shown that in many situations ordinary difference-based estimators are more appropriate for estimating the variance, because they control the bias much better and hence have a much better overall performance. It is also demonstrated that Rice's estimator does not always behave well. Data-driven guidelines are given to select the estimator with the smallest MSE.

Triple‐goal estimates in two‐stage hierarchical models

Abstract: The beauty of the Bayesian approach is its ability to structure complicated models, inferential goals and analyses. To take full advantage of it, methods should be linked to an inferential goal via a loss function. For example, in the two-stage, compound sampling model the posterior means are optimal under squared error loss. However, they can perform poorly in estimating the histogram of the parameters or in ranking them. ‘Triple-goal’ estimates are motivated by the desire to have a set of estimates that produce good ranks, a good parameter histogram and good co-ordinate-specific estimates. No set of estimates can simultaneously optimize these three goals and we seek a set that strikes an effective trade-off. We evaluate and compare three candidate approaches: the posterior means, the constrained Bayes estimates of Louis and Ghosh, and a new approach that optimizes estimation of the histogram and the ranks. Mathematical and simulation-based analyses support the superiority of the new approach and document its excellent performance for the three inferential goals.

Bias-corrected confidence bands in nonparametric regression

Abstract: Summary. Bias-corrected confidence bands for general nonparametric regression models are considered. We use local polynomial fitting to construct the confidence bands and combine the cross-validation method and the plug-in method to select the bandwidths. Related asymptotic results are obtained. Our simulations show that confidence bands constructed by local polynomial fitting have much better coverage than those constructed by using the Nadaraya‐Watson estimator. The results are also applicable to nonparametric autoregressive time series models.

Testing heteroscedasticity in nonparametric regression

Abstract: The importance of being able to detect heteroscedasticity in regression is widely recognized because efficient inference for the regression function requires that heteroscedasticity is taken into account. In this paper a simple consistent test for heteroscedasticity is proposed in a nonparametric regression set-up. The test is based on an estimator for the best L 2 -approximation of the variance function by a constant. Under mild assumptions asymptotic normality of the corresponding test statistic is established even under arbitrary fixed alternatives. Confidence intervals are obtained for a corresponding measure of heteroscedasticity. The finite sample performance and robustness of these procedures are investigated in a simulation study and Box-type corrections are suggested for small sample sizes.

Nonparametric estimation of the mode of a distribution of random curves

Abstract: Summary. Motivated by the need to develop meaningful empirical approximations to a 'typical' data value, we introduce methods for density and mode estimation when data are in the form of random curves. Our approach is based on finite dimensional approximations via generalized Fourier expansions on an empirically chosen basis. The mode estimation problem is reduced to a problem of kernel-type multivariate estimation from vector data and is solved using a new recursive algorithm for finding the empirical mode. The algorithm may be used as an aid to the identification of clusters in a set of data curves. Bootstrap methods are employed to select the bandwidth.

Bivariate quantile smoothing splines

Abstract: It has long been recognized that the mean provides an inadequate summary whereas the set of quantiles can supply a more complete description of a sample. We introduce bivariate quantile smoothing splines, which belong to the space of bilinear tensor product splines, as nonparametric estimators for the conditional quantile functions in a two-dimensional design space. The estimators can be computed by using standard linear programming techniques and can further be used as building-blocks for conditional quantile estimations in higher dimensions. For moderately large data sets, we recommend penalized bivariate B-splines as approximate solutions. We use real and simulated data to illustrate the methodology proposed.

Fast EM-type implementations for mixed effects models

Abstract: The mixed effects model, in its various forms, is a common model in applied statistics. A useful strategy for fitting this model implements EM-type algorithms by treating the random effects as missing data. Such implementations, however, can be painfully slow when the variances of the random effects are small relative to the residual variance. In this paper, we apply the ‘working parameter’ approach to derive alternative EM-type implementations for fitting mixed effects models, which we show empirically can be hundreds of times faster than the common EM-type implementations. In our limited simulations, they also compare well with the routines in S-PLUS® and Stata® in terms of both speed and reliability. The central idea of the working parameter approach is to search for efficient data augmentation schemes for implementing the EM algorithm by minimizing the augmented information over the working parameter, and in the mixed effects setting this leads to a transfer of the mixed effects variances into the regression slope parameters. We also describe a variation for computing the restricted maximum likelihood estimate and an adaptive algorithm that takes advantage of both the standard and the alternative EM-type implementations.

Nonparametric validation of similar distributions and assessment of goodness of fit

Abstract: Summary. In this paper the problem of assessing the similarity of two cumulative distribution functions F and G is considered. An asymptotic test based on an (x-trimmed version of Mallows distance r,(F, G) between F and G is suggested, thus demonstrating the similarity of F and G within a preassigned r,(F, G) neighbourhood at a controlled type I error rate. The test proposed is applied to the validation of goodness of fit and for the nonparametric assessment of bioequivalence. It is shown that r,(F, G) can be interpreted as average and population equivalence. Our approach is illustrated by various examples.

Polynomial regression with errors in the variables

Abstract: A polynomial functional relationship with errors in both variables can be consistently estimated by constructing an ordinary least squares estimator for the regression coefficients, assuming hypothetically the latent true regressor variable to be known, and then adjusting for the errors. If normality of the error variables can be assumed, the estimator can be simplified considerably. Only the variance of the errors in the regressor variable and its covariance with the errors of the response variable need to be known. If the variance of the errors in the dependent variable is also known, another estimator can be constructed.

Some algebra and geometry for hierarchical models, applied to diagnostics

Abstract: Recent advances in computing make it practical to use complex hierarchical models. However, the complexity makes it difficult to see how features of the data determine the fitted model. This paper describes an approach to diagnostics for hierarchical models, specifically linear hierarchical models with additive normal or t-errors. The key is to express hierarchical models in the form of ordinary linear models by adding artificial `cases' to the data set corresponding to the higher levels of the hierarchy. The error term of this linear model is not homoscedastic, but its covariance structure is much simpler than that usually used in variance component or random effects models. The re-expression has several advantages. First, it is extremely general, covering dynamic linear models, random effect and mixed effect models, and pairwise difference models, among others. Second, it makes more explicit the geometry of hierarchical models, by analogy with the geometry of linear models. Third, the analogy with linear models provides a rich source of ideas for diagnostics for all the parts of hierarchical models. This paper gives diagnostics to examine candidate added variables, transformations, collinearity, case influence and residuals.

Calibrating the excess mass and dip tests of modality

Abstract: Summary. Nonparametric tests of modality are a distribution-free way of assessing evidence about inhomogeneity in a population, provided that the potential subpopulations are sufficiently well separated. They include the excess mass and dip tests, which are equivalent in univariate settings and are alternatives to the bandwidth test. Only very conservative forms of the excess mass and dip tests are available at present, however, and for that reason they are generally not competitive with the bandwidth test. In the present paper we develop a practical approach to calibrating the excess mass and dip tests to improve their level accuracy and power substantially. Our method exploits the fact that the limiting distribution of the excess mass statistic under the null hypothesis depends on unknowns only through a constant, which may be estimated. Our calibrated test exploits this fact and is shown to have greater power and level accuracy than the bandwidth test has. The latter tends to be quite conservative, even in an asymptotic sense. Moreover, the calibrated test avoids difficulties that the bandwidth test has with spurious modes in the tails, which often must be discounted through subjective intervention of the experimenter.

Robust models in probability sampling

Abstract: In the estimation of a population mean or total from a random sample, certain methods based on linear models are known to be automatically design consistent, regardless of how well the underlying model describes the population. A sufficient condition is identified for this type of robustness to model failure; the condition, which we call ‘internal bias calibration’, relates to the combination of a model and the method used to fit it. Included among the internally bias-calibrated models, in addition to the aforementioned linear models, are certain canonical link generalized linear models and nonparametric regressions constructed from them by a particular style of local likelihood fitting. Other models can often be made robust by using a suboptimal fitting method. Thus the class of model-based, but design consistent, analyses is enlarged to include more realistic models for certain types of survey variable such as binary indicators and counts. Particular applications discussed are the estimation of the size of a population subdomain, as arises in tax auditing for example, and the estimation of a bootstrap tail probability.

Analysis of longitudinal binary data from multiphase sampling

Abstract: The efficient use of surrogate or auxiliary information has been investigated within both model-based and design-based approaches to data analysis, particularly in the context of missing data. Here we consider the use of such data in epidemiological studies of disease incidence in which surrogate measures of disease status are available for all subjects at two time points, but definitive diagnoses are available only in stratified subsamples. We briefly review methods for the analysis of two-phase studies of disease prevalence at a single time point, and we discuss the extension of four of these methods to the analysis of incidence studies. Their performance is compared with special reference to a study of the incidence of senile dementia.

Model-based inference for categorical survey data subject to non-ignorable non-response

Abstract: We consider non-response models for a single categorical response with categorical covariates whose values are always observed. We present Bayesian methods for ignorable models and a particular non-ignorable model, and we argue that standard methods of model comparison are inappropriate for comparing ignorable and non-ignorable models. Uncertainty about ignorability of non-response is incorporated by introducing parameters describing the extent of non-ignorability into a pattern mixture specification and integrating over the prior uncertainty associated with these parameters. Our approach is illustrated using polling data from the 1992 British general election panel survey. We suggest sample size adjustments for surveys when non-ignorable non-response is expected.

Residuals for the linear model with general covariance structure

Abstract: A general theory is presented for residuals from the general linear model with correlated errors. It is demonstrated that there are two fundamental types of residual associated with this model, referred to here as the marginal and the conditional residual. These measure respectively the distance to the global aspects of the model as represented by the expected value and the local aspects as represented by the conditional expected value. These residuals may be multivariate. Some important dualities are developed which have simple implications for diagnostics. The results are illustrated by reference to model diagnostics in time series and in classical multivariate analysis with independent cases.

A class of local likelihood methods and near-parametric asymptotics

Abstract: observations in the neighbourhood of t. The paper studies the sense in which f (t, ^ t ) is closer to the true distribution g(t) than the usual estimate f (t, ^) is. Asymptotic results are presented for the case in which the model misspecification becomes vanishingly small as the sample size tends to1 .I n this setting, the relative entropy risk of the local method is better than that of maximum likelihood. The form of optimum weights for the local likelihood is obtained and illustrated for the normal distribution.

Bayesian versus frequentist measures of error in small area estimation

Abstract: Summary. This paper compares analytically and empirically the frequentist and Bayesian measures of error in small area estimation. The model postulated is the nested error regression model which allows for random small area effects to represent the joint effect of small area characteristics that are not accounted for by the fixed regressor variables. When the variance components are known, then, under a uniform prior for the regression coefficients and normality of the error terms, the frequentist and the Bayesian approaches yield the same predictors and prediction mean-squared errors (MSEs) (defined accordingly). When the variance components are unknown, it is common practice to replace the unknown variances by sample estimates in the expressions for the optimal predictors, so that the resulting empirical predictors remain the same under the two approaches. The use of this paradigm requires, however, modifications to the expressions of the prediction MSE to account for the extra variability induced by the need to estimate the variance components. The main focus of this paper is to review and compare the modifications to prediction MSEs proposed in the literature under the two approaches, with special emphasis on the orders of the bias of the resulting approximations to the true MSEs. Some new approximations based on Monte Carlo simulation are also proposed and compared with the existing methods. The advantage of these approximations is their simplicity and generality. Finite sample frequentist properties of the various methods are explored by a simulation study. The main conclusions of this study are that the use of second-order bias corrections generally yields better results in terms of the bias of the MSE approximations and the coverage properties of confidence intervals for the small area means. The Bayesian methods are found to have good frequentist properties, but they can be inferior to the frequentist methods. The second-order approximations under both approaches have, however, larger variances than the corresponding first-order approximations which in most cases result in higher MSEs of the MSE approximations.

Testing for pairwise serial independence via the empirical distribution function

Abstract: Built on Skaug and Tjostheim's approach, this paper proposes a new test for serial independence by comparing the pairwise empirical distribution functions of a time series with the products of its marginals for various lags, where the number of lags increases with the sample size and different lags are assigned different weights. Typically, the more recent information receives a larger weight. The test has some appealing attributes. It is consistent against all pairwise dependences and is powerful against alternatives whose dependence decays to zero as the lag increases. Although the test statistic is a weighted sum of degenerate Cramer–von Mises statistics, it has a null asymptotic N(0, 1) distribution. The test statistic and its limit distribution are invariant to any order preserving transformation. The test applies to time series whose distributions can be discrete or continuous, with possibly infinite moments. Finally, the test statistic only involves ranking the observations and is computationally simple. It has the advantage of avoiding smoothed nonparametric estimation. A simulation experiment is conducted to study the finite sample performance of the proposed test in comparison with some related tests.

An optimization interpretation of integration and back‐fitting estimators for separable nonparametric models

Abstract: We provide an optimization interpretation of both back-fitting and integration estimators for additive nonparametric regression. We find that the integration estimator is a projection with respect to a product measure. We also provide further understanding of the back-fitting method.

Boundary detection through dynamic polygons

Abstract: A method for the Bayesian restoration of noisy binary images portraying an object with constant grey level on a background is presented. The restoration, performed by fitting a polygon with any number of sides to the object's outline, is driven by a new probabilistic model for the generation of polygons in a compact subset of R 2 , which is used as a prior distribution for the polygon. Some measurability issues raised by the correct specification of the model are addressed. The simulation from the prior and the calculation of the a posteriori mean of grey levels are carried out through reversible jump Markov chain Monte Carlo computation, whose implementation and convergence properties are also discussed. One example of restoration of a synthetic image is presented and compared with existing pixel-based methods.

On an adaptive transformation–retransformation estimate of multivariate location

Abstract: An affine equivariant estimate of multivariate location based on an adaptive transformation and retransformation approach is studied. The work is primarily motivated by earlier work on different versions of the multivariate median and their properties. We explore an issue related to efficiency and equivariance that was originally raised by Bickel and subsequently investigated by Brown and Hettmansperger. Our estimate has better asymptotic performance than the vector of co-ordinatewise medians when the variables are substantially correlated. The finite sample performance of the estimate is investigated by using Monte Carlo simulations. Some examples are presented to demonstrate the effect of the adaptive transformation-retransformation strategy in the construction of multivariate location estimates for real data.