Showing papers on "Resampling published in 2002"

A prediction-based resampling method for estimating the number of clusters in a dataset

Abstract: Microarray technology is increasingly being applied in biological and medical research to address a wide range of problems, such as the classification of tumors. An important statistical problem associated with tumor classification is the identification of new tumor classes using gene-expression profiles. Two essential aspects of this clustering problem are: to estimate the number of clusters, if any, in a dataset; and to allocate tumor samples to these clusters, and assess the confidence of cluster assignments for individual samples. Here we address the first of these problems. We have developed a new prediction-based resampling method, Clest, to estimate the number of clusters in a dataset. The performance of the new and existing methods were compared using simulated data and gene-expression data from four recently published cancer microarray studies. Clest was generally found to be more accurate and robust than the six existing methods considered in the study. Focusing on prediction accuracy in conjunction with resampling produces accurate and robust estimates of the number of clusters.

Varying‐coefficient models and basis function approximations for the analysis of repeated measurements

Abstract: SUMMARY A global smoothing procedure is developed using basis function approximations for estimating the parameters of a varying-coefficient model with repeated measurements. Inference procedures based on a resampling subject bootstrap are proposed to construct confidence regions and to perform hypothesis testing. Conditional biases and variances of our estimators and their asymptotic consistency are developed explicitly. Finite sample properties of our procedures are investigated through a simulation study. Application of the proposed approach is demonstrated through an example in epidemiology. In contrast to the existing methods, this approach applies whether or not the covariates are timeinvariant and does not require binning of the data when observations are sparse at distinct observation times.

Data analytic methods for the analysis of multilevel questions: A comparison of intraclass correlation coefficients, rwg(j), hierarchical linear modeling, within- and between-analysis, and random group resampling

Abstract: Researchers investigating organizations and leadership in particular are increasingly being called upon to theorize multilevel models and to utilize multilevel data analytic techniques. However, the literature provides relatively little guidance for researchers to identify which of the multilevel methodologies are appropriate for their particular questions. In this final article, the statistical procedures used in the multilevel data analyses in the previous articles of this special issue are compared. Specifically, intraclass correlation coefficients (ICCs), rwg(j), hierarchical linear modeling (HLM), within- and between-analysis (WABA), and random group resampling (RGR) are examined and their results and conclusions discussed. Following comparisons of these methods, recommendations for their use are presented.

Permutation tests for equality of distributions in high‐dimensional settings

Abstract: Motivated by applications in high-dimensional settings, we suggest a test of the hypothesis H-0 that two sampled distributions are identical. It is assumed that two independent datasets are drawn from the respective populations, which may be very general. In particular, the distributions may be multivariate or infinite-dimensional, in the latter case representing, for example, the distributions of random functions from one Euclidean space to another. Our test uses a measure of distance between data. This measure should be symmetric but need not satisfy the triangle inequality, so it is not essential that it be a metric. The test is based on ranking the pooled dataset, with respect to the distance and relative to any fixed data value, and repeating this operation for each fixed datum. A permutation argument enables a critical point to be chosen such that the test has concisely known significance level, conditional on the set of all pairwise distances.

A Resampling Approach to Cluster Validation

Abstract: The concept of cluster stability is introduced as a means for assessing the validity of data partitionings found by clustering algorithms. It allows us to explicitly quantify the quality of a clustering solution, without being dependent on external information. The principle of maximizing the cluster stability can be interpreted as choosing the most self-consistent data partitioning. We present an empirical estimator for the theoretically derived stability index, based on imitating independent sample-sets by way of resampling. Experiments on both toy-examples and real-world problems effectively demonstrate that the proposed validation principle is highly suited for model selection.

A wavelet-based procedure for process fault detection

Abstract: To detect faults in a time-dependent process, we apply a discrete wavelet transform (DWT) to several independently replicated data sets generated by that process. The DWT can capture irregular data patterns such as sharp "jumps" better than the Fourier transform and standard statistical procedures without adding much computational complexity. Our wavelet coefficient selection method effectively balances model parsimony against data reconstruction error. The few selected wavelet coefficients serve as the "reduced-size" data set to facilitate an efficient decision-making method in situations with potentially large-volume data sets. We develop a general procedure to detect process faults based on differences between the reduced-size data sets obtained from the nominal (in-control) process and from a new instance of the target process that must be tested for an out-of-control condition. The distribution of the test statistic is constructed first using normal distribution theory and then with a new resampling procedure called "reversed jackknifing" that does not require any restrictive distributional assumptions. A Monte Carlo study demonstrates the effectiveness of these procedures. Our methods successfully detect process faults for quadrupole mass spectrometry samples collected from a rapid thermal chemical vapor deposition process.

Generalized Method of Moments, Efficient Bootstrapping, and Improved Inference

Abstract: Generalized method of moments (GMM) has been an important innovation in econometrics. Its usefulness has motivated a search for good inference procedures based on GMM. This article presents a novel method of bootstrapping for GMM based on resampling from the empirical likelihood distribution that imposes the moment restrictions. We show that this approach yields a large-sample improvement and is efficient, and give examples. We also discuss the development of GMM and other recent work on improved inference.

A resampling approach to estimate the stability of one-dimensional or multidimensional independent components

Abstract: When applying unsupervised learning techniques in biomedical data analysis, a key question is whether the estimated parameters of the studied system are reliable. In other words, can we assess the quality of the result produced by our learning technique? We propose resampling methods to tackle this question and illustrate their usefulness for blind-source separation (BSS). We demonstrate that our proposed reliability estimation can be used to discover stable one-dimensional or multidimensional independent components, to choose the appropriate BSS-model, to enhance significantly the separation performance, and, most importantly, to flag components that carry physical meaning. Application to different biomedical testbed data sets (magnetoencephalography (MEG)/electrocardiography (ECG)-recordings) underline the usefulness of our approach.

An invariance principle for sieve bootstrap in time series

Abstract: This paper establishes an invariance principle applicable for the asymptotic analysis of sieve bootstrap in time series. The sieve bootstrap is based on the approximation of a linear process by a finite autoregressive process of order increasing with the sample size, and resampling from the approximated autoregression. In this context, we prove an invariance principle for the bootstrap samples obtained from the approximated autoregressive process. It is of the strong form and holds almost surely for all sample realizations. Our development relies upon the strong approximation and the Beveridge-Nelson representation of linear processes. For illustrative purposes, we apply our results and show the asymptotic validity of the sieve bootstrap for Dickey-Fuller unit root tests for the model driven by a general linear process with independent and identically distributed innovations. We thus provide a theoretical justification on the use of the bootstrap Dickey-Fuller tests for general unit root models, in place of the testing procedures by Said and Dickey and by Phillips.

Selecting the working correlation structure in generalized estimating equations with application to the lung health study

Abstract: The generalized estimating equation (GEE) approach is becoming more and more popular in handling correlated response data, for example in longitudi- nal studies. An attractive property of the GEE is that one can use some working correlation structure that may be wrong, but the resulting regression coefficient estimate is still consistent and asymptotically normal. One convenient choice is the independence model: treat the correlated responses as if they were independent. However with time-varying covariates there is a dilemma: using the independence model may be very inefficient (Fitzmaurice (1995)); using a non-diagonal working correlation matrix may violate an important assumption in GEE, producing biased estimates (Pepe and Anderson (1994)). It would be desirable to be able to distin- guish these two situations based on the data at hand. More generally, selecting an appropriate working correlation structure, as an aspect of model selection, may improve estimation efficiency. In this paper we propose some resampling-based methods (i.e., the bootstrap and cross-validation) to do this. The methodology is demonstrated by application to the Lung Health Study (LHS) data to investigate the effects of smoking cessation on lung function and on the symptom of chronic cough. In addition, Pepe and Anderson's result is verified using the LHS data.

Inconsistency of Resampling Algorithms for High-Breakdown Regression Estimators and a New Algorithm

Abstract: Because high-breakdown estimators (HBEs) are impractical to compute exactly in large samples, approximate algorithms are used. The algorithm generally produces an estimator with a lower consistency rate and breakdown value than the exact theoretical estimator. This discrepancy grows with the sample size, with the implication that huge computations are needed for good approximations in large high-dimension samples. The workhorse for HBEs has been the “elemental set,” or “basic resampling,” algorithm. This turns out to be completely ineffective in high dimensions with high levels of contamination. However, enriching it with a “concentration” step turns it into a method that can handle even high levels of contamination, provided that the regression outliers are located on random cases. It remains ineffective if the regression outliers are concentrated on high-leverage cases. We focus on the multiple regression problem, but several of the broad conclusions—notably, those of the inadequacy of fixed numbers of ...

Selection of Meta-parameters for Support Vector Regression

Abstract: We propose practical recommendations for selecting metaparameters for SVM regression (that is, ? -insensitive zone and regularization parameter C). The proposed methodology advocates analytic parameter selection directly from the training data, rather than resampling approaches commonly used in SVM applications. Good generalization performance of the proposed parameter selection is demonstrated empirically using several lowdimensional and high-dimensional regression problems. In addition, we compare generalization performance of SVM regression (with proposed choice?) with robust regression using 'least-modulus' loss function (? =0). These comparisons indicate superior generalization performance of SVM regression.

Bootstrapping Regression Models Appendix to An R and S-PLUS Companion to Applied Regression

Abstract: Bootstrapping is a general approach to statistical inference based on building a sampling distribution for a statistic by resampling from the data at hand. The term ‘bootstrapping,’ due to Efron (1979), is an allusion to the expression ‘pulling oneself up by one’s bootstraps’ – in this case, using the sample data as a population from which repeated samples are drawn. At first blush, the approach seems circular, but has been shown to be sound. Two S libraries for bootstrapping are associated with extensive treatments of the subject: Efron and Tibshirani’s (1993) bootstrap library, and Davison and Hinkley’s (1997) boot library. Of the two, boot, programmed by A. J. Canty, is somewhat more capable, and will be used for the examples in this appendix. There are several forms of the bootstrap, and, additionally, several other resampling methods that are related to it, such as jackknifing, cross-validation, randomization tests ,a ndpermutation tests. I will stress the nonparametric bootstrap. Suppose that we draw a sample S = {X1 ,X 2, ..., Xn} from a population P = {x1 ,x 2, ..., xN } ;i magine further, at least for the time being, that N is very much larger than n ,a nd thatS is either a simple random sample or an independent random sample from P; 1 I will briefly consider other sampling schemes at the end of the appendix. It will also help initially to think of the elements of the population (and, hence, of the sample) as scalar values, but they could just as easily be vectors (i.e., multivariate). Now suppose that we are interested in some statistic T = t(S) as an estimate of the corresponding population parameter θ = t(P). Again, θ could be a vector of parameters and T the corresponding vector of estimates, but for simplicity assume that θ is a scalar. A traditional approach to statistical inference is to make assumptions about the structure of the population (e.g., an assumption of normality), and, along with the stipulation of random sampling, to use these assumptions to derive the sampling distribution of T , on which classical inference is based. In certain instances, the exact distribution of T may be intractable, and so we instead derive its asymptotic distribution. This familiar approach has two potentially important deficiencies:

Estimating riverine fish population size from single- and multiple-pass removal sampling using a hierarchical model

Abstract: A hierarchical model is described for estimating population size from single- and multiple-pass removal sampling. The model is appropriate for two-stage sampling schemes, typified by surveys of riverine fish populations, in which multiple sites are surveyed, but a low number of passes are undertaken at each site. The model estimates the average population size within the target area from the raw catch data, and thus allows for differences in the sampling procedure at each site, such as including single-pass sampling. The model also uses the data from all sites to estimate the population size at each individual site. This results in generally improved precision for multiple-pass sites and provides comparable estimates from single-pass sites. A Bayesian approach is described for estimating the parameters of the hierarchical model using sampling importance resampling (SIR). An empirical Bayesian approach, which ignores prior uncertainty but is simpler to implement, is also described. Application of the hiera...

Design and analysis of quality of life studies in clinical trials : interdisciplinary statistics

Abstract: INTRODUCTION Health-Related Quality of Life Measuring Health-Related Quality of Life Example 1: Adjuvant Breast Cancer Trial Example 2: Advanced Non-Small-Cell Lung Cancer (NSCLC) Example 3: Renal Cell Carcinoma Trial Summary STUDY DESIGN AND PROTOCOL DEVELOPMENT Introduction Background and Rationale Research Objectives Selection of Subjects Longitudinal Designs Selection of a Quality of Life Measure Conduct Summary MODELS FOR LONGITUDINAL STUDIES Introduction Building the Analytic Models Building Repeated Measures Models Building Growth Curve Models Summary MISSING DATA Introduction Patterns of Missing Data Mechanisms of Missing Data Summary ANALYTIC METHODS FOR IGNORABLE MISSING DATA Introduction Repeated Univariate Analyses Multivariate Methods Baseline Assessment as a Covariate Change from Baseline Empirical Bayes Estimates Summary SIMPLE IMPUTATION Introduction Mean Value Substitution Explicit Regression Models Last Value Carried Forward Underestimation of Variance Sensitivity Analysis Summary MULTIPLE IMPUTATION Introduction Overview of Multiple Imputation Explicit Univariate Regression Closest Neighbor and Predictive Mean Matching Approximate Bayesian Bootstrap Multivariate Procedures for Nonmonotone Missing Data Combining the M Analyses Sensitivity Analyses Imputation vs. Analytic Models Implications for Design Summary PATTERN MIXTURE MODELS Introduction Bivariate Data (Two Repeated Measures) Monotone Dropout Parametric Models Additional Reading Algebraic Details Summary RANDOM-EFFECTS MIXTURE, SHARED-PARAMETER, AND SELECTION MODELS Introduction Conditional Linear Model Joint Mixed-Effects and Time to Dropout Selection Model for Monotone Dropout Advanced Readings Summary SUMMARY MEASURES Introduction Choosing a Summary Measure Constructing Summary Measures Summary Statistics across Time Summarizing Across HRQoL Domains or Subscales Advanced Notes Summary MULTIPLE ENDPOINTS Introduction Background Concepts and Definitions Multivariate Statistics Univariate Statistics Resampling Techniques Summary DESIGN: ANALYSIS PLANS Introduction General Analysis Plan Models for Longitudinal Data Multiplicity of Endpoints Sample Size and Power Reported Results Summary APPENDICES BIBLIOGRAPHY

A new sequential importance sampling method and its application to the two-dimensional hydrophobic-hydrophilic model

Abstract: The sequential importance sampling method and its various modifications have been developed intensively and used effectively in diverse research areas ranging from polymer simulation to signal processing and statistical inference. We propose a new variant of the method, sequential importance sampling with pilot-exploration resampling (SISPER), and demonstrate its successful application in folding polypeptide chains described by a two-dimensional hydrophobic-hydrophilic (HP) lattice model. We show by numerical results that SISPER outperformed several existing approaches, e.g., a genetic algorithm, the pruned-enriched Rosenbluth method, and the evolutionary Monte Carlo, in finding the ground folding states of 2D square-lattice HP sequences. In a few difficult cases, the new method can find the ground states without using any prior structural information on the chain. We also discuss the potential applications of SISPER in more general problems.

Empirical likelihood for autoregressive models, with applications to unstable time series

Abstract: Empirical likelihood is developed for autoregressive models with inno- vations that form a martingale difference sequence. Limiting distributions of the log empirical likelihood ratio statistic for both the stable and unstable cases are established. Behavior of the log empirical likelihood ratio statistic is considered in nearly nonstationary models to assess the local power of unit root tests and to construct confidence intervals. Resampling methods are proposed to improve the finite-sample performance of empirical likelihood statistics. This paper shows that empirical likelihood methodology compares favorably with existing methods and demonstrates its potential for time series with more general innovation structures.

Binary partitioning for continuous longitudinal data: categorizing a prognostic variable

Abstract: We investigate a binary partitioning algorithm in the case of a continuous repeated measures outcome. The procedure is based on the use of the likelihood ratio statistic to evaluate the performance of individual splits. The procedure partitions a set of longitudinal data into two mutually exclusive groups based on an optimal split of a continuous prognostic variable. A permutation test is used to assess the level of significance associated with the optimal split, and a bootstrap confidence interval is obtained for the optimal split.

The Fisher-Pitman permutation test: an attractive alternative to the F test.

Abstract: The Fisher-Pitman permutation test is shown to possess significant advantages over conventional alternatives when analyzing differences among independent samples with unequal variances.

Frequency Domain Bootstrap for Time Series

Abstract: The paper discusses frequency domain bootstrap methods for time series including some recent developments. Attention is focused on nonparametric resampling methods of the periodogram and their application to statistical inference in the frequency domain.

Very fast algorithms for evaluating the stability of ML and Bayesian phylogenetic trees from sequence data.

Abstract: Evolutionary trees sit at the core of all realistic models describing a set of related sequences, including alignment, homology search, ancestral protein reconstruction and 2D/3D structural change. It is important to assess the stochastic error when estimating a tree, including models using the most realistic likelihood-based optimizations, yet computation times may be many days or weeks. If so, the bootstrap is computationally prohibitive. Here we show that the extremely fast \resampling of estimated log likelihoods" or RELL method behaves well under more general circumstances than previously examined. RELL approximates the bootstrap (BP) proportions of trees better that some bootstrap methods that rely on fast heuristics to search the tree space. The BIC approximation of the Bayesian posterior probability (BPP) of trees is made more accurate by including an additional term related to the determinant of the information matrix (which may also be obtained as a product of gradient or score vectors). Such estimates are shown to be very close to MCMC chain values. Our analysis of mammalian mitochondrial amino acid sequences suggest that when model breakdown occurs, as it typically does for sequences separated by more than a few million years, the BPP values are far too peaked and the real uctuations in the likelihood of the data are many times larger than expected. Accordingly, several ways to incorporate the bootstrap and other types of direct resampling with MCMC procedures are outlined. Genes evolve by a process which involves some sites following a tree close to, but not identical with, the species tree. It is seen that under such a likelihood model BP (bootstrap proportions) and BPP estimates may still be reasonable estimates of the species tree. Since many of the methods studied are very fast computationally, there is no reason to ignore stochastic error even with the slowest ML or likelihood based methods.

Bump-hunting for the proficiency tester--searching for multimodality.

Abstract: Kernel density estimation is a method for producing a smooth density approximation to a dataset and avoiding some of the problems associated with histograms. If it is used with a degree of smoothing determined by a fitness for purpose criterion, it can be applied to proficiency test data in order to test for multimodality in the z-scores. The bootstrap is an essential additional technique to determine how rugged the initially estimated kernel density is: the random resampling of the data in the bootstrap simulates a complete blind repeat of the proficiency test. In addition, useful estimates of the standard error of a mode can be thus obtained. It is suggested that a mode and its standard error can be used as an assigned value and its standard uncertainty.

Penalized partial likelihood regression for right-censored data with bootstrap selection of the penalty parameter.

Abstract: The Cox proportional hazards model is often used for estimating the association between covariates and a potentially censored failure time, and the corresponding partial likelihood estimators are used for the estimation and prediction of relative risk of failure. However, partial likelihood estimators are unstable and have large variance when collinearity exists among the explanatory variables or when the number of failures is not much greater than the number of covariates of interest. A penalized (log) partial likelihood is proposed to give more accurate relative risk estimators. We show that asymptotically there always exists a penalty parameter for the penalized partial likelihood that reduces mean squared estimation error for log relative risk, and we propose a resampling method to choose the penalty parameter. Simulations and an example show that the bootstrap-selected penalized partial likelihood estimators can, in some instances, have smaller bias than the partial likelihood estimators and have smaller mean squared estimation and prediction errors of log relative risk. These methods are illustrated with a data set in multiple myeloma from the Eastern Cooperative Oncology Group.

Application of resampling techniques to estimate exact significance levels for covariate selection during nonlinear mixed effects model building: some inferences.

Abstract: Purpose. One of the main objectives of the nonlinear mixed effects modeling is to provide rational individualized dosing strategies by explaining the interindividual variability using intrinsic and/or extrinsic factors (covariates). The aim of the current study was to evaluate, using computer simulations and real data, methods for estimating the exact significance level for including or excluding a covariate during model building.

A Methodology for Neural Spatial Interaction Modeling

Abstract: This paper attempts to develop a mathematically rigid and unified framework for neural spatial interaction modeling. Families of classical neural network models, but also less classical ones such as product unit neural network ones are considered for the cases of unconstrained and singly constrained spatial interaction flows. Current practice appears to suffer from least squares and normality assumptions that ignore the true integer nature of the flows and approximate a discrete-valued process by an almost certainly misrepresentative continuous distribution. To overcome this deficiency we suggest a more suitable estimation approach, maximum likelihood estimation under more realistic distributional assumptions of Poisson processes, and utilize a global search procedure, called Alopex, to solve the maximum likelihood estimation problem. To identify the transition from underfitting to overfitting we split the data into training, internal validation and test sets. The bootstrapping pairs approach with replacement is adopted to combine the purity of data splitting with the power of a resampling procedure to overcome the generally neglected issue of fixed data splitting and the problem of scarce data. In addition, the approach has power to provide a better statistical picture of the prediction variability, Finally, a benchmark comparison against the classical gravity models illustrates the superiority of both, the unconstrained and the origin constrained neural network model versions in terms of generalization performance measured by Kullback and Leibler's information criterion.