Resampling

About: Resampling is a research topic. Over the lifetime, 5428 publications have been published within this topic receiving 242291 citations.

Parametric and Nonparametric Methods for the Statistical Evaluation of Human ID Algorithms

Abstract: This paper reviews some of the major issues associated with the statistical evaluation of Human Identification algorithms, emphasizing comparisons between algorithms on the same set of sample images. A general notation is developed and common performance metrics are defined. A simple success/failure evaluation methodology where recognition rate depends upon a binomially distributed random variable, recognition count, is developed and the conditions under which this model is appropriate are discussed. Some nonparametric techniques are also introduced, including bootstrapping. When applied to estimating the distribution of recognition count for a single set of i.i.d. sampled probe images, bootstrapping is noted as equivalent to the parametric binomial model. Bootstrapping applied to recognition rate over resampled sets of images can be problematic. Specifically, sampling with replacement to form image probe sets is shown to introduce a conflict between assumptions required by bootstrapping and the way recognition rate is computed. In part to overcome this difficulty with bootstrapping, a different nonparametric Monte Carlo method is introduced, and its utility illustrated with an extended example. This method permutes the choice of gallery and probe images. It is used to answer two questions. Question 1: How much does recognition rate vary when comparing images of individuals taken on different days using the same camera? Question 2: When is the observed difference in recognition rates for two distinct algorithms significant relative to this variation? Two important general features of nonparametric methods are illustrated by the Monte Carlo study. First, within some broad limits, resampling generates sample distributions for any statistic of interest. Second, through careful choice of an appropriate statistic and subsequent estimation of its distribution, domain specific hypotheses may be readily formulated and tested.

Density Estimation under Constraints

Abstract: We suggest a general method for tackling problems of density estimation under constraints. It is, in effect, a particular form of the weighted bootstrap, in which resampling weights are chosen so as to minimize distance from the empirical or uniform bootstrap distribution subject to the constraints being satisfied. A number of constraints are treated as examples. They include conditions on moments, quantiles, and entropy, the latter as a device for imposing qualitative conditions such as those of unimodality or “interestingness.” For example, without altering the data or the amount of smoothing, we may construct a density estimator that enjoys the same mean, median, and quartiles as the data. Different measures of distance·give rise to slightly different results.

Tests of geographical correlation with adjustment for explanatory variables: an application to dyspnoea in the elderly.

Abstract: We propose a test of correlation of the residuals in generalized linear models which is a generalization of the spatial autocorrelation test based on Moran's I It allows adjustment for sizes of geographical areas and for explanatory variables A formula is given to compute the weights according to the alternative hypothesis We compare inference using the distribution in the model and using the permutation distribution A simulation study showed that the model-based test may be very conservative and this leads to a loss of power compared to the permutation test or to the model-based test with correction for estimated parameters As this latter is intractable for very large samples when the model includes explanatory variables, we recommend the use of the permutation test The permutation test is used to study geographical correlation of dyspnoea in the elderly