A general non-parametric technique is proposed for the analysis of a complex multimodal feature space and to delineate arbitrarily shaped clusters in it. The basic computational module of the technique is an old pattern recognition procedure: the mean shift. For discrete data, we prove the convergence of a recursive mean shift procedure to the nearest stationary point of the underlying density function and, thus, its utility in detecting the modes of the density. The relation of the mean shift procedure to the Nadaraya-Watson estimator from kernel regression and the robust M-estimators; of location is also established. Algorithms for two low-level vision tasks discontinuity-preserving smoothing and image segmentation - are described as applications. In these algorithms, the only user-set parameter is the resolution of the analysis, and either gray-level or color images are accepted as input. Extensive experimental results illustrate their excellent performance.

Mean shift: a robust approach toward feature space analysis

An essential textbook for any student or researcher in biology needing to design experiments, sample programs or analyse the resulting data The text begins with a revision of estimation and hypothesis testing methods, covering both classical and Bayesian philosophies, before advancing to the analysis of linear and generalized linear models Topics covered include linear and logistic regression, simple and complex ANOVA models (for factorial, nested, block, split-plot and repeated measures and covariance designs), and log-linear models Multivariate techniques, including classification and ordination, are then introduced Special emphasis is placed on checking assumptions, exploratory data analysis and presentation of results The main analyses are illustrated with many examples from published papers and there is an extensive reference list to both the statistical and biological literature The book is supported by a website that provides all data sets, questions for each chapter and links to software

/pdf/experimental-design-and-data-analysis-for-biologists-8ybva8d53x.pdf

Experimental Design and Data Analysis for Biologists

We present VOSviewer, a freely available computer program that we have developed for constructing and viewing bibliometric maps. Unlike most computer programs that are used for bibliometric mapping, VOSviewer pays special attention to the graphical representation of bibliometric maps. The functionality of VOSviewer is especially useful for displaying large bibliometric maps in an easy-to-interpret way. The paper consists of three parts. In the first part, an overview of VOSviewer’s functionality for displaying bibliometric maps is provided. In the second part, the technical implementation of specific parts of the program is discussed. Finally, in the third part, VOSviewer’s ability to handle large maps is demonstrated by using the program to construct and display a co-citation map of 5,000 major scientific journals.

/pdf/software-survey-vosviewer-a-computer-program-for-505jc6bwmd.pdf

Software survey: VOSviewer, a computer program for bibliometric mapping.

A common concern when faced with multivariate data with missing values is whether the missing data are missing completely at random (MCAR); that is, whether missingness depends on the variables in the data set. One way of assessing this is to compare the means of recorded values of each variable between groups defined by whether other variables in the data set are missing or not. Although informative, this procedure yields potentially many correlated statistics for testing MCAR, resulting in multiple-comparison problems. This article proposes a single global test statistic for MCAR that uses all of the available data. The asymptotic null distribution is given, and the small-sample null distribution is derived for multivariate normal data with a monotone pattern of missing data. The test reduces to a standard t test when the data are bivariate with missing data confined to a single variable. A limited simulation study of empirical sizes for the test applied to normal and nonnormal data suggests th...

https://www.jstor.org/stable/pdfplus/2290157.pdf

A Test of Missing Completely at Random for Multivariate Data with Missing Values

Introduction to support vector learning roadmap. Part 1 Theory: three remarks on the support vector method of function estimation, Vladimir Vapnik generalization performance of support vector machines and other pattern classifiers, Peter Bartlett and John Shawe-Taylor Bayesian voting schemes and large margin classifiers, Nello Cristianini and John Shawe-Taylor support vector machines, reproducing kernel Hilbert spaces, and randomized GACV, Grace Wahba geometry and invariance in kernel based methods, Christopher J.C. Burges on the annealed VC entropy for margin classifiers - a statistical mechanics study, Manfred Opper entropy numbers, operators and support vector kernels, Robert C. Williamson et al. Part 2 Implementations: solving the quadratic programming problem arising in support vector classification, Linda Kaufman making large-scale support vector machine learning practical, Thorsten Joachims fast training of support vector machines using sequential minimal optimization, John C. Platt. Part 3 Applications: support vector machines for dynamic reconstruction of a chaotic system, Davide Mattera and Simon Haykin using support vector machines for time series prediction, Klaus-Robert Muller et al pairwise classification and support vector machines, Ulrich Kressel. Part 4 Extensions of the algorithm: reducing the run-time complexity in support vector machines, Edgar E. Osuna and Federico Girosi support vector regression with ANOVA decomposition kernels, Mark O. Stitson et al support vector density estimation, Jason Weston et al combining support vector and mathematical programming methods for classification, Bernhard Scholkopf et al.

Advances in kernel methods: support vector learning

1. Introduction.- 1.1 Smoothing Methods: a Nonparametric/Parametric Compromise.- 1.2 Uses of Smoothing Methods.- 1.3 Outline of the Chapters.- Background material.- Computational issues.- Exercises.- 2. Simple Univariate Density Estimation.- 2.1 The Histogram.- 2.2 The Frequency Polygon.- 2.3 Varying the Bin Width.- 2.4 The Effectiveness of Simple Density Estimators.- Background material.- Computational issues.- Exercises.- 3. Smoother Univariate Density Estimation.- 3.1 Kernel Density Estimation.- 3.2 Problems with Kernel Density Estimation.- 3.3 Adjustments and Improvements to Kernel Density Estimation.- 3.4 Local Likelihood Estimation.- 3.5 Roughness Penalty and Spline-Based Methods.- 3.6 Comparison of Univariate Density Estimators.- Background material.- Computational issues.- Exercises.- 4. Multivariate Density Estimation.- 4.1 Simple Density Estimation Methods.- 4.2 Kernel Density Estimation.- 4.3 Other Estimators.- 4.4 Dimension Reduction and Projection Pursuit.- 4.5 The State of Multivariate Density Estimation.- Background material.- Computational issues.- Exercises.- 5. Nonparametrie Regression.- 5.1 Scatter Plot Smoothing and Kernel Regression.- 5.2 Local Polynomial Regression.- 5.3 Bandwidth Selection.- 5.4 Locally Varying the Bandwidth.- 5.5 Outliers and Autocorrelation.- 5.6 Spline Smoothing.- 5.7 Multiple Predictors and Additive Models.- 5.8 Comparing Nonparametric Regression Methods.- Background material.- Computational issues.- Exercises.- 6. Smoothing Ordered Categorical Data.- 6.1 Smoothing and Ordered Categorical Data.- 6.2 Smoothing Sparse Multinomials.- 6.3 Smoothing Sparse Contingency Tables.- 6.4 Categorical Data, Regression, and Density Estimation.- Background material.- Computational issues.- Exercises.- 7. Further Applications of Smoothing.- 7.1 Discriminant Analysis.- 7.2 Goodness-of-Fit Tests.- 7.3 Smoothing-Based Parametric Estimation.- 7.4 The Smoothed Bootstrap.- Background material.- Computational issues.- Exercises.- Appendices.- A. Descriptions of the Data Sets.- B. More on Computational Issues.- References.- Author Index.

Smoothing Methods in Statistics

Summary. Many different methods have been proposed to construct nonparametric estimates of a smooth regression function, including local polynomial, (convolution) kernel and smoothing spline estimators. Each of these estimators uses a smoothing parameter to control the amount of smoothing performed on a given data set. In this paper an improved version of a criterion based on the Akaike information criterion (AIC), termed AICc, is derived and examined as a way to choose the smoothing parameter. Unlike plug-in methods, AICc can be used to choose smoothing parameters for any linear smoother, including local quadratic and smoothing spline estimators. The use of AICc avoids the large variability and tendency to undersmooth (compared with the actual minimizer of average squared error) seen when other 'classical' approaches (such as generalized cross-validation or the AIC) are used to choose the smoothing parameter. Monte Carlo simulations demonstrate that the AICc-based smoothing parameter is competitive with a plug-in method (assuming that one exists) when the plug-in method works well but also performs well when the plug-in approach fails or is unavailable.

https://www.stat.washington.edu/courses/stat527/s14/readings/j_royal_stat_soc_b1998.pdf

Smoothing parameter selection in nonparametric regression using an improved Akaike information criterion

Exploring and identifying structure is even more important for multivariate data than univariate data, given the difficulties in graphically presenting multivariate data and the comparative lack of parametric models to represent it. Unfortunately, such exploration is also inherently more difficult.

Multivariate Density Estimation

We consider the problem of identifying and testing multiple outliers in linear models. The available outlier identification methods often do not succeed in detecting multiple outliers because they are affected by the observations they are supposed to identify. We introduce two test procedures for the detection of multiple outliers that appear to be less sensitive to this problem. Both procedures attempt to separate the data into a set of “clean” data points and a set of points that contain the potential outliers. The potential outliers are then tested to see how extreme they are relative to the clean subset, using an appropriately scaled version of the prediction error. The procedures are illustrated and compared to various existing methods, using several data sets known to contain multiple outliers. Also, the performances of both procedures are investigated by a Monte Carlo study. The data sets and the Monte Carlo indicate that both procedures are effective in the detection of multiple outliers ...

https://www.jstor.org/stable/pdfplus/2291266

Procedures for the Identification of Multiple Outliers in Linear Models

Tree induction and logistic regression are two standard, off-the-shelf methodsfor building models for classification. We present a large-scale experimentalcomparison of logistic regression and tree induction, assessing classification accuracyand the quality of rankings based on class-membership probabilities. Weuse a learning-curve analysis to examine the relationship of these measures tothe size of the training set. The results of the study show several remarkablethings. (I) Contrary to prior observations, logistic regression does not generallyoutperform tree induction. (2) More specifically, and not surprisingly, logisticregression is better for smaller training sets and tree induction for larger datasets. Importantly, this often holds for training sets drawn from the same domain(i.e., the learning curves cross), so conclusions about induction-algorithmsuperiority on a given domain must be based on an analysis of the learningcurves. (3) Contrary to conventional wisdom, tree induction is effective at producingprobability-based rankings, although apparently comparatively less sofor a given training--set size than at making classifications. Finally, (4) the domainson which tree induction and logistic regression are ultimately preferablecan be characterized surprisingly well by a simple measure of signal-to-noiseratio.

https://papers.ssrn.com/sol3/Delivery.cfm/2451_14161.pdf?abstractid=1283003&mirid=1&type=2

Jeffrey S. Simonoff

Papers

Smoothing Methods in Statistics

Smoothing parameter selection in nonparametric regression using an improved Akaike information criterion

Multivariate Density Estimation

Procedures for the Identification of Multiple Outliers in Linear Models

Tree Induction Vs. Logistic Regression: a Learning-Curve Analysis