Scikit-learn is a Python module integrating a wide range of state-of-the-art machine learning algorithms for medium-scale supervised and unsupervised problems. This package focuses on bringing machine learning to non-specialists using a general-purpose high-level language. Emphasis is put on ease of use, performance, documentation, and API consistency. It has minimal dependencies and is distributed under the simplified BSD license, encouraging its use in both academic and commercial settings. Source code, binaries, and documentation can be downloaded from http://scikit-learn.sourceforge.net.

/pdf/scikit-learn-machine-learning-in-python-23clfh9q83.pdf

Scikit-learn: Machine Learning in Python

In comparative high-throughput sequencing assays, a fundamental task is the analysis of count data, such as read counts per gene in RNA-seq, for evidence of systematic changes across experimental conditions. Small replicate numbers, discreteness, large dynamic range and the presence of outliers require a suitable statistical approach. We present DESeq2, a method for differential analysis of count data, using shrinkage estimation for dispersions and fold changes to improve stability and interpretability of estimates. This enables a more quantitative analysis focused on the strength rather than the mere presence of differential expression. The DESeq2 package is available at http://www.bioconductor.org/packages/release/bioc/html/DESeq2.html
 .

/pdf/moderated-estimation-of-fold-change-and-dispersion-for-rna-3ywnbkcan0.pdf

Moderated estimation of fold change and dispersion for RNA-seq data with DESeq2

This paper presents a method for extracting distinctive invariant features from images that can be used to perform reliable matching between different views of an object or scene. The features are invariant to image scale and rotation, and are shown to provide robust matching across a substantial range of affine distortion, change in 3D viewpoint, addition of noise, and change in illumination. The features are highly distinctive, in the sense that a single feature can be correctly matched with high probability against a large database of features from many images. This paper also describes an approach to using these features for object recognition. The recognition proceeds by matching individual features to a database of features from known objects using a fast nearest-neighbor algorithm, followed by a Hough transform to identify clusters belonging to a single object, and finally performing verification through least-squares solution for consistent pose parameters. This approach to recognition can robustly identify objects among clutter and occlusion while achieving near real-time performance.

/pdf/distinctive-image-features-from-scale-invariant-keypoints-3waurjqzke.pdf

Distinctive Image Features from Scale-Invariant Keypoints

LIBSVM is a library for Support Vector Machines (SVMs). We have been actively developing this package since the year 2000. The goal is to help users to easily apply SVM to their applications. LIBSVM has gained wide popularity in machine learning and many other areas. In this article, we present all implementation details of LIBSVM. Issues such as solving SVM optimization problems theoretical convergence multiclass classification probability estimates and parameter selection are discussed in detail.

/pdf/libsvm-a-library-for-support-vector-machines-2gdxasg9zv.pdf

LIBSVM: A library for support vector machines

SUMMARY We propose a new method for estimation in linear models. The 'lasso' minimizes the residual sum of squares subject to the sum of the absolute value of the coefficients being less than a constant. Because of the nature of this constraint it tends to produce some coefficients that are exactly 0 and hence gives interpretable models. Our simulation studies suggest that the lasso enjoys some of the favourable properties of both subset selection and ridge regression. It produces interpretable models like subset selection and exhibits the stability of ridge regression. There is also an interesting relationship with recent work in adaptive function estimation by Donoho and Johnstone. The lasso idea is quite general and can be applied in a variety of statistical models: extensions to generalized regression models and tree-based models are briefly described.

/pdf/regression-shrinkage-and-selection-via-the-lasso-6mx35txfr5.pdf

Regression Shrinkage and Selection via the Lasso

For such a simple algorithm, it is fascinating and remarkable what a rich diversity of interpretations, views, perspectives and explanations have emerged of AdaBoost. Originally, AdaBoost was proposed as a “boosting” algorithm in the technical sense of the word: given access to “weak” classifiers, just slightly better in performance than random guessing, and given sufficient data, a true boosting algorithm can provably produce a combined classifier with nearly perfect accuracy (Freund and Schapire, 1997). AdaBoost has this property, but it also has been shown to be deeply connected with a surprising range of other topics, such as game theory, on-line learning, linear programming, logistic regression and maximum entropy (Breiman, 1999; Collins et al., 2002; Demiriz et al., 2002; Freund and Schapire, 1996, 1997; Kivinen and Warmuth, 1999; Lebanon and Lafferty, 2002). As we discuss further below, AdaBoost can been seen as a method for maximizing the “margins” or confidences of the predictions made by its generated classifier (Schapire et al., 1998). The current paper by Mease and Wyner, of course, focuses on another perspective, the so-called statistical view of boosting. This interpretation, particularly as expounded by Friedman et al. (2000), focuses on the algorithm as a stagewise procedure for minimizing the exponential loss function, which is related to the loss minimized in logistic regression, and whose minimization can be viewed, in a certain sense, as providing estimates of the conditional probability of the label. Taken together, these myriad interpretations of AdaBoost form a robust theory of the algorithm that provides understanding from an extraordinary range of points of view in which each perspective tells us something unique about the algorithm. The statistical view, for instance, has been of tremendous value, allowing for the practical conversion of AdaBoost’s predictions into conditional probabilities, as well as the algorithm’s generalization and extension to many other loss functions and learning problems. Still, each perspective has its weaknesses, which are important to identify to keep our theory in touch with reality. The current paper is superb in exposing empirical phenomena that are apparently difficult to understand according to the statistical view. From a theoretical perspective, the statistical interpretation has other weaknesses. As discussed by Mease and Wyner, this interpretation does not explain AdaBoost’s observed tendency not to overfit, particularly in the absence of regularization

/pdf/response-to-mease-and-wyner-evidence-contrary-to-the-2348ozuigg.pdf

Response to Mease and Wyner, Evidence Contrary to the Statistical View of Boosting, JMLR 9:131-156, 2008

Author(s): Davis, Philip J.; Alston-Garnjost, Margaret; Barbaro-Galtieri, Angela.; Flatte, Stanley M.; Friedman, Jerome H.; Lynch, Gerald R.; Rabin, Monroe S.; Solmitz, Frank T.

/pdf/investigation-of-low-mass-k-pi-pi-systems-in-12-gev-c-k-sup-14lbcpc3q5.pdf

INVESTIGATION OF LOW-MASS K$pi$$pi$ SYSTEMS IN 12-GeV/c K$sup +$p INTERACTIONS.

Several new model independent techniques for the analysis of multidimensional data are presented and applied to exclusive multiparticle production. For the reaction pp-+pprr'n+~c-~from 12 to 28 GeV/c, an algorithm that directly compares two multidimensional point distributions is used to show that the shape of d120/d($?)12 is energy independent while d60/dx6 (x=p,,/pmax) varies dramatically with beam energy. A multidimensional test for independence is used to show that the multivariate differential cross-section approximately factors into its cylindrical momentum components, d180/d(a18 r d%/dp;, ' d60/d(p12+ * d60/d$. The shape of d60/dcp6 i 6 also shown to be compatible with that predicted solely by kinematics. (Submitted to The Physical Review) *Work supported by the U.S. Atomic Energy Commission. Multiparticle production presents a difficult problem in data analysis due to the large number of independent observables necessary to completely describe the data. Excluding spin information, an n-particle final state requires jn-4 independent measurables for a complete description. The normalized multivariate differential cross section can be thought of as a probability density function in the chosen measurables, x x 1 2 ' l ‘X3n-4’ defined over their physically allowed values. Each event can be represented as a point in the multidimensional space . whose coordinates are the measurables of the event. This point contains all the information in the event and, thus, the collection or swarm of experimental points in this space contains all the information from the experiment. The purpose of data analysis is to infer the properties of the unknown probability density function (p.d.f.) from the experimental point sample in the multidimensional space. For exclusive experiments where all 3n-4 independent observables are measured, this p.d.f. is directly related to the transition matrix element squared for the reaction. This report describes the results of applying several newly developed nonparametric (model independent) techniques for investigating the properties of multidimensional point swarms 1,2 to the reaction pp+pp~+~+~r-lrfrom from 12 to 28 GeV/c.3 An algorithm that directly compares the shapes of two multidimensional point distributions is used to study the energy dependence of the multivariate differential cross-section, while a multivariate independence test is used to study its factorization properties.

Measurement of multivariate scaling and factorization in exclusive multiparticle production

rCOSA is a software package interfaced to the R language. It implements statistical techniques for clustering objects on subsets of attributes in multivariate data. The main output of COSA is a dissimilarity matrix that one can subsequently analyze with a variety of proximity analysis methods. Our package extends the original COSA software (Friedman and Meulman, 2004) by adding functions for hierarchical clustering methods, least squares multidimensional scaling, partitional clustering, and data visualization. In the many publications that cite the COSA paper by Friedman and Meulman (2004), the COSA program is actually used only a small number of times. This can be attributed to the fact that this original implementation is not very easy to install and use. Moreover, the available software is out-of-date. Here, we introduce an up-to-date software package and a clear guidance for this advanced technique. The software package and related links are available for free at: https://github.com/mkampert/rCOSA
 .

https://link.springer.com/content/pdf/10.1007%2Fs00357-017-9240-z.pdf

Jerome H. Friedman

Papers

Response to Mease and Wyner, Evidence Contrary to the Statistical View of Boosting, JMLR 9:131-156, 2008

INVESTIGATION OF LOW-MASS K$pi$$pi$ SYSTEMS IN 12-GeV/c K$sup +$p INTERACTIONS.

Measurement of multivariate scaling and factorization in exclusive multiparticle production

rCOSA: A Software Package for Clustering Objects on Subsets of Attributes

[Flexible Parsimonious Smoothing and Additive Modeling]: Response