Scikit-learn is a Python module integrating a wide range of state-of-the-art machine learning algorithms for medium-scale supervised and unsupervised problems. This package focuses on bringing machine learning to non-specialists using a general-purpose high-level language. Emphasis is put on ease of use, performance, documentation, and API consistency. It has minimal dependencies and is distributed under the simplified BSD license, encouraging its use in both academic and commercial settings. Source code, binaries, and documentation can be downloaded from http://scikit-learn.sourceforge.net.

/pdf/scikit-learn-machine-learning-in-python-23clfh9q83.pdf

Scikit-learn: Machine Learning in Python

In comparative high-throughput sequencing assays, a fundamental task is the analysis of count data, such as read counts per gene in RNA-seq, for evidence of systematic changes across experimental conditions. Small replicate numbers, discreteness, large dynamic range and the presence of outliers require a suitable statistical approach. We present DESeq2, a method for differential analysis of count data, using shrinkage estimation for dispersions and fold changes to improve stability and interpretability of estimates. This enables a more quantitative analysis focused on the strength rather than the mere presence of differential expression. The DESeq2 package is available at http://www.bioconductor.org/packages/release/bioc/html/DESeq2.html
 .

/pdf/moderated-estimation-of-fold-change-and-dispersion-for-rna-3ywnbkcan0.pdf

Moderated estimation of fold change and dispersion for RNA-seq data with DESeq2

This paper presents a method for extracting distinctive invariant features from images that can be used to perform reliable matching between different views of an object or scene. The features are invariant to image scale and rotation, and are shown to provide robust matching across a substantial range of affine distortion, change in 3D viewpoint, addition of noise, and change in illumination. The features are highly distinctive, in the sense that a single feature can be correctly matched with high probability against a large database of features from many images. This paper also describes an approach to using these features for object recognition. The recognition proceeds by matching individual features to a database of features from known objects using a fast nearest-neighbor algorithm, followed by a Hough transform to identify clusters belonging to a single object, and finally performing verification through least-squares solution for consistent pose parameters. This approach to recognition can robustly identify objects among clutter and occlusion while achieving near real-time performance.

/pdf/distinctive-image-features-from-scale-invariant-keypoints-3waurjqzke.pdf

Distinctive Image Features from Scale-Invariant Keypoints

LIBSVM is a library for Support Vector Machines (SVMs). We have been actively developing this package since the year 2000. The goal is to help users to easily apply SVM to their applications. LIBSVM has gained wide popularity in machine learning and many other areas. In this article, we present all implementation details of LIBSVM. Issues such as solving SVM optimization problems theoretical convergence multiclass classification probability estimates and parameter selection are discussed in detail.

/pdf/libsvm-a-library-for-support-vector-machines-2gdxasg9zv.pdf

LIBSVM: A library for support vector machines

SUMMARY We propose a new method for estimation in linear models. The 'lasso' minimizes the residual sum of squares subject to the sum of the absolute value of the coefficients being less than a constant. Because of the nature of this constraint it tends to produce some coefficients that are exactly 0 and hence gives interpretable models. Our simulation studies suggest that the lasso enjoys some of the favourable properties of both subset selection and ridge regression. It produces interpretable models like subset selection and exhibits the stability of ridge regression. There is also an interesting relationship with recent work in adaptive function estimation by Donoho and Johnstone. The lasso idea is quite general and can be applied in a variety of statistical models: extensions to generalized regression models and tree-based models are briefly described.

/pdf/regression-shrinkage-and-selection-via-the-lasso-6mx35txfr5.pdf

Regression Shrinkage and Selection via the Lasso

Introduction To Tree Classification

https://www.slac.stanford.edu/pubs/slacpubs/4250/slac-pub-4254.pdf

An introduction to real-time graphical techniques for analyzing multivariate data

Lasso and Elastic-Net Regularized Generalized Linear Models [R package glmnet version 4.0-2]

A new data science tool named wavelet-based gradient boosting is proposed and tested. The approach is special case of componentwise linear least squares gradient boosting, and involves wavelet functions of the original predictors. Wavelet-based gradient boosting takes advantages of the approximate $$\ell _1$$l1 penalization induced by gradient boosting to give appropriate penalized additive fits. The method is readily implemented in R and produces parsimonious and interpretable regression fits and classifiers.

http://matt-wand.utsacademics.info/publicns/Dubossarsky16.pdf

Wavelet-based gradient boosting

The goal of this dissertation is to study and develop automatic prediction technology that is accurate, fast and interpretable. The focus here is on decision tree ensembles methods and extensions. Characteristics of popular ensemble methods such as bagging, random forests and boosting are examined and leveraged to create new predictive methodology. The classic ensembles are integrated in an unifying paradigm, the Important Sampled Learning Ensembles. This framework explains some of the properties of these ensembles and suggests modifications that can significantly enhance their accuracy while dramatically improving their computational performance. The ISLES are two-stage algorithms having at the front-end a base learners ensemble generation routine followed by post-processing algorithms that perform a fast gradient directed regularized fit for regression, robust regression and classification. The post-processing algorithms developed here can also serve as a stand-alone toolkit for fitting large linear systems. Decision tree ensembles can generate rules that are fit together with the gradient directed regularized linear algorithms, leading to accurate and interpretable RuleFit models. ISLE and RuleFit are flexible methodologies, able to automatically handle non-linearities and interactions, mixtures of categorical and continuous variables with missing data, as well as feature selection.

Jerome H. Friedman

Papers

Introduction To Tree Classification

An introduction to real-time graphical techniques for analyzing multivariate data

Lasso and Elastic-Net Regularized Generalized Linear Models [R package glmnet version 4.0-2]

Wavelet-based gradient boosting

Ensemble learning for prediction