Scikit-learn is a Python module integrating a wide range of state-of-the-art machine learning algorithms for medium-scale supervised and unsupervised problems. This package focuses on bringing machine learning to non-specialists using a general-purpose high-level language. Emphasis is put on ease of use, performance, documentation, and API consistency. It has minimal dependencies and is distributed under the simplified BSD license, encouraging its use in both academic and commercial settings. Source code, binaries, and documentation can be downloaded from http://scikit-learn.sourceforge.net.

/pdf/scikit-learn-machine-learning-in-python-23clfh9q83.pdf

Scikit-learn: Machine Learning in Python

SUMMARY We propose a new method for estimation in linear models. The 'lasso' minimizes the residual sum of squares subject to the sum of the absolute value of the coefficients being less than a constant. Because of the nature of this constraint it tends to produce some coefficients that are exactly 0 and hence gives interpretable models. Our simulation studies suggest that the lasso enjoys some of the favourable properties of both subset selection and ridge regression. It produces interpretable models like subset selection and exhibits the stability of ridge regression. There is also an interesting relationship with recent work in adaptive function estimation by Donoho and Johnstone. The lasso idea is quite general and can be applied in a variety of statistical models: extensions to generalized regression models and tree-based models are briefly described.

/pdf/regression-shrinkage-and-selection-via-the-lasso-6mx35txfr5.pdf

Regression Shrinkage and Selection via the Lasso

Introduction to a Transient World. Fourier Kingdom. Discrete Revolution. Time Meets Frequency. Frames. Wavelet Zoom. Wavelet Bases. Wavelet Packet and Local Cosine Bases. An Approximation Tour. Estimations are Approximations. Transform Coding. Appendix A: Mathematical Complements. Appendix B: Software Toolboxes.

http://ahvazdownload.persiangig.com/document/article/39.pdf

A wavelet tour of signal processing

Summary. We propose the elastic net, a new regularization and variable selection method. Real world data and a simulation study show that the elastic net often outperforms the lasso, while enjoying a similar sparsity of representation. In addition, the elastic net encourages a grouping effect, where strongly correlated predictors tend to be in or out of the model together.The elastic net is particularly useful when the number of predictors (p) is much bigger than the number of observations (n). By contrast, the lasso is not a very satisfactory variable selection method in the

/pdf/regularization-and-variable-selection-via-the-elastic-net-4z8q6j5o0i.pdf

Regularization and variable selection via the elastic net

MOTIVATION: Next-generation sequencing technologies generate millions of short sequence reads, which are usually aligned to a reference genome. In many applications, the key information required for downstream analysis is the number of reads mapping to each genomic feature, for example to each exon or each gene. The process of counting reads is called read summarization. Read summarization is required for a great variety of genomic analyses but has so far received relatively little attention in the literature. RESULTS: We present featureCounts, a read summarization program suitable for counting reads generated from either RNA or genomic DNA sequencing experiments. featureCounts implements highly efficient chromosome hashing and feature blocking techniques. It is considerably faster than existing methods (by an order of magnitude for gene-level summarization) and requires far less computer memory. It works with either single or paired-end reads and provides a wide range of options appropriate for different sequencing applications. AVAILABILITY AND IMPLEMENTATION: featureCounts is available under GNU General Public License as part of the Subread (http://subread.sourceforge.net) or Rsubread (http://www.bioconductor.org) software packages.

featureCounts: an efficient general-purpose program for assigning sequence reads to genomic features

If $\{g_\theta(t)\}$ is a regular family of probability densities on the real line, with corresponding hazard rates $\{h_\theta(t)\}$, then the Fisher information for $\theta$ can be expressed in terms of the hazard rate as follows: $\mathscr{I}_\theta \equiv \int \big(\frac{\dot{g}_\theta}{g_\theta}\big)^2 g_\theta = \int \big(\frac{\dot{h}_\theta}{h_\theta}\big)^2 g_\theta, \theta \in \mathbb{R},$ where the dot denotes $\partial/\partial\theta$. This identity shows that the hazard rate transform of a probability density has an unexpected length-preserving property. We explore this property in continuous and discrete settings, some geometric consequences and curvature formulas, its connection with martingale theory and its relation to statistical issues in the theory of life-time distributions and censored data.

/pdf/fisher-s-information-in-terms-of-the-hazard-rate-2dflony73t.pdf

Fisher's information in terms of the hazard rate'

We illustrate with contemporary examples Hotelling's geometric approach to simultaneous probability calculations. Hotelling reduces the evaluation of certain normal theory significance probabilities to finding the volume of a tube about a curve in a hypersphere, and shows that this volume is often exactly given by length times cross-sectional area. We review Hotelling's result together with some recent complements, and then use the approach to set simultaneous prediction regions for some data from gait analysis, to study Andrews' plots in multivariate data analysis, and to construct significance tests for projection pursuit regression. A by-product is a numerical criterion for tube self-overlap, relevant, for example, to uniqueness of certain nonlinear least squares estimates.

/pdf/hotelling-s-theorem-on-the-volume-of-tubes-some-3j02sp0api.pdf

Hotelling's Theorem on the Volume of Tubes: Some Illustrations in Simultaneous Inference and Data Analysis

When the data are high dimensional, widely used multivariate statistical methods such as principal component analysis can behave in unexpected ways. In settings where the dimension of the observations is comparable to the sample size, upward bias in sample eigenvalues and inconsistency of sample eigenvectors are among the most notable phenomena that appear. These phenomena, and the limiting behavior of the rescaled extreme sample eigenvalues, have recently been investigated in detail under the spiked covariance model. The behavior of the bulk of the sample eigenvalues under weak distributional assumptions on the observations has been described. These results have been exploited to develop new estimation and hypothesis testing methods for the population covariance matrix. Furthermore, partly in response to these phenomena, alternative classes of estimation procedures have been developed by exploiting sparsity of the eigenvectors or the covariance matrix. This paper gives an orientation to these areas.

PCA in High Dimensions: An Orientation

We study the problem of estimating the leading eigenvectors of a high-dimensional
 population covariance matrix based on independent Gaussian observations. We establish lower
 bounds on the rates of convergence of the estimators of the leading eigenvectors under
 $l^q$-sparsity constraints when an $l^2$ loss function is used. We also propose an
 estimator of the leading eigenvectors based on a coordinate selection scheme combined with
 PCA and show that the proposed estimator achieves the optimal rate of convergence under a
 sparsity regime. Moreover, we establish that under certain scenarios, the usual PCA
 achieves the minimax convergence rate.

Augmented sparse principal component analysis for high dimensional data

Asymptotic normality of the posterior distribution of a parameter in a stochastic process is shown to hold under conditions which do little more than ensure consistency of a maximum likelihood estimator. Much more stringent conditions are required to ensure asymptotic normality of the MLE. This contrast, which has implications of considerable significance, does not emerge in the classical context of independent and identically distributed observations.

/pdf/on-asymptotic-posterior-normality-for-stochastic-processes-1kovbmzxib.pdf

Iain M. Johnstone

Papers

Fisher's information in terms of the hazard rate'

Hotelling's Theorem on the Volume of Tubes: Some Illustrations in Simultaneous Inference and Data Analysis

PCA in High Dimensions: An Orientation

Augmented sparse principal component analysis for high dimensional data

On Asymptotic Posterior Normality for Stochastic Processes