Scikit-learn is a Python module integrating a wide range of state-of-the-art machine learning algorithms for medium-scale supervised and unsupervised problems. This package focuses on bringing machine learning to non-specialists using a general-purpose high-level language. Emphasis is put on ease of use, performance, documentation, and API consistency. It has minimal dependencies and is distributed under the simplified BSD license, encouraging its use in both academic and commercial settings. Source code, binaries, and documentation can be downloaded from http://scikit-learn.sourceforge.net.

/pdf/scikit-learn-machine-learning-in-python-23clfh9q83.pdf

Scikit-learn: Machine Learning in Python

SUMMARY We propose a new method for estimation in linear models. The 'lasso' minimizes the residual sum of squares subject to the sum of the absolute value of the coefficients being less than a constant. Because of the nature of this constraint it tends to produce some coefficients that are exactly 0 and hence gives interpretable models. Our simulation studies suggest that the lasso enjoys some of the favourable properties of both subset selection and ridge regression. It produces interpretable models like subset selection and exhibits the stability of ridge regression. There is also an interesting relationship with recent work in adaptive function estimation by Donoho and Johnstone. The lasso idea is quite general and can be applied in a variety of statistical models: extensions to generalized regression models and tree-based models are briefly described.

/pdf/regression-shrinkage-and-selection-via-the-lasso-6mx35txfr5.pdf

Regression Shrinkage and Selection via the Lasso

Introduction to a Transient World. Fourier Kingdom. Discrete Revolution. Time Meets Frequency. Frames. Wavelet Zoom. Wavelet Bases. Wavelet Packet and Local Cosine Bases. An Approximation Tour. Estimations are Approximations. Transform Coding. Appendix A: Mathematical Complements. Appendix B: Software Toolboxes.

http://ahvazdownload.persiangig.com/document/article/39.pdf

A wavelet tour of signal processing

Summary. We propose the elastic net, a new regularization and variable selection method. Real world data and a simulation study show that the elastic net often outperforms the lasso, while enjoying a similar sparsity of representation. In addition, the elastic net encourages a grouping effect, where strongly correlated predictors tend to be in or out of the model together.The elastic net is particularly useful when the number of predictors (p) is much bigger than the number of observations (n). By contrast, the lasso is not a very satisfactory variable selection method in the

/pdf/regularization-and-variable-selection-via-the-elastic-net-4z8q6j5o0i.pdf

Regularization and variable selection via the elastic net

MOTIVATION: Next-generation sequencing technologies generate millions of short sequence reads, which are usually aligned to a reference genome. In many applications, the key information required for downstream analysis is the number of reads mapping to each genomic feature, for example to each exon or each gene. The process of counting reads is called read summarization. Read summarization is required for a great variety of genomic analyses but has so far received relatively little attention in the literature. RESULTS: We present featureCounts, a read summarization program suitable for counting reads generated from either RNA or genomic DNA sequencing experiments. featureCounts implements highly efficient chromosome hashing and feature blocking techniques. It is considerably faster than existing methods (by an order of magnitude for gene-level summarization) and requires far less computer memory. It works with either single or paired-end reads and provides a wide range of options appropriate for different sequencing applications. AVAILABILITY AND IMPLEMENTATION: featureCounts is available under GNU General Public License as part of the Subread (http://subread.sourceforge.net) or Rsubread (http://www.bioconductor.org) software packages.

featureCounts: an efficient general-purpose program for assigning sequence reads to genomic features

Compressed sensing posits that, within limits, one can undersample a sparse signal and yet reconstruct it accurately. Knowing the precise limits to such undersampling is important both for theory and practice. We present a formula that characterizes the allowed undersampling of generalized sparse objects. The formula applies to Approximate Message Passing (AMP) algorithms for compressed sensing, which are here generalized to employ denoising operators besides the traditional scalar soft thresholding denoiser. This paper gives several examples including scalar denoisers not derived from convex penalization -- the firm shrinkage nonlinearity and the minimax nonlinearity -- and also nonscalar denoisers -- block thresholding, monotone regression, and total variation minimization. 
Let the variables eps = k/N and delta = n/N denote the generalized sparsity and undersampling fractions for sampling the k-generalized-sparse N-vector x_0 according to y=Ax_0. Here A is an n\times N measurement matrix whose entries are iid standard Gaussian. The formula states that the phase transition curve delta = delta(eps) separating successful from unsuccessful reconstruction of x_0 by AMP is given by: delta = M(eps| Denoiser), where M(eps| Denoiser) denotes the per-coordinate minimax mean squared error (MSE) of the specified, optimally-tuned denoiser in the directly observed problem y = x + z. In short, the phase transition of a noiseless undersampling problem is identical to the minimax MSE in a denoising problem.

Accurate Prediction of Phase Transitions in Compressed Sensing via a Connection to Minimax Denoising

Detecting the presence of a signal embedded in noise from a multi-sensor system is a fundamental problem in signal and array processing. In this paper we consider the case where the noise covariance matrix is arbitrary and unknown but we are given both signal bearing and noise-only samples. Using a matrix perturbation approach, combined with known results on the eigenvalues of inverse Wishart matrices, we study the behavior of the largest eigenvalue of the relevant covariance matrix, and derive an approximate expression for the detection probability of Roy's largest root test. The accuracy of our expressions is confirmed by simulations.

/pdf/detection-performance-of-roy-s-largest-root-test-when-the-1i2eht4dv8.pdf

Detection performance of Roy's largest root test when the noise covariance matrix is arbitrary

We establish a connection between admissible simultaneous estimation and recurrence of reversible Markov chains on ℤ
 +
  Specifically, to each generalized Bayes estimator of the mean of a vector of p independent Poisson variables for a weighted quadratic loss, we associate a variational problem and a reversible birth and death chain on ℤ
 +
  The variational problem is closely related to the Dirichlet principle for reversible chains studied recently by Griffeath, Liggett and Lyons Under side conditions, admissibility of the estimator is equivalent to zero infimal energy in the variational problem and to recurrence of the Markov chain This yields analytic and probabilistic criteria for inadmissibility which are applied to establish a broad class of results and previous conjectures

Admissible estimation, Dirichlet principles and recurrence of birth-death chains on ℤ + p

Monte Carlo “swindles” or variance reduction techniques exploit the experimenter's knowledge of the stochastic structure governing the simulated data to construct more precise estimates of unknown parameters. Alternatively, one can reduce the number of replications (and thus the cost) needed to gain a desired level of precision. This article reviews the common case of swindles based on variance decompositions for estimating efficiencies and variances of location and regression estimators. We then propose a new swindle based on Fisher's efficient score function that can be applied profitably to a much wider range of situations than can the Gaussian-over-independent swindles used in many studies of robust estimators. We compare these methods by performing simulations for the efficiencies of location estimates and by placing them in a simple geometric framework. As an illustration, the score function swindle is used to estimate the variances of Pitman estimates of location for samples from selected ...

Efficient Scores, Variance Decompositions, and Monte Carlo Swindles

There are standard modifications of certain compactly supported wavelets that yield orthonormal bases on a bounded interval. We extend one such construction to those wavelets, such as ‘coiflets', that may have fewer vanishing moments than had to be assumed previously. Our motivation lies in function estimation in statistics. We use these boundary-modified coiflets to show that the discrete wavelet transform of finite data from sampled regression models asymptotically provides a close approximation to the wavelet transform of the continuous Gaussian white noise model. In particular, estimation errors in the discrete setting of computational practice need not be essentially larger than those expected in the continuous setting of statistical theory.

/pdf/boundary-coiflets-for-wavelet-shrinkage-in-function-50eo13mjb0.pdf

Iain M. Johnstone

Papers

Accurate Prediction of Phase Transitions in Compressed Sensing via a Connection to Minimax Denoising

Detection performance of Roy's largest root test when the noise covariance matrix is arbitrary

Admissible estimation, Dirichlet principles and recurrence of birth-death chains on ℤ + p

Efficient Scores, Variance Decompositions, and Monte Carlo Swindles

Boundary coiflets for wavelet shrinkage in function estimation