Scikit-learn is a Python module integrating a wide range of state-of-the-art machine learning algorithms for medium-scale supervised and unsupervised problems. This package focuses on bringing machine learning to non-specialists using a general-purpose high-level language. Emphasis is put on ease of use, performance, documentation, and API consistency. It has minimal dependencies and is distributed under the simplified BSD license, encouraging its use in both academic and commercial settings. Source code, binaries, and documentation can be downloaded from http://scikit-learn.sourceforge.net.

/pdf/scikit-learn-machine-learning-in-python-23clfh9q83.pdf

Scikit-learn: Machine Learning in Python

SUMMARY We propose a new method for estimation in linear models. The 'lasso' minimizes the residual sum of squares subject to the sum of the absolute value of the coefficients being less than a constant. Because of the nature of this constraint it tends to produce some coefficients that are exactly 0 and hence gives interpretable models. Our simulation studies suggest that the lasso enjoys some of the favourable properties of both subset selection and ridge regression. It produces interpretable models like subset selection and exhibits the stability of ridge regression. There is also an interesting relationship with recent work in adaptive function estimation by Donoho and Johnstone. The lasso idea is quite general and can be applied in a variety of statistical models: extensions to generalized regression models and tree-based models are briefly described.

/pdf/regression-shrinkage-and-selection-via-the-lasso-6mx35txfr5.pdf

Regression Shrinkage and Selection via the Lasso

Introduction to a Transient World. Fourier Kingdom. Discrete Revolution. Time Meets Frequency. Frames. Wavelet Zoom. Wavelet Bases. Wavelet Packet and Local Cosine Bases. An Approximation Tour. Estimations are Approximations. Transform Coding. Appendix A: Mathematical Complements. Appendix B: Software Toolboxes.

http://ahvazdownload.persiangig.com/document/article/39.pdf

A wavelet tour of signal processing

Summary. We propose the elastic net, a new regularization and variable selection method. Real world data and a simulation study show that the elastic net often outperforms the lasso, while enjoying a similar sparsity of representation. In addition, the elastic net encourages a grouping effect, where strongly correlated predictors tend to be in or out of the model together.The elastic net is particularly useful when the number of predictors (p) is much bigger than the number of observations (n). By contrast, the lasso is not a very satisfactory variable selection method in the

/pdf/regularization-and-variable-selection-via-the-elastic-net-4z8q6j5o0i.pdf

Regularization and variable selection via the elastic net

MOTIVATION: Next-generation sequencing technologies generate millions of short sequence reads, which are usually aligned to a reference genome. In many applications, the key information required for downstream analysis is the number of reads mapping to each genomic feature, for example to each exon or each gene. The process of counting reads is called read summarization. Read summarization is required for a great variety of genomic analyses but has so far received relatively little attention in the literature. RESULTS: We present featureCounts, a read summarization program suitable for counting reads generated from either RNA or genomic DNA sequencing experiments. featureCounts implements highly efficient chromosome hashing and feature blocking techniques. It is considerably faster than existing methods (by an order of magnitude for gene-level summarization) and requires far less computer memory. It works with either single or paired-end reads and provides a wide range of options appropriate for different sequencing applications. AVAILABILITY AND IMPLEMENTATION: featureCounts is available under GNU General Public License as part of the Subread (http://subread.sourceforge.net) or Rsubread (http://www.bioconductor.org) software packages.

featureCounts: an efficient general-purpose program for assigning sequence reads to genomic features

ContextThe recent increase in ability to diagnose prostatic
adenocarcinoma has created a dilemma for treatment decisions.ObjectiveTo determine whether prostate cancer progression is
associated with a modified version of the Gleason grading system
together with selected morphologic and clinical variables.DesignRetrospective analysis of a cohort of patients with
peripheral zone prostate cancers who underwent surgery between August
1983 and July 1992.SettingUniversity hospital.PatientsRadical prostatectomy specimens from 379 men treated only
by surgical excision were prospectively studied for 8 morphologic
variables using previously standardized techniques. Variables were
percentage of each cancer occupied by Gleason grade 4/5 (% Gleason
grade 4/5, the Stanford modified Gleason scale), cancer volume,
vascular invasion, lymph node involvement, seminal vesicle invasion,
capsular penetration, positive surgical margin, prostate weight, and
preoperative prostate-specific antigen (PSA) level.Main Outcome MeasureBiochemical progression of prostate cancer as
indicated by serum PSA level of 0.07 ng/mL and increasing.ResultsCancer grade expressed as % Gleason grade 4/5 and cancer
volume were highly predictive of disease progression. In a Cox
proportional hazards model that included % Gleason grade 4/5, the
traditional Gleason score was not an independent predictor of treatment
failure. Positive lymph node findings and intraprostatic vascular
invasion were the only other variables that remained significant at the
.01 level.ConclusionThe % Gleason grade 4/5, cancer volume, positive lymph
node findings, and intraprostatic vascular invasion were independently
associated with prostate cancer progression, defined by an increasing
PSA level. Techniques to accurately measure cancer volume and %
Gleason grade 4/5 are needed to better predict which patient will
experience cancer progression. The commonly accepted predictors of
progression—capsular penetration and positive surgical margins—were
not independently predictive of failure after radical
prostatectomy.

/pdf/biological-determinants-of-cancer-progression-in-men-with-16wfph4k0t.pdf

Biological determinants of cancer progression in men with prostate cancer.

The purpose of model selection algorithms such as All Subsets, Forward 
Selection and Backward Elimination is to choose a linear model on the basis of the same set of data to which the model will be applied. Typically we have available a large collection of possible covariates from which we hope to select a parsimonious set for the efficient prediction of a response variable. Least Angle Regression (LARS), a new model selection algorithm, is a useful and less greedy version of traditional forward selection methods. 
Three main properties are derived: (1) A simple modification of the LARS algorithm implements the Lasso, an attractive version of ordinary least squares that constrains the sum of the absolute regression coefficients; the LARS modification calculates all possible Lasso estimates for a given problem, using an order of magnitude less computer time than previous methods. 
(2) A different LARS modification efficiently implements Forward Stagewise linear regression, another promising new model selection method;

Least Angle Regression

An empirical Bayes approach to the estimation of possibly sparse sequences observed in Gaussian white noise is set out and investigated. The prior considered is a mixture of an atom of probability at zero and a heavy-tailed density y, with the mixing weight chosen by marginal maximum likelihood, in the hope of adapting between sparse and dense sequences. If estimation is then carried out using the posterior median, this is a random thresholding procedure. Other thresholding rules employing the same threshold can also be used. Probability bounds on the threshold chosen by the marginal maximum likelihood approach lead to overall risk bounds over classes of signal sequences of length n, allowing for sparsity of various kinds and degrees. The signal classes considered are nearly black sequences where only a proportion η is allowed to be nonzero, and sequences with normalized p norm bounded by η, for η > 0 and 0 1. Simulations show excellent performance. For appropriately chosen functions y, the method is computationally tractable and software is available. The extension to a modified thresholding method relevant to the estimation of very sparse sequences is also considered.

/pdf/needles-and-straw-in-haystacks-empirical-bayes-estimates-of-55czd6yf5g.pdf

Needles and straw in haystacks: Empirical Bayes estimates of possibly sparse sequences

We attempt to recover an n-dimensional vector observed in white noise, where n is large and the vector is known to be sparse, but the degree of sparsity is unknown. We consider three different ways of defining sparsity of a vector: using the fraction of nonzero terms; imposing power-law decay bounds on the ordered entries; and controlling the lp norm for p small. We obtain a procedure which is asymptotically minimax for l r loss, simultaneously throughout a range of such sparsity classes. The optimal procedure is a data-adaptive thresholding scheme, driven by control of the False Discovery Rate (FDR). FDR control is a relatively recent innovation in simultaneous testing, ensuring that at most a certain fraction of the rejected null hypotheses will correspond to false rejections. In our treatment, the FDR control parameter qn also plays a determining role in asymptotic minimaxity. If q = lim qn ∈ [0,1/2] and also qn > γ/log(n) we get sharp asymptotic minimaxity, simultaneously, over a wide range of sparse parameter spaces and loss functions. On the other hand, q = lim qn ∈ (1/2,1], forces the risk to exceed the minimax risk by a factor growing with q. To our knowledge, this relation between ideas in simultaneous inference and asymptotic decision theory is new. Our work provides a new perspective on a class of model selection rules which has been introduced recently by several authors. These new rules impose complexity penalization of the form 2 � log( potential model size / actual model size ). We exhibit a close connection with FDR-controlling procedures under stringent control of the false discovery rate.

/pdf/adapting-to-unknown-sparsity-by-controlling-the-false-3924cjsinb.pdf

Adapting to unknown sparsity by controlling the false discovery rate

SUMMARY Maximum entropy (ME) inversion is a non-linear inversion technique for inverse problems where the object to be recovered is known to be positive. It has been applied in areas ranging from radio astronomy to various forms of spectroscopy, sometimes with dramatic success. In some cases, ME has attained an order of magnitude finer resolution and/or an order of magnitude smaller noise level than that obtainable by standard linear methods. The dramatic successes all seem to occur in cases where the object to be recovered is 'nearly black': essentially zero in the vast majority of samples. We show that near-blackness is required, both for signal-to-noise enhancements and for superresolution. However, other methods-in particular, minimum /1-norm reconstruction-may exploit near-blackness to an even greater extent.

https://rss.onlinelibrary.wiley.com/doi/pdf/10.1111/j.2517-6161.1992.tb01864.x

Iain M. Johnstone

Papers

Biological determinants of cancer progression in men with prostate cancer.

Least Angle Regression

Needles and straw in haystacks: Empirical Bayes estimates of possibly sparse sequences

Adapting to unknown sparsity by controlling the false discovery rate

Maximum entropy and the nearly black object