Scikit-learn is a Python module integrating a wide range of state-of-the-art machine learning algorithms for medium-scale supervised and unsupervised problems. This package focuses on bringing machine learning to non-specialists using a general-purpose high-level language. Emphasis is put on ease of use, performance, documentation, and API consistency. It has minimal dependencies and is distributed under the simplified BSD license, encouraging its use in both academic and commercial settings. Source code, binaries, and documentation can be downloaded from http://scikit-learn.sourceforge.net.

/pdf/scikit-learn-machine-learning-in-python-23clfh9q83.pdf

Scikit-learn: Machine Learning in Python

In comparative high-throughput sequencing assays, a fundamental task is the analysis of count data, such as read counts per gene in RNA-seq, for evidence of systematic changes across experimental conditions. Small replicate numbers, discreteness, large dynamic range and the presence of outliers require a suitable statistical approach. We present DESeq2, a method for differential analysis of count data, using shrinkage estimation for dispersions and fold changes to improve stability and interpretability of estimates. This enables a more quantitative analysis focused on the strength rather than the mere presence of differential expression. The DESeq2 package is available at http://www.bioconductor.org/packages/release/bioc/html/DESeq2.html
 .

/pdf/moderated-estimation-of-fold-change-and-dispersion-for-rna-3ywnbkcan0.pdf

Moderated estimation of fold change and dispersion for RNA-seq data with DESeq2

SUMMARY We propose a new method for estimation in linear models. The 'lasso' minimizes the residual sum of squares subject to the sum of the absolute value of the coefficients being less than a constant. Because of the nature of this constraint it tends to produce some coefficients that are exactly 0 and hence gives interpretable models. Our simulation studies suggest that the lasso enjoys some of the favourable properties of both subset selection and ridge regression. It produces interpretable models like subset selection and exhibits the stability of ridge regression. There is also an interesting relationship with recent work in adaptive function estimation by Donoho and Johnstone. The lasso idea is quite general and can be applied in a variety of statistical models: extensions to generalized regression models and tree-based models are briefly described.

/pdf/regression-shrinkage-and-selection-via-the-lasso-6mx35txfr5.pdf

Regression Shrinkage and Selection via the Lasso

We propose a deep convolutional neural network architecture codenamed Inception that achieves the new state of the art for classification and detection in the ImageNet Large-Scale Visual Recognition Challenge 2014 (ILSVRC14). The main hallmark of this architecture is the improved utilization of the computing resources inside the network. By a carefully crafted design, we increased the depth and width of the network while keeping the computational budget constant. To optimize quality, the architectural decisions were based on the Hebbian principle and the intuition of multi-scale processing. One particular incarnation used in our submission for ILSVRC14 is called GoogLeNet, a 22 layers deep network, the quality of which is assessed in the context of classification and detection.

/pdf/going-deeper-with-convolutions-1yobw2o2ds.pdf

Going deeper with convolutions

Deep learning is a form of machine learning that enables computers to learn from experience and understand the world in terms of a hierarchy of concepts. Because the computer gathers knowledge from experience, there is no need for a human computer operator to formally specify all the knowledge that the computer needs. The hierarchy of concepts allows the computer to learn complicated concepts by building them out of simpler ones; a graph of these hierarchies would be many layers deep. This book introduces a broad range of topics in deep learning. The text offers mathematical and conceptual background, covering relevant concepts in linear algebra, probability theory and information theory, numerical computation, and machine learning. It describes deep learning techniques used by practitioners in industry, including deep feedforward networks, regularization, optimization algorithms, convolutional networks, sequence modeling, and practical methodology; and it surveys such applications as natural language processing, speech recognition, computer vision, online recommendation systems, bioinformatics, and videogames. Finally, the book offers research perspectives, covering such theoretical topics as linear factor models, autoencoders, representation learning, structured probabilistic models, Monte Carlo methods, the partition function, approximate inference, and deep generative models. Deep Learning can be used by undergraduate or graduate students planning careers in either industry or research, and by software engineers who want to begin using deep learning in their products or platforms. A website offers supplementary material for both readers and instructors.

Deep Learning

We tackle the problem of the estimation of a vector of means from a single vector-valued observation $y$. Whereas previous work reduces the size of the estimates for the largest (absolute) sample elements via shrinkage (like James-Stein) or biases estimated via empirical Bayes methodology, we take a novel approach. We adapt recent developments by Lee et al (2013) in post selection inference for the Lasso to the orthogonal setting, where sample elements have different underlying signal sizes. This is exactly the setup encountered when estimating many means. It is shown that other selection procedures, like selecting the $K$ largest (absolute) sample elements and the Benjamini-Hochberg procedure, can be cast into their framework, allowing us to leverage their results. Point and interval estimates for signal sizes are proposed. These seem to perform quite well against competitors, both recent and more tenured. 
Furthermore, we prove an upper bound to the worst case risk of our estimator, when combined with the Benjamini-Hochberg procedure, and show that it is within a constant multiple of the minimax risk over a rich set of parameter spaces meant to evoke sparsity.

Post-selection point and interval estimation of signal sizes in Gaussian samples

We discuss a method called “cluster scoring” for supervised learningfrom a set of gene expression experiments. Cluster scoring generalizes methodsthat rank individual genes based on their correlation with an outcome measure.It begins with a clustering of the genes, for example from hierarchical clustering,and then computes outcome scores both for individual genes and the average geneexpression for each of the clusters. A permutation method is used to identify thesigniﬁcant subset of these scores. We illustrate the method on both simulated data,and data from a study of lymphoma. Key words and phrases: Clustering, microarrays, supervised learning. 1. BackgroundIn this paper we present a method for “supervised learning” from gene ex-pression data. As our starting point, we have gene expression data from a collec-tion of experimental samples, each hybridized to a micro-array— either cDNAor oligo arrays. Brown and Botstein (1999) and Dudoit, Yang, Callow and Speed(2000) contain useful background information on microarrays.We denote the expression values by

/pdf/exploratory-screening-of-genes-and-clusters-from-microarray-1hna81d9ea.pdf

Exploratory screening of genes and clusters from microarray experiments

Nearest neighbor classification expects the class conditional probabilities to be locally constant, and suffers from bias in high dimensions We propose a locally adaptive form of nearest neighbor classification to try to finesse this curse of dimensionality. We use a local linear discriminant analysis to estimate an effective metric for computing neighborhoods. We determine the local decision boundaries from centroid information, and then shrink neighborhoods in directions orthogonal to these local decision boundaries, and elongate them parallel to the boundaries. Thereafter, any neighborhood-based classifier can be employed, using the modified neighborhoods. The posterior probabilities tend to be more homogeneous in the modified neighborhoods. We also propose a method for global dimension reduction, that combines local dimension information. In a number of examples, the methods demonstrate the potential for substantial improvements over nearest neighbour classification.

Discriminant adaptive nearest neighbor classification

We describe the analysis of some matched-pair binary data arising from a study designed to investigate whether cellular-telephone use is associated with motor-vehicle collisions. Conditional and random effects approaches to the problem are derived and compared. Driving intermittency is a potential confounder whose effect is assessed by strategic choices of the control period and by application of the bootstrap. The marked discrepancy between the conditional and random approaches merits further study.



Cet article decrit l'analyse de quelques donnees binaires de paires assorties survenant lors d'une etude visant a determiner si l'usage du telephone cellulaire est associe aux collisions de vehicules motorises. Des approaches d'effects conditionnes et aletoires au probleme sont derivees et comparees. L'intermittence de la conduite est un element potentiellement confondant, dont l'effect est evalue par des choix strategiques de la periode de contrǒle et par l'application du bootstrap. La difference marquee entre l'approche conditionnelle et l'approche aleatoire merite d'ětre etudiee plus profondement.

Cellular telephones and motor‐vehicle collisions: Some variations on matched‐pairs analysis

In a regression or classification setting where we wish to predict Y from x1,x2,…, xp, we suppose that an additional set of ’coaching‘ variables z1,z2,…, zm are available in our training sample. These might be variables that are difficult to measure, and they will not be available when we predict Y from x1,x2,…, xp in the future. We consider two methods of making use of the coaching variables in order to improve the prediction of Y from x1,x2,…, xp. The relative merits of these approaches are discussed and compared in a number of examples.

Robert Tibshirani

Papers

Post-selection point and interval estimation of signal sizes in Gaussian samples

Exploratory screening of genes and clusters from microarray experiments

Discriminant adaptive nearest neighbor classification

Cellular telephones and motor‐vehicle collisions: Some variations on matched‐pairs analysis

Coaching variables for regression and classification