Scikit-learn is a Python module integrating a wide range of state-of-the-art machine learning algorithms for medium-scale supervised and unsupervised problems. This package focuses on bringing machine learning to non-specialists using a general-purpose high-level language. Emphasis is put on ease of use, performance, documentation, and API consistency. It has minimal dependencies and is distributed under the simplified BSD license, encouraging its use in both academic and commercial settings. Source code, binaries, and documentation can be downloaded from http://scikit-learn.sourceforge.net.

/pdf/scikit-learn-machine-learning-in-python-23clfh9q83.pdf

Scikit-learn: Machine Learning in Python

In comparative high-throughput sequencing assays, a fundamental task is the analysis of count data, such as read counts per gene in RNA-seq, for evidence of systematic changes across experimental conditions. Small replicate numbers, discreteness, large dynamic range and the presence of outliers require a suitable statistical approach. We present DESeq2, a method for differential analysis of count data, using shrinkage estimation for dispersions and fold changes to improve stability and interpretability of estimates. This enables a more quantitative analysis focused on the strength rather than the mere presence of differential expression. The DESeq2 package is available at http://www.bioconductor.org/packages/release/bioc/html/DESeq2.html
 .

/pdf/moderated-estimation-of-fold-change-and-dispersion-for-rna-3ywnbkcan0.pdf

Moderated estimation of fold change and dispersion for RNA-seq data with DESeq2

SUMMARY We propose a new method for estimation in linear models. The 'lasso' minimizes the residual sum of squares subject to the sum of the absolute value of the coefficients being less than a constant. Because of the nature of this constraint it tends to produce some coefficients that are exactly 0 and hence gives interpretable models. Our simulation studies suggest that the lasso enjoys some of the favourable properties of both subset selection and ridge regression. It produces interpretable models like subset selection and exhibits the stability of ridge regression. There is also an interesting relationship with recent work in adaptive function estimation by Donoho and Johnstone. The lasso idea is quite general and can be applied in a variety of statistical models: extensions to generalized regression models and tree-based models are briefly described.

/pdf/regression-shrinkage-and-selection-via-the-lasso-6mx35txfr5.pdf

Regression Shrinkage and Selection via the Lasso

We propose a deep convolutional neural network architecture codenamed Inception that achieves the new state of the art for classification and detection in the ImageNet Large-Scale Visual Recognition Challenge 2014 (ILSVRC14). The main hallmark of this architecture is the improved utilization of the computing resources inside the network. By a carefully crafted design, we increased the depth and width of the network while keeping the computational budget constant. To optimize quality, the architectural decisions were based on the Hebbian principle and the intuition of multi-scale processing. One particular incarnation used in our submission for ILSVRC14 is called GoogLeNet, a 22 layers deep network, the quality of which is assessed in the context of classification and detection.

/pdf/going-deeper-with-convolutions-1yobw2o2ds.pdf

Going deeper with convolutions

Deep learning is a form of machine learning that enables computers to learn from experience and understand the world in terms of a hierarchy of concepts. Because the computer gathers knowledge from experience, there is no need for a human computer operator to formally specify all the knowledge that the computer needs. The hierarchy of concepts allows the computer to learn complicated concepts by building them out of simpler ones; a graph of these hierarchies would be many layers deep. This book introduces a broad range of topics in deep learning. The text offers mathematical and conceptual background, covering relevant concepts in linear algebra, probability theory and information theory, numerical computation, and machine learning. It describes deep learning techniques used by practitioners in industry, including deep feedforward networks, regularization, optimization algorithms, convolutional networks, sequence modeling, and practical methodology; and it surveys such applications as natural language processing, speech recognition, computer vision, online recommendation systems, bioinformatics, and videogames. Finally, the book offers research perspectives, covering such theoretical topics as linear factor models, autoencoders, representation learning, structured probabilistic models, Monte Carlo methods, the partition function, approximate inference, and deep generative models. Deep Learning can be used by undergraduate or graduate students planning careers in either industry or research, and by software engineers who want to begin using deep learning in their products or platforms. A website offers supplementary material for both readers and instructors.

Deep Learning

Discussion and further topics

We propose a new sparse regression method called the component lasso, based on a simple idea. The method uses the connected-components structure of the sample covariance matrix to split the problem into smaller ones. It then applies the lasso to each subproblem separately, obtaining a coefficient vector for each one. Finally, it uses non-negative least squares to recombine the different vectors into a single solution. This step is useful in selecting and reweighting components that are correlated with the response. We prove that the component lasso is strongly sign consistent in a block-diagonal setting. Simulated and real data examples show that the component lasso can outperform standard regression methods such as the lasso and elastic net, achieving a lower mean squared error as well as better support recovery. The modular structure of the algorithm also lends itself naturally to parallel computation. The Canadian Journal of Statistics 43: 624–646; 2015 © 2015 Statistical Society of Canada


Resume


Les auteurs proposent une nouvelle methode de regression eparse intitulee le lasso par composante qui se base sur une idee toute simple. Ils utilisent la structure en composantes connectees de la matrice de covariance empirique afin de fragmenter le probleme. Ils appliquent alors le lasso separement a chaque sous-probleme et obtiennent un vecteur de coefficients pour chacun. Ils utilisent finalement les moindres carres non negatifs afin de recombiner les differents vecteurs en une solution unique. Cette derniere etape s'avere utile dans la selection et la reponderation des composantes qui sont correlees avec la reponse. Les auteurs demontrent la forte convergence en signe du lasso par composante dans le contexte d'une matrice diagonale par blocs. Ils presentent des exemples avec des donnees reelles et simulees qui montrent que le lasso par composante peut surclasser les methodes de regression habituelles telles que le lasso et le filet elastique, tant au niveau de l'erreur quadratique moyenne que de la reconstitution du support. La structure modulaire de l'algorithme s'adapte par ailleurs naturellement au calcul parallele. La revue canadienne de statistique 43: 624–646; 2015 © 2015 Societe statistique du Canada

A component lasso

In some multivariate problems with missing data, pairs of variables exist that are never observed together. For example, some modern biological tools can produce data of this form. As a result of this structure, the covariance matrix is only partially identifiable, and point estimation requires that identifying assumptions be made. These assumptions can introduce an unknown and potentially large bias into the inference. This paper presents a method based on semidefinite programming for automatically quantifying this potential bias by computing the range of possible equal-likelihood inferred values for convex functions of the covariance matrix. We focus on the bias of missing value imputation via conditional expectation and show that our method can give an accurate assessment of the true error in cases where estimates based on sampling uncertainty alone are overly optimistic.

Sensitivity Analysis for Inference with Partially Identifiable Covariance Matrices

FLT3 Mutations Determine the Clinical Outcome in Children with De Novo Acute Myelogenous Leukemia (AML) and Normal Karyotype: Pediatric Oncology Group (POG) Study # 9421.

We substantially agree with the comments of Catchpoole et al in response to our editorial. The idea that analyses of multidimensional, highly complex datasets of cancer and host factors will lead to more precise and successful individualized therapies is immensely appealing. We agree that building the algorithms for truly personalized medicine indeed will require paradigm shifts in clinical and translational research. The path to tailored therapies in individuals will be paved in large part by a deeper understanding of the complexity and heterogeneity of cancers. Genomics has contributed much to this understanding, as exemplified by the reclassification of breast cancers into many distinct subtypes based on gene expression profiles. Our endorsement of the so-called virtue of complexity is an appeal to strengthen the scientific rigor of genomic studies in cancer. Increasing the number of patients and samples is necessary but far from sufficient. Other important factors in conducting and reporting such studies include patient stratification, integration of various highthroughput technologies, novel trial designs and bioinformatics approaches, integration of emerging concepts in cancer biology, and transparent and complete presentation of statistical analyses. Despite the limitations of reductionism, genomic signatures derived from such approaches have proved useful in defining risks for relapse and appropriate patients for adjuvant therapies in early-stage breast cancers. Moreover, the development of predictive therapeutic biomarkers based on the understanding of molecular pathways, networks, and drug mechanisms has much to contribute to the personalization of cancer therapies, as illustrated by the importance of RAS mutation status in colorectal cancers treated with epidermal growth factor receptor–targeted antibodies. Ultimately, however, we agree that the development of truly personalized therapies will require ascertaining the key differences among individuals as well as similarities between cohorts within a disease type. Proof that tailoring makes a difference, particularly in a highly curable disease like pediatric acute lymphoblastic leukemia, is itself a major challenge in clinical trials designs.

Robert Tibshirani

Papers

Discussion and further topics

A component lasso

Sensitivity Analysis for Inference with Partially Identifiable Covariance Matrices

FLT3 Mutations Determine the Clinical Outcome in Children with De Novo Acute Myelogenous Leukemia (AML) and Normal Karyotype: Pediatric Oncology Group (POG) Study # 9421.

Reply to D.R. Catchpoole et al