Scikit-learn is a Python module integrating a wide range of state-of-the-art machine learning algorithms for medium-scale supervised and unsupervised problems. This package focuses on bringing machine learning to non-specialists using a general-purpose high-level language. Emphasis is put on ease of use, performance, documentation, and API consistency. It has minimal dependencies and is distributed under the simplified BSD license, encouraging its use in both academic and commercial settings. Source code, binaries, and documentation can be downloaded from http://scikit-learn.sourceforge.net.

/pdf/scikit-learn-machine-learning-in-python-23clfh9q83.pdf

Scikit-learn: Machine Learning in Python

In comparative high-throughput sequencing assays, a fundamental task is the analysis of count data, such as read counts per gene in RNA-seq, for evidence of systematic changes across experimental conditions. Small replicate numbers, discreteness, large dynamic range and the presence of outliers require a suitable statistical approach. We present DESeq2, a method for differential analysis of count data, using shrinkage estimation for dispersions and fold changes to improve stability and interpretability of estimates. This enables a more quantitative analysis focused on the strength rather than the mere presence of differential expression. The DESeq2 package is available at http://www.bioconductor.org/packages/release/bioc/html/DESeq2.html
 .

/pdf/moderated-estimation-of-fold-change-and-dispersion-for-rna-3ywnbkcan0.pdf

Moderated estimation of fold change and dispersion for RNA-seq data with DESeq2

SUMMARY We propose a new method for estimation in linear models. The 'lasso' minimizes the residual sum of squares subject to the sum of the absolute value of the coefficients being less than a constant. Because of the nature of this constraint it tends to produce some coefficients that are exactly 0 and hence gives interpretable models. Our simulation studies suggest that the lasso enjoys some of the favourable properties of both subset selection and ridge regression. It produces interpretable models like subset selection and exhibits the stability of ridge regression. There is also an interesting relationship with recent work in adaptive function estimation by Donoho and Johnstone. The lasso idea is quite general and can be applied in a variety of statistical models: extensions to generalized regression models and tree-based models are briefly described.

/pdf/regression-shrinkage-and-selection-via-the-lasso-6mx35txfr5.pdf

Regression Shrinkage and Selection via the Lasso

We propose a deep convolutional neural network architecture codenamed Inception that achieves the new state of the art for classification and detection in the ImageNet Large-Scale Visual Recognition Challenge 2014 (ILSVRC14). The main hallmark of this architecture is the improved utilization of the computing resources inside the network. By a carefully crafted design, we increased the depth and width of the network while keeping the computational budget constant. To optimize quality, the architectural decisions were based on the Hebbian principle and the intuition of multi-scale processing. One particular incarnation used in our submission for ILSVRC14 is called GoogLeNet, a 22 layers deep network, the quality of which is assessed in the context of classification and detection.

/pdf/going-deeper-with-convolutions-1yobw2o2ds.pdf

Going deeper with convolutions

Deep learning is a form of machine learning that enables computers to learn from experience and understand the world in terms of a hierarchy of concepts. Because the computer gathers knowledge from experience, there is no need for a human computer operator to formally specify all the knowledge that the computer needs. The hierarchy of concepts allows the computer to learn complicated concepts by building them out of simpler ones; a graph of these hierarchies would be many layers deep. This book introduces a broad range of topics in deep learning. The text offers mathematical and conceptual background, covering relevant concepts in linear algebra, probability theory and information theory, numerical computation, and machine learning. It describes deep learning techniques used by practitioners in industry, including deep feedforward networks, regularization, optimization algorithms, convolutional networks, sequence modeling, and practical methodology; and it surveys such applications as natural language processing, speech recognition, computer vision, online recommendation systems, bioinformatics, and videogames. Finally, the book offers research perspectives, covering such theoretical topics as linear factor models, autoencoders, representation learning, structured probabilistic models, Monte Carlo methods, the partition function, approximate inference, and deep generative models. Deep Learning can be used by undergraduate or graduate students planning careers in either industry or research, and by software engineers who want to begin using deep learning in their products or platforms. A website offers supplementary material for both readers and instructors.

Deep Learning

We present a systematic assessment of polygenic risk score (PRS) prediction across more than 1,500 traits using genetic and phenotype data in the UK Biobank. We report 813 sparse PRS models with significant (p < 2.5 x 10−5) incremental predictive performance when compared against the covariate-only model that considers age, sex, types of genotyping arrays, and the principal component loadings of genotypes. We report a significant correlation between the number of genetic variants selected in the sparse PRS model and the incremental predictive performance (Spearman’s ⍴ = 0.61, p = 2.2 x 10−59 for quantitative traits, ⍴ = 0.21, p = 9.6 x 10−4 for binary traits). The sparse PRS model trained on European individuals showed limited transferability when evaluated on non-European individuals in the UK Biobank. We provide the PRS model weights on the Global Biobank Engine (https://biobankengine.stanford.edu/prs).

Significant sparse polygenic risk scores across 813 traits in UK Biobank

When applying hierarchical clustering algorithms to cluster patient samples from microarray data, the clustering patterns generated by most algorithms tend to be dominated by groups of highly differentially expressed genes that have closely related expression patterns. Sometimes, these genes may not be relevant to the biological process under study or their functions may already be known. The problem is that these genes can potentially drown out the effects of other genes that are relevant or have novel functions. We propose a procedure called complementary hierarchical clustering that is designed to uncover the structures arising from these novel genes that are not as highly expressed. Simulation studies show that the procedure is effective when applied to a variety of examples. We also define a concept called relative gene importance that can be used to identify the influential genes in a given clustering. Finally, we analyze a microarray data set from 295 breast cancer patients, using clustering with the correlation-based distance measure. The complementary clustering reveals a grouping of the patients which is uncorrelated with a number of known prognostic signatures and significantly differing distant metastasis-free probabilities.

/pdf/complementary-hierarchical-clustering-3an9olntpw.pdf

Complementary Hierarchical Clustering

Given a predictor of outcome derived from a high-dimensional
 dataset, pre-validation is a useful technique for comparing it
 to competing predictors on the same dataset. For microarray
 data, it allows one to compare a newly derived predictor for
 disease outcome to standard clinical predictors on the same
 dataset. We study pre-validation analytically to determine if
 the inferences drawn from it are valid. We show that while
 pre-validation generally works well, the straightforward “one
 degree of freedom” analytical test from pre-validation can be
 biased and we propose a permutation test to remedy this problem.
 In simulation studies, we show that the permutation test has the
 nominal level and achieves roughly the same power as the
 analytical test.

/pdf/a-study-of-pre-validation-36vs0vylth.pdf

A study of pre-validation

Helicobacter pylori causes gastric preneoplasia and neoplasia Eradicating H pylori can result in partial regression of preneoplastic lesions; however, the molecular underpinning of this change is unknown To identify molecular changes in the gastric mucosa following H pylori eradication, we used cDNA microarrays (with each array containing approximately 30,300 genes) to analyze 54 gastric biopsies from a randomized, placebo-controlled trial of H pylori therapy The 54 biopsies were obtained from 27 subjects (13 from the treatment and 14 from the placebo group) with chronic gastritis, atrophy, and/or intestinal metaplasia Each subject contributed one biopsy before and another biopsy 1 year after the intervention Significant analysis of microarrays (SAM) was used to compare the gene expression profiles of pre-intervention and post-intervention biopsies In the treatment group, SAM identified 30 genes whose expression changed significantly from baseline to 1 year after treatment (0 up-regulated and 30 down-regulated) In the placebo group, the expression of 55 genes differed significantly over the 1-year period (32 up-regulated and 23 down-regulated) Five genes involved in cell-cell adhesion and lining (TACSTD1 and MUC13), cell cycle differentiation (S100A10), and lipid metabolism and transport (FABP1 and MTP) were down-regulated over time in the treatment group but up-regulated in the placebo group Immunohistochemistry for one of these differentially expressed genes (FABP1) confirmed the changes in gene expression observed by microarray In conclusion, H pylori eradication may stop or reverse ongoing molecular processes in the stomach Further studies are needed to evaluate the use of these genes as markers for gastric cancer risk

https://cebp.aacrjournals.org/content/cebp/15/2/272.full.pdf

Changes of Gene Expression in Gastric Preneoplasia following Helicobacter pylori Eradication Therapy

Since its first proposal in statistics (Tibshirani, 1996), the lasso has been an effective method for simultaneous variable selection and estimation. A number of packages have been developed to solve the lasso efficiently. However as large datasets become more prevalent, many algorithms are constrained by efficiency or memory bounds. In this paper, we propose a meta algorithm batch screening iterative lasso (BASIL) that can take advantage of any existing lasso solver and build a scalable lasso solution for large datasets. We also introduce snpnet, an R package that implements the proposed algorithm on top of glmnet (Friedman et al., 2010a) for large-scale single nucleotide polymorphism (SNP) datasets that are widely studied in genetics. We demonstrate results on a large genotype-phenotype dataset from the UK Biobank, where we achieve state-of-the-art heritability estimation on quantitative and qualitative traits including height, body mass index, asthma and high cholesterol.

/pdf/a-fast-and-flexible-algorithm-for-solving-the-lasso-in-large-2fgt7pe3h1.pdf

Robert Tibshirani

Papers

Significant sparse polygenic risk scores across 813 traits in UK Biobank

Complementary Hierarchical Clustering

A study of pre-validation

Changes of Gene Expression in Gastric Preneoplasia following Helicobacter pylori Eradication Therapy

A Fast and Flexible Algorithm for Solving the Lasso in Large-scale and Ultrahigh-dimensional Problems