Summary. We propose the elastic net, a new regularization and variable selection method. Real world data and a simulation study show that the elastic net often outperforms the lasso, while enjoying a similar sparsity of representation. In addition, the elastic net encourages a grouping effect, where strongly correlated predictors tend to be in or out of the model together.The elastic net is particularly useful when the number of predictors (p) is much bigger than the number of observations (n). By contrast, the lasso is not a very satisfactory variable selection method in the

/pdf/regularization-and-variable-selection-via-the-elastic-net-4z8q6j5o0i.pdf

Regularization and variable selection via the elastic net

We develop fast algorithms for estimation of generalized linear models with convex penalties. The models include linear regression, two-class logistic regression, and multinomial regression problems while the penalties include l(1) (the lasso), l(2) (ridge regression) and mixtures of the two (the elastic net). The algorithms use cyclical coordinate descent, computed along a regularization path. The methods can handle large problems and can also deal efficiently with sparse features. In comparative timings we find that the new algorithms are considerably faster than competing methods.

/pdf/regularization-paths-for-generalized-linear-models-via-39r1t606ac.pdf

Regularization Paths for Generalized Linear Models via Coordinate Descent

Today's Web-enabled deluge of electronic data calls for automated methods of data analysis. Machine learning provides these, developing methods that can automatically detect patterns in data and then use the uncovered patterns to predict future data. This textbook offers a comprehensive and self-contained introduction to the field of machine learning, based on a unified, probabilistic approach. The coverage combines breadth and depth, offering necessary background material on such topics as probability, optimization, and linear algebra as well as discussion of recent developments in the field, including conditional random fields, L1 regularization, and deep learning. The book is written in an informal, accessible style, complete with pseudo-code for the most important algorithms. All topics are copiously illustrated with color images and worked examples drawn from such application domains as biology, text processing, computer vision, and robotics. Rather than providing a cookbook of different heuristic methods, the book stresses a principled model-based approach, often using the language of graphical models to specify models in a concise and intuitive way. Almost all the models described have been implemented in a MATLAB software package--PMTK (probabilistic modeling toolkit)--that is freely available online. The book is suitable for upper-level undergraduates with an introductory-level college math background and beginning graduate students.

Machine Learning : A Probabilistic Perspective

Summary. We consider the problem of selecting grouped variables (factors) for accurate prediction in regression. Such a problem arises naturally in many practical situations with the multifactor analysis-of-variance problem as the most important and well-known example. Instead of selecting factors by stepwise backward elimination, we focus on the accuracy of estimation and consider extensions of the lasso, the LARS algorithm and the non-negative garrotte for factor selection. The lasso, the LARS algorithm and the non-negative garrotte are recently proposed regression methods that can be used to select individual variables. We study and propose efficient algorithms for the extensions of these methods for factor selection and show that these extensions give superior performance to the traditional stepwise backward elimination method in factor selection problems. We study the similarities and the differences between these methods. Simulations and real examples are used to illustrate the methods.

/pdf/model-selection-and-estimation-in-regression-with-grouped-45w0oq79ry.pdf

Model selection and estimation in regression with grouped variables

The lasso is a popular technique for simultaneous estimation and variable selection. Lasso variable selection has been shown to be consistent under certain conditions. In this work we derive a necessary condition for the lasso variable selection to be consistent. Consequently, there exist certain scenarios where the lasso is inconsistent for variable selection. We then propose a new version of the lasso, called the adaptive lasso, where adaptive weights are used for penalizing different coefficients in the l1 penalty. We show that the adaptive lasso enjoys the oracle properties; namely, it performs as well as if the true underlying model were given in advance. Similar to the lasso, the adaptive lasso is shown to be near-minimax optimal. Furthermore, the adaptive lasso can be solved by the same efficient algorithm for solving the lasso. We also discuss the extension of the adaptive lasso in generalized linear models and show that the oracle properties still hold under mild regularity conditions. As a bypro...

/pdf/the-adaptive-lasso-and-its-oracle-properties-p1nmst08nh.pdf

The adaptive lasso and its oracle properties

The multilocus analysis of polymorphisms has emerged as a vital ingredient of population genetics and evolutionary biology. A fundamental assumption used for existing multilocus analysis approaches is Hardy-Weinberg equilibrium (HWE) at which maternally- and paternally-derived gametes combine randomly during fertilization. Given the fact that natural populations are rarely panmictic, these approaches will have a significant limitation for practical use. We present a robust model for multilocus linkage disequilibrium analysis which does not rely on the assumption of random mating. This new disequilibrium model capitalizes on Weir’s definition of zygotic disequilibria and is based on an open-pollinated design in which multiple maternal individuals and their half-sib families are sampled from a natural population. This design captures two levels of associations: one is at the upper level that describes the pattern of co-segregation between different loci in the parental population and the other is at the lower level that specifies the extent of co-transmission of non-alleles at different loci from parents to their offspring. An MCMC method was implemented to estimate genetic parameters that define these associations. Simulation studies were used to validate the statistical behavior of the new model.

/pdf/model-and-algorithm-for-linkage-disequilibrium-analysis-in-a-1nxi6dovwe.pdf

Model and algorithm for linkage disequilibrium analysis in a non-equilibrium population.

Motivated by an empirical analysis of data from a genome-wide association study on obesity, measured by the body mass index (BMI), we propose a two-step gene-detection procedure for generalized varying coefficient mixed-effects models with ultrahigh dimensional covariates. The proposed procedure selects significant single nucleotide polymorphisms (SNPs) impacting the mean BMI trend, some of which have already been biologically proven to be “fat genes.” The method also discovers SNPs that significantly influence the age-dependent variability of BMI. The proposed procedure takes into account individual variations of genetic effects and can also be directly applied to longitudinal data with continuous, binary or count responses. We employ Monte Carlo simulation studies to assess the performance of the proposed method and further carry out causal inference for the selected SNPs.

Feature selection for generalized varying coefficient mixed-effect models with application to obesity gwas

Dendrobium huoshanense is both a traditional herbal medicine and a plant of high ornamental and medicinal value. We used transcriptomics and metabolomics to investigate the effects of growth year on the secondary metabolites of D. huoshanense stems obtained from four different years of cultivation. In this study, a total of 428 differentially accumulated metabolites (DAMs) and 1802 differentially expressed genes (DEGs) were identified. The KEGG enrichment analysis of DEGs and DAMs revealed significant differences in “Flavonoid biosynthesis”, “Phenylpropanoid biosynthesis” and “Flavone and flavonol biosynthesis”. We summarize the biosynthesis pathway of flavonoids in D. huoshanense, providing new insights into the biosynthesis and regulation mechanisms of flavonoids in D. huoshanense. Additionally, we identified two candidate genes, FLS (LOC110107557) and F3’H (LOC110095936), which are highly involved in flavonoid biosynthesis pathway, by WGCNA analysis. The aim of this study is to investigate the effects of growth year on secondarily metabolites in the plant and provide a theoretical basis for determining a reasonable harvesting period for D. huoshanense.

Integration of Transcriptome and Metabolome Provides New Insights to Flavonoids Biosynthesis in Dendrobium huoshanense

We propose a copula-based joint modeling framework for mixed longitudinal responses. Our approach permits all model parameters to vary with time, and thus will enable researchers to reveal dynamic response-predictor relationships and response-response associations. We call the new class of models TIMECOP because we model dependence using a time-varying copula. We develop a one-step estimation procedure for the TIMECOP parameter vector, and also describe how to estimate standard errors. We investigate the finite sample performance of our procedure via three simulation studies, one of which shows that our procedure performs well under ignorable missingness. We also illustrate the applicability of our approach by analyzing binary and continuous responses from the Women's Interagency HIV Study and a smoking cessation program.

/pdf/time-varying-copula-models-for-longitudinal-data-2pzzhfcgni.pdf

Runze Li

Papers

Model and algorithm for linkage disequilibrium analysis in a non-equilibrium population.

Feature selection for generalized varying coefficient mixed-effect models with application to obesity gwas

Integration of Transcriptome and Metabolome Provides New Insights to Flavonoids Biosynthesis in Dendrobium huoshanense

Time-varying copula models for longitudinal data.

An overview on variable selection for survival data analysis