Tuning parameter selection in high dimensional penalized likelihood
Yingying Fan,Cheng Yong Tang +1 more
Reads0
Chats0
TLDR
In this article, the authors proposed to select the tuning parameter by optimizing the generalized information criterion with an appropriate model complexity penalty, which diverges at the rate of some power of ǫ(p) depending on the tail probability behavior of the response variables.Abstract:
Summary
Determining how to select the tuning parameter appropriately is essential in penalized likelihood methods for high dimensional data analysis. We examine this problem in the setting of penalized likelihood methods for generalized linear models, where the dimensionality of covariates p is allowed to increase exponentially with the sample size n. We propose to select the tuning parameter by optimizing the generalized information criterion with an appropriate model complexity penalty. To ensure that we consistently identify the true model, a range for the model complexity penalty is identified in the generlized information criterion. We find that this model complexity penalty should diverge at the rate of some power of log (p) depending on the tail probability behaviour of the response variables. This reveals that using the Akaike information criterion or Bayes information criterion to select the tuning parameter may not be adequate for consistently identifying the true model. On the basis of our theoretical study, we propose a uniform choice of the model complexity penalty and show that the approach proposed consistently identifies the true model among candidate models with asymptotic probability 1. We justify the performance of the procedure proposed by numerical simulations and a gene expression data analysis.read more
Citations
More filters
Journal ArticleDOI
Cross-Fitted Residual Regression for High-Dimensional Heteroscedasticity Pursuit
TL;DR: This paper presents a meta-modelling framework that automates the very labor-intensive and therefore time-heavy and therefore expensive process of integrating data from a number of models into a single model.
Journal ArticleDOI
A comparative machine learning approach for entropy-based damage detection using output-only correlation signal
Journal ArticleDOI
Logical and test consistency in pairwise multiple comparisons
TL;DR: The authors reformulated pairwise comparisons as penalized likelihood estimation problems based on the L 1 penalties on the pairwise differences and showed that under appropriate conditions, all nulls are not rejected and all alternatives are rejected with probability going to one.
Journal ArticleDOI
Sequential Scaled Sparse Factor Regression
TL;DR: In this article, large-scale association analysis between multivariate responses and predictors is of great practical importance, as exemplified by modern business applications including social media marketing and online advertising.
Dissertation
Latent Class Models: Design and Diagnosis
TL;DR: This dissertation presents a meta-modelling system that automates the very labor-intensive and therefore time-heavy and expensive process of manually cataloging and cataloging individual pieces of data to provide real-time information about their owners.
References
More filters
Journal ArticleDOI
Regression Shrinkage and Selection via the Lasso
TL;DR: A new method for estimation in linear models called the lasso, which minimizes the residual sum of squares subject to the sum of the absolute value of the coefficients being less than a constant, is proposed.
Journal ArticleDOI
Estimating the Dimension of a Model
TL;DR: In this paper, the problem of selecting one of a number of models of different dimensions is treated by finding its Bayes solution, and evaluating the leading terms of its asymptotic expansion.
Book
Generalized Linear Models
Peter McCullagh,John A. Nelder +1 more
TL;DR: In this paper, a generalization of the analysis of variance is given for these models using log- likelihoods, illustrated by examples relating to four distributions; the Normal, Binomial (probit analysis, etc.), Poisson (contingency tables), and gamma (variance components).
Book
The Elements of Statistical Learning: Data Mining, Inference, and Prediction
TL;DR: In this paper, the authors describe the important ideas in these areas in a common conceptual framework, and the emphasis is on concepts rather than mathematics, with a liberal use of color graphics.
Book ChapterDOI
Information Theory and an Extension of the Maximum Likelihood Principle
TL;DR: In this paper, it is shown that the classical maximum likelihood principle can be considered to be a method of asymptotic realization of an optimum estimate with respect to a very general information theoretic criterion.