It is shown that in the context of generalized linear models, such methods possess model selection consistency with oracle properties even for dimensionality of nonpolynomial order of sample size, for a class of penalized likelihood approaches using folded-concave penalty functions, which were introduced to ameliorate the bias problems of convex penalty functions.
Abstract:
Penalized likelihood methods are fundamental to ultrahigh dimensional variable selection. How high dimensionality such methods can handle remains largely unknown. In this paper, we show that in the context of generalized linear models, such methods possess model selection consistency with oracle properties even for dimensionality of nonpolynomial (NP) order of sample size, for a class of penalized likelihood approaches using folded-concave penalty functions, which were introduced to ameliorate the bias problems of convex penalty functions. This fills a long-standing gap in the literature where the dimensionality is allowed to grow slowly with the sample size. Our results are also applicable to penalized likelihood with the L1-penalty, which is a convex function at the boundary of the class of folded-concave penalty functions under consideration. The coordinate optimization is implemented for finding the solution paths, whose performance is evaluated by a few simulation examples and the real data analysis.
TL;DR: This work proposes a two-stage regularization framework for identifying and estimating important covariate effects while selecting and estimating optimal instruments in the high-dimensional setting where the dimensionality of covariates and instruments are both allowed to grow exponentially with the sample size.
TL;DR: In this article, a weighted composite quantile regression (WCQR) estimation approach is proposed for model selection for nonlinear models with a diverging number of parameters, which is augmented using a data-driven weighting scheme.
TL;DR: An extension of LASSO, namely, prior LassO (pLASSO), is proposed, to incorporate that prior information into penalized generalized linear models and shows great robustness to the misspecification.
TL;DR: This work proposes efficient procedures for learning a sparse Ising model based on a penalized composite conditional likelihood with nonconcave penalties and demonstrates its finite sample performance via simulation studies and illustrated by studying the Human Immunodeficiency Virus type 1 protease structure.
TL;DR: It is shown that under suitable technical conditions, the structure of the undirected graphical model can be consistently estimated in the high dimensional setting, when the dimensionality of the model is allowed to diverge with the sample size.
TL;DR: A new method for estimation in linear models called the lasso, which minimizes the residual sum of squares subject to the sum of the absolute value of the coefficients being less than a constant, is proposed.
TL;DR: In this paper, a generalization of the analysis of variance is given for these models using log- likelihoods, illustrated by examples relating to four distributions; the Normal, Binomial (probit analysis, etc.), Poisson (contingency tables), and gamma (variance components).
TL;DR: This is the rst book on generalized linear models written by authors not mostly associated with the biological sciences, and it is thoroughly enjoyable to read.
TL;DR: In this article, upper bounds for the probability that the sum S of n independent random variables exceeds its mean ES by a positive number nt are derived for certain sums of dependent random variables such as U statistics.
Q1. What are the contributions mentioned in the paper "Nonconcave penalized likelihood with np-dimensionality" ?
In this paper, the authors show that in the context of generalized linear models, such methods possess model selection consistency with oracle properties even for dimensionality of nonpolynomial ( NP ) order of sample size, for a class of penalized likelihood approaches using folded-concave penalty functions, which were introduced to ameliorate the bias problems of convex penalty functions.
Q2. What is the condition for the local maximizer of the nonconcave penalized likelihood?
Then is a strict local maximizer of the nonconcave penalized likelihood defined by (3) if(7)(8)(9)where and respectively denote the submatrices of formed by columns in and its complement, ,is a subvector of formed by all nonzero components, and .
Q3. What is the dimensionality of the penalized least squares?
In this case, the dimensionality that the penalized least-squares can handle is as high as when, which is usually smaller than that for the case of .
Q4. What is the condition of the Gaussian linear regression model?
Condition (16) controls the uniform growth rate of the -norm of these multiple regression coefficients, a notion of weak correlation between and .
Q5. What is the definition of a coordinate subspace?
A subspace of is called coordinate subspace if it is spanned by a subset of the natural basis , where each is the -vector with th component 1 and 0 elsewhere.
Q6. what is the maxi-mizer of the penalized likelihood?
Then there exists a strict local maxi-mizer of the penalized likelihood such that with probability tending to 1 as and, where is a subvector of formed by components in .
Q7. What is the simplest way to show that the second derivative of the penalty function does not exist?
More generally, when the second derivative of the penalty function does not necessarily exist, it is easy to show that the second part of the matrix can be replaced by a diagonal matrix with maximum absolute element bounded by .
Q8. What is the concavity of the convex set?
By the concavity of , the authors can easily show that for , is a closed convex set with and being its interior points and the level set is its boundary.
Q9. What is the second order approximation in ICA?
When is quadratic in , e.g., for the Gaussian linear regression model, the second order approximation in ICA is exact at each step.
Q10. Why do the authors examine the implications of Theorem 2?
Due to its popularity, the authors now examine the implications of Theorem 2 in the context of penalized least-squares and penalized likelihood.