It is shown that in the context of generalized linear models, such methods possess model selection consistency with oracle properties even for dimensionality of nonpolynomial order of sample size, for a class of penalized likelihood approaches using folded-concave penalty functions, which were introduced to ameliorate the bias problems of convex penalty functions.
Abstract:
Penalized likelihood methods are fundamental to ultrahigh dimensional variable selection. How high dimensionality such methods can handle remains largely unknown. In this paper, we show that in the context of generalized linear models, such methods possess model selection consistency with oracle properties even for dimensionality of nonpolynomial (NP) order of sample size, for a class of penalized likelihood approaches using folded-concave penalty functions, which were introduced to ameliorate the bias problems of convex penalty functions. This fills a long-standing gap in the literature where the dimensionality is allowed to grow slowly with the sample size. Our results are also applicable to penalized likelihood with the L1-penalty, which is a convex function at the boundary of the class of folded-concave penalty functions under consideration. The coordinate optimization is implemented for finding the solution paths, whose performance is evaluated by a few simulation examples and the real data analysis.
TL;DR: It is shown that, if an FCP-regularized SAA formulation is solved locally, then the required number of samples can be significantly reduced in approximating the global solution of a convex SP: the sample size is only required to be poly-logarithmic in the number of dimensions.
TL;DR: In this article, a flexible and generally applicable missing data mechanism, which contains both ignorable and nonignorable missing data mechanisms assumptions, is presented, and the regularization approach for variable selection can be adapted to the situation under this missing data Mechanism.
TL;DR: This paper studies the adaptive group Lasso in the frame of high-dimensional generalized linear models and finds that the selection consistency and asymptotic normality of the adaptive groups Lasso are established under appropriate conditions.
TL;DR: Regularized selection of important fixed-effect variables in linear mixed-effect models along with maximum penalized likelihood estimation of both fixed and random-effect parameters based on general non-concave penalties are considered.
TL;DR: A new methodology of nonsparse learning with latent variables (NSL) is proposed to simultaneously recover the significant observable predictors and latent factors as well as their effects and properties including model selection consistency and oracle inequalities under various prediction and estimation losses are established.
TL;DR: A new method for estimation in linear models called the lasso, which minimizes the residual sum of squares subject to the sum of the absolute value of the coefficients being less than a constant, is proposed.
TL;DR: In this paper, a generalization of the analysis of variance is given for these models using log- likelihoods, illustrated by examples relating to four distributions; the Normal, Binomial (probit analysis, etc.), Poisson (contingency tables), and gamma (variance components).
TL;DR: This is the rst book on generalized linear models written by authors not mostly associated with the biological sciences, and it is thoroughly enjoyable to read.
TL;DR: In this article, upper bounds for the probability that the sum S of n independent random variables exceeds its mean ES by a positive number nt are derived for certain sums of dependent random variables such as U statistics.
Q1. What are the contributions mentioned in the paper "Nonconcave penalized likelihood with np-dimensionality" ?
In this paper, the authors show that in the context of generalized linear models, such methods possess model selection consistency with oracle properties even for dimensionality of nonpolynomial ( NP ) order of sample size, for a class of penalized likelihood approaches using folded-concave penalty functions, which were introduced to ameliorate the bias problems of convex penalty functions.
Q2. What is the condition for the local maximizer of the nonconcave penalized likelihood?
Then is a strict local maximizer of the nonconcave penalized likelihood defined by (3) if(7)(8)(9)where and respectively denote the submatrices of formed by columns in and its complement, ,is a subvector of formed by all nonzero components, and .
Q3. What is the dimensionality of the penalized least squares?
In this case, the dimensionality that the penalized least-squares can handle is as high as when, which is usually smaller than that for the case of .
Q4. What is the condition of the Gaussian linear regression model?
Condition (16) controls the uniform growth rate of the -norm of these multiple regression coefficients, a notion of weak correlation between and .
Q5. What is the definition of a coordinate subspace?
A subspace of is called coordinate subspace if it is spanned by a subset of the natural basis , where each is the -vector with th component 1 and 0 elsewhere.
Q6. what is the maxi-mizer of the penalized likelihood?
Then there exists a strict local maxi-mizer of the penalized likelihood such that with probability tending to 1 as and, where is a subvector of formed by components in .
Q7. What is the simplest way to show that the second derivative of the penalty function does not exist?
More generally, when the second derivative of the penalty function does not necessarily exist, it is easy to show that the second part of the matrix can be replaced by a diagonal matrix with maximum absolute element bounded by .
Q8. What is the concavity of the convex set?
By the concavity of , the authors can easily show that for , is a closed convex set with and being its interior points and the level set is its boundary.
Q9. What is the second order approximation in ICA?
When is quadratic in , e.g., for the Gaussian linear regression model, the second order approximation in ICA is exact at each step.
Q10. Why do the authors examine the implications of Theorem 2?
Due to its popularity, the authors now examine the implications of Theorem 2 in the context of penalized least-squares and penalized likelihood.