# EfficientL 1 regularized logistic regression

TL;DR: Theoretical results show that the proposed efficient algorithm for L1 regularized logistic regression is guaranteed to converge to the global optimum, and experiments show that it significantly outperforms standard algorithms for solving convex optimization problems.

Abstract: L1 regularized logistic regression is now a workhorse of machine learning: it is widely used for many classification problems, particularly ones with many features. L1 regularized logistic regression requires solving a convex optimization problem. However, standard algorithms for solving convex optimization problems do not scale well enough to handle the large datasets encountered in many practical settings. In this paper, we propose an efficient algorithm for L1 regularized logistic regression. Our algorithm iteratively approximates the objective function by a quadratic approximation at the current point, while maintaining the L1 constraint. In each iteration, it uses the efficient LARS (Least Angle Regression) algorithm to solve the resulting L1 constrained quadratic optimization problem. Our theoretical results show that our algorithm is guaranteed to converge to the global optimum. Our experiments show that our algorithm significantly outperforms standard algorithms for solving convex optimization problems. Moreover, our algorithm outperforms four previously published algorithms that were specifically designed to solve the L1 regularized logistic regression problem.

...read more

##### Citations

6,284 citations

716 citations

596 citations

^{1}

580 citations

### Cites methods from "EfficientL 1 regularized logistic r..."

...Lee et al. (2006) propose the algorithm irls-lars, inspired by Newton’s method, which iteratively minimizes the function’s second order Taylor expansion, subject to linear constraints....

[...]

...Lee et al. (2006) propose the algorithm irls-lars, inspired by Newton s method, which iteratively minimizes the function s second order Taylor expansion, subject to linear constraints....

[...]

##### References

36,018 citations

### "EfficientL 1 regularized logistic r..." refers methods in this paper

...(Tibshirani 1996) Several algorithms have been developed to solve L1 constrained least squares problems....

[...]

...See, Tibshirani (1996) for details.)...

[...]

...(Tibshirani 1996) Several algorithms have been developed to solve L1 constrained least squares problems....

[...]

[...]

33,299 citations

23,204 citations

12,776 citations

### "EfficientL 1 regularized logistic r..." refers methods in this paper

...We tested each algorithm’s performance on 12 different datasets, consisting of 9 UCI datasets (Newman et al. 1998), one artificial dataset called Madelon from the NIPS 2003 workshop on feature extraction,3 and two gene expression datasets (Microarray 1 and 2).4 Table 2 gives details on the number…...

[...]

...We tested each algorithm’s performance on 12 different real datasets, consisting of 9 UCI datasets (Newman et al. 1998) and 3 gene expression datasets (Microarray 1, 2 and 3) 3....

[...]