scispace - formally typeset
Topic

Feature selection

About: Feature selection is a(n) research topic. Over the lifetime, 41478 publication(s) have been published within this topic receiving 1024563 citation(s). The topic is also known as: attribute selection.

...read more

Papers
  More

Open accessJournal ArticleDOI: 10.1111/J.1467-9868.2005.00503.X
Hui Zou1, Trevor Hastie1Institutions (1)
Abstract: Summary. We propose the elastic net, a new regularization and variable selection method. Real world data and a simulation study show that the elastic net often outperforms the lasso, while enjoying a similar sparsity of representation. In addition, the elastic net encourages a grouping effect, where strongly correlated predictors tend to be in or out of the model together.The elastic net is particularly useful when the number of predictors (p) is much bigger than the number of observations (n). By contrast, the lasso is not a very satisfactory variable selection method in the

...read more

  • Table 1: Prostate cancer data: comparing different methods
    Table 1: Prostate cancer data: comparing different methods
  • Figure 2: Exact solutions for the lasso, ridge and the naive elastic net (naive ENet) in an orthogonal design. Shrinkage parameters are λ1 = 2, λ2 = 1.
    Figure 2: Exact solutions for the lasso, ridge and the naive elastic net (naive ENet) in an orthogonal design. Shrinkage parameters are λ1 = 2, λ2 = 1.
  • Table 4: Summary of leukemia classification results
    Table 4: Summary of leukemia classification results
  • Table 2: Median of MSE, inside () are the corresponding std. errors based on B = 500 Bootstrap.
    Table 2: Median of MSE, inside () are the corresponding std. errors based on B = 500 Bootstrap.
  • Figure 6: Leukemia classification and gene selection by the elastic net(λ = 0.01). The early stopping strategy (the upper plot) finds the optimal classifier with much less computational cost. With early stopping, the number of steps is much more convenient than s, the fraction of L1 norm, since computing s depends on the fit at the last step of the LARS-EN algorithm, the actual values of s are not available in 10-fold cross-validation if the LARS-EN algorithm is early stopped. On the training set, steps=200 is equivalent to s = 0.50, indicated by the broken vertical line in the lower plot.
    Figure 6: Leukemia classification and gene selection by the elastic net(λ = 0.01). The early stopping strategy (the upper plot) finds the optimal classifier with much less computational cost. With early stopping, the number of steps is much more convenient than s, the fraction of L1 norm, since computing s depends on the fit at the last step of the LARS-EN algorithm, the actual values of s are not available in 10-fold cross-validation if the LARS-EN algorithm is early stopped. On the training set, steps=200 is equivalent to s = 0.50, indicated by the broken vertical line in the lower plot.
  • + 6

13,722 Citations


Open accessJournal ArticleDOI: 10.1162/153244303322753616
Isabelle Guyon, André Elisseeff1Institutions (1)
Abstract: Variable and feature selection have become the focus of much research in areas of application for which datasets with tens or hundreds of thousands of variables are available. These areas include text processing of internet documents, gene expression array analysis, and combinatorial chemistry. The objective of variable selection is three-fold: improving the prediction performance of the predictors, providing faster and more cost-effective predictors, and providing a better understanding of the underlying process that generated the data. The contributions of this special issue cover a wide range of aspects of such problems: providing a better definition of the objective function, feature construction, feature ranking, multivariate feature selection, efficient search methods, and feature validity assessment methods.

...read more

13,554 Citations


Proceedings ArticleDOI: 10.1109/CVPR.1994.323794
Jianbo Shi1, Tomasi2Institutions (2)
21 Jun 1994-
Abstract: No feature-based vision system can work unless good features can be identified and tracked from frame to frame. Although tracking itself is by and large a solved problem, selecting features that can be tracked well and correspond to physical points in the world is still hard. We propose a feature selection criterion that is optimal by construction because it is based on how the tracker works, and a feature monitoring method that can detect occlusions, disocclusions, and features that do not correspond to points in the world. These methods are based on a new tracking algorithm that extends previous Newton-Raphson style search methods to work under affine image transformations. We test performance with several simulations and experiments. >

...read more

Topics: Feature (computer vision) (61%), Feature extraction (56%), Feature selection (54%) ...read more

8,046 Citations


Open accessJournal ArticleDOI: 10.1016/S0004-3702(97)00043-X
Abstract: In the feature subset selection problem, a learning algorithm is faced with the problem of selecting a relevant subset of features upon which to focus its attention, while ignoring the rest. To achieve the best possible performance with a particular learning algorithm on a particular training set, a feature subset selection method should consider how the algorithm and the training set interact. We explore the relation between optimal feature subset selection and relevance. Our wrapper method searches for an optimal feature subset tailored to a particular algorithm and a domain. We study the strengths and weaknesses of the wrapper approach and show a series of improved designs. We compare the wrapper approach to induction without feature subset selection and to Relief, a filter approach to feature subset selection. Significant improvement in accuracy is achieved for some datasets for the two families of induction algorithms used: decision trees and Naive-Bayes.

...read more

7,958 Citations


Open accessJournal ArticleDOI: 10.1214/009053604000000067
Bradley Efron1, Trevor Hastie1, Iain M. Johnstone1, Robert Tibshirani1  +16 moreInstitutions (13)
Abstract: The purpose of model selection algorithms such as All Subsets, Forward Selection and Backward Elimination is to choose a linear model on the basis of the same set of data to which the model will be applied. Typically we have available a large collection of possible covariates from which we hope to select a parsimonious set for the efficient prediction of a response variable. Least Angle Regression (LARS), a new model selection algorithm, is a useful and less greedy version of traditional forward selection methods. Three main properties are derived: (1) A simple modification of the LARS algorithm implements the Lasso, an attractive version of ordinary least squares that constrains the sum of the absolute regression coefficients; the LARS modification calculates all possible Lasso estimates for a given problem, using an order of magnitude less computer time than previous methods. (2) A different LARS modification efficiently implements Forward Stagewise linear regression, another promising new model selection method; this connection explains the similar numerical results previously observed for the Lasso and Stagewise, and helps us understand the properties of both methods, which are seen as constrained versions of the simpler LARS algorithm. (3) A simple approximation for the degrees of freedom of a LARS estimate is available, from which we derive a Cp estimate of prediction error; this allows a principled choice among the range of possible LARS estimates. LARS and its variants are computationally efficient: the paper describes a publicly available algorithm that requires only the same order of magnitude of computational effort as ordinary least squares applied to the full set of covariates.

...read more

Topics: Elastic net regularization (56%), Model selection (55%), Linear model (55%) ...read more

7,274 Citations


Performance
Metrics
No. of papers in the topic in previous years
YearPapers
2022127
20213,652
20203,705
20193,673
20183,180
20172,867

Top Attributes

Show by:

Topic's top 5 most impactful authors

Mengjie Zhang

113 papers, 4.8K citations

Huan Liu

83 papers, 18.7K citations

Bing Xue

83 papers, 3.7K citations

Amparo Alonso-Betanzos

64 papers, 1.7K citations

Verónica Bolón-Canedo

60 papers, 1.5K citations

Network Information
Related Topics (5)
Support vector machine

73.6K papers, 1.7M citations

96% related
Dimensionality reduction

21.9K papers, 579.2K citations

96% related
Supervised learning

20.8K papers, 710.5K citations

94% related
Feature vector

48.8K papers, 954.4K citations

93% related
Kernel method

11.3K papers, 501K citations

92% related