Random Forests
Leo Breiman
- Vol. 45, Iss: 1, pp 5-32
TLDR
Internal estimates monitor error, strength, and correlation and these are used to show the response to increasing the number of features used in the forest, and are also applicable to regression.Abstract:
Random forests are a combination of tree predictors such that each tree depends on the values of a random vector sampled independently and with the same distribution for all trees in the forest. The generalization error for forests converges a.s. to a limit as the number of trees in the forest becomes large. The generalization error of a forest of tree classifiers depends on the strength of the individual trees in the forest and the correlation between them. Using a random selection of features to split each node yields error rates that compare favorably to Adaboost (Y. Freund & R. Schapire, Machine Learning: Proceedings of the Thirteenth International conference, aaa, 148–156), but are more robust with respect to noise. Internal estimates monitor error, strength, and correlation and these are used to show the response to increasing the number of features used in the splitting. Internal estimates are also used to measure variable importance. These ideas are also applicable to regression.read more
Citations
More filters
Journal ArticleDOI
Simple decision rules for classifying human cancers from gene expression profiles
TL;DR: The k-TSP classifier performs as efficiently as Prediction Analysis of Microarray and support vector machine, and outperforms other learning methods (decision trees, k-nearest neighbour and naïve Bayes) and is easy to interpret as the classifier involves only a small number of informative genes.
Proceedings ArticleDOI
Practical Evasion of a Learning-Based Classifier: A Case Study
Nedim rndic,Pavel Laskov +1 more
TL;DR: A taxonomy for practical evasion strategies and adapt known evasion algorithms to implement specific scenarios in the authors' taxonomy is developed and a substantial drop of PDFrate's classification scores and detection accuracy is revealed after it is exposed even to simple attacks.
Journal ArticleDOI
Meta-analysis of cellular toxicity for cadmium-containing quantum dots
Eunkeu Oh,Rong Liu,Andre E. Nel,Kelly Boeneman Gemill,Muhammad Bilal,Yoram Cohen,Igor L. Medintz +6 more
TL;DR: The approach of integrating quantitative and categorical data provides a roadmap for interrogating the wide-ranging toxicity data in the literature and suggests that meta-analysis can help develop methods for predicting the toxicity of engineered nanomaterials.
Journal ArticleDOI
An analysis of diversity measures
TL;DR: A theoretical analysis on six existing diversity measures is presented, show underlying relationships between them, and relate them to the concept of margin, which is more explicitly related to the success of ensemble learning algorithms.
Journal ArticleDOI
Estimating PM2.5 Concentrations in the Conterminous United States Using the Random Forest Approach.
Xuefei Hu,Jessica H. Belle,Xia Meng,Avani Wildani,Lance A. Waller,Matthew J. Strickland,Yang Liu +6 more
TL;DR: A random forest model incorporating aerosol optical depth data, meteorological fields, and land use variables to estimate daily 24 h averaged ground-level PM2.5 concentrations over the conterminous United States in 2011 is developed.
References
More filters
Journal ArticleDOI
Bagging predictors
TL;DR: Tests on real and simulated data sets using classification and regression trees and subset selection in linear regression show that bagging can give substantial gains in accuracy.
Proceedings Article
Experiments with a new boosting algorithm
Yoav Freund,Robert E. Schapire +1 more
TL;DR: This paper describes experiments carried out to assess how well AdaBoost with and without pseudo-loss, performs on real learning problems and compared boosting to Breiman's "bagging" method when used to aggregate various classifiers.
Journal ArticleDOI
The random subspace method for constructing decision forests
TL;DR: A method to construct a decision tree based classifier is proposed that maintains highest accuracy on training data and improves on generalization accuracy as it grows in complexity.
Journal ArticleDOI
An Experimental Comparison of Three Methods for Constructing Ensembles of Decision Trees: Bagging, Boosting, and Randomization
TL;DR: In this article, the authors compared the effectiveness of randomization, bagging, and boosting for improving the performance of the decision-tree algorithm C4.5 and found that in situations with little or no classification noise, randomization is competitive with bagging but not as accurate as boosting.
Journal ArticleDOI
An Empirical Comparison of Voting Classification Algorithms: Bagging, Boosting, and Variants
Eric Bauer,Ron Kohavi +1 more
TL;DR: It is found that Bagging improves when probabilistic estimates in conjunction with no-pruning are used, as well as when the data was backfit, and that Arc-x4 behaves differently than AdaBoost if reweighting is used instead of resampling, indicating a fundamental difference.