Book ChapterDOI
How many trees in a random forest
Thais Mayumi Oshiro,Pedro Santoro Perez,José Augusto Baranauskas +2 more
- pp 154-168
TLDR
Analysis of whether there is an optimal number of trees within a Random Forest finds an experimental relationship for the AUC gain when doubling the number of Trees in any forest and states there is a threshold beyond which there is no significant gain, unless a huge computational environment is available.Abstract:
Random Forest is a computationally efficient technique that can operate quickly over large datasets. It has been used in many recent research projects and real-world applications in diverse domains. However, the associated literature provides almost no directions about how many trees should be used to compose a Random Forest. The research reported here analyzes whether there is an optimal number of trees within a Random Forest, i.e., a threshold from which increasing the number of trees would bring no significant performance gain, and would only increase the computational cost. Our main conclusions are: as the number of trees grows, it does not always mean the performance of the forest is significantly better than previous forests (fewer trees), and doubling the number of trees is worthless. It is also possible to state there is a threshold beyond which there is no significant gain, unless a huge computational environment is available. In addition, it was found an experimental relationship for the AUC gain when doubling the number of trees in any forest. Furthermore, as the number of trees grows, the full set of attributes tend to be used within a Random Forest, which may not be interesting in the biomedical domain. Additionally, datasets' density-based metrics proposed here probably capture some aspects of the VC dimension on decision trees and low-density datasets may require large capacity machines whilst the opposite also seems to be true.read more
Citations
More filters
Journal ArticleDOI
Machine learning
TL;DR: Machine learning addresses many of the same research questions as the fields of statistics, data mining, and psychology, but with differences of emphasis.
Journal ArticleDOI
A survey on semi-supervised learning
TL;DR: This survey aims to provide researchers and practitioners new to the field as well as more advanced readers with a solid understanding of the main approaches and algorithms developed over the past two decades, with an emphasis on the most prominent and currently relevant work.
Journal ArticleDOI
Hyperparameters and tuning strategies for random forest
TL;DR: A literature review on the parameters' influence on the prediction performance and on variable importance measures is provided, and the application of one of the most established tuning strategies, model‐based optimization (MBO), is demonstrated.
Journal ArticleDOI
Hyperparameters and Tuning Strategies for Random Forest
TL;DR: In this article, the authors provide a literature review on the parameters' influence on the prediction performance and on variable importance measures, and demonstrate the application of one of the most established tuning strategies, model-based optimization (MBO).
Journal ArticleDOI
A machine learning calibration model using random forests to improve sensor performance for lower-cost air quality monitoring
Naomi Zimmerman,Albert A. Presto,Sriniwasa P. N. Kumar,Jason Gu,Aliaksei Hauryliuk,Ellis S. Robinson,Allen L. Robinson,R. Subramanian +7 more
TL;DR: In this paper, the Real-time Affordable Multi-Pollutant (RAMP) sensor package is used to measure CO, NO2, O3, and CO2.
References
More filters
Journal ArticleDOI
Controlling the false discovery rate: a practical and powerful approach to multiple testing
Yoav Benjamini,Yosef Hochberg +1 more
TL;DR: In this paper, a different approach to problems of multiple significance testing is presented, which calls for controlling the expected proportion of falsely rejected hypotheses -the false discovery rate, which is equivalent to the FWER when all hypotheses are true but is smaller otherwise.
Journal ArticleDOI
Random Forests
TL;DR: Internal estimates monitor error, strength, and correlation and these are used to show the response to increasing the number of features used in the forest, and are also applicable to regression.
Journal ArticleDOI
The WEKA data mining software: an update
TL;DR: This paper provides an introduction to the WEKA workbench, reviews the history of the project, and, in light of the recent 3.6 stable release, briefly discusses what has been added since the last stable version (Weka 3.4) released in 2003.
Journal ArticleDOI
Bagging predictors
TL;DR: Tests on real and simulated data sets using classification and regression trees and subset selection in linear regression show that bagging can give substantial gains in accuracy.