Experiments with a new boosting algorithm
Citations
[...]
79,257 citations
Cites background or methods from "Experiments with a new boosting alg..."
...But none of these these three forests do as well as Adaboost (Freund & Schapire, 1996) or other algorithms that work by adaptive reweighting (arcing) of the training set (see Breiman, 1998b; Dieterrich, 1998; Bauer & Kohavi, 1999)....
[...]
...In its original version, Adaboost (Freund & Schapire, 1996) is a deterministic algorithm that selects the weights on the training set for input to the next classifier based on the misclassifications in the previous classifiers....
[...]
20,196 citations
17,764 citations
Cites background or methods from "Experiments with a new boosting alg..."
...Suppose that for a particular loss (y; F ) and/or base learner h(x; a) the solution to (9) is di cult to obtain....
[...]
...In machine learning, (9) (10) is called \boosting" where y 2 f 1; 1g and (y; F ) is either an exponential loss criterion e yF (Freund and Schapire 1996, Schapire and Singer 1998) or negative binomial log{likelihood (Friedman, Hastie, and...
[...]
...and the approximation updated Fm(x) = Fm 1(x) + mh(x; am): Basically, instead of obtaining the solution under a smoothness constraint (9), the constraint is applied to the unconstrained (rough) solution f gm(xi)g N i=1 (7)....
[...]
...Given the current approximation Fm 1(x) at the mth iteration, the function mh(x; am) (9) (10) is the best greedy step towards the minimizing solution F (x) (1), under the constraint that the step \direction" h(x; am) be a member of the parameterized class of functions h(x; a)....
[...]
...In the special case where y 2 f 1; 1g and the loss function (y; F ) depends on y and F only through their product (y; F ) = (yF ), the analogy of boosting (9) (10) to steepest{descent minimization has been noted in the machine learning literature (Breiman 1997a, Ratsch, Onoda, and Muller 1998)....
[...]
15,813 citations
Cites background or methods from "Experiments with a new boosting alg..."
...Since it was first introduced, several successful experiments have been conducted using AdaBoost, including work by the authors [12], Drucker and Cortes [8], Jackson and Craven [16], Quinlan [21], and Breiman [3]....
[...]
...Empirical tests [12] have shown that pseudo-loss is generally more successful when the weak learners use very restricted hypotheses....
[...]
10,217 citations
Cites methods from "Experiments with a new boosting alg..."
...For this problem, as usually done in the literature [20,21,5,25] 75% and 25% samples are randomly chosen for training and testing at each trial, respectively....
[...]
...57% with 20 nodes, which is obviously higher than all the results so far reported in the literature using various popular algorithms such as SVM [20], SAOCIF [21], Cascade-Correlation algorithm [21], bagging and boosting methods [5], C4....
[...]
References
21,674 citations
[...]
16,118 citations
15,813 citations
8,046 citations
4,081 citations
"Experiments with a new boosting alg..." refers methods in this paper
...These include: (1) an algorithm that searches for very simple prediction rules which test on a single attribute (similar to Holte’s very simple classification rules [14]); (2) an algorithm that searches for a single good decision rule that tests on a conjunction of attribute tests (similar in flavor to the rule-formation part of Cohen’s RIPPER algorithm [3] and Fürnkranz and Widmer’s IREP algorithm [11]); and (3) Quinlan’s C4....
[...]