Showing papers by "Jerome H. Friedman published in 1997"
TL;DR: In this article, it was shown that the bias and variance components of the estimation error combine to influence classification in a very different way than with squared error on the probabilities themselves, and that certain types of (very high) bias can be canceled by low variance to produce accurate classification.
Abstract: The classification problem is considered in which an output variable y assumes discrete values with respective probabilities that depend upon the simultaneous values of a set of input variables x = {x_1,....,x_n}. At issue is how error in the estimates of these probabilities affects classification error when the estimates are used in a classification rule. These effects are seen to be somewhat counter intuitive in both their strength and nature. In particular the bias and variance components of the estimation error combine to influence classification in a very different way than with squared error on the probabilities themselves. Certain types of (very high) bias can be canceled by low variance to produce accurate classification. This can dramatically mitigate the effect of the bias associated with some simple estimators like “naive” Bayes, and the bias induced by the curse-of-dimensionality on nearest-neighbor procedures. This helps explain why such simple methods are often competitive with and sometimes superior to more sophisticated ones for classification, and why “bagging/aggregating” classifiers can often improve accuracy. These results also suggest simple modifications to these procedures that can (sometimes dramatically) further improve their classification performance.
1,066 citations
TL;DR: In this paper, the curds and whey method is proposed to take advantage of correlations between the response variables to improve predictive accuracy compared with the usual procedure of doing individual regressions of each response variable on the common set of predictor variables.
Abstract: We look at the problem of predicting several response variables from the same set of explanatory variables. The question is how to take advantage of correlations between the response variables to improve predictive accuracy compared with the usual procedure of doing individual regressions of each response variable on the common set of predictor variables. A new procedure is introduced called the curds and whey method. Its use can substantially reduce prediction errors when there are correlations between responses while maintaining accuracy even if the responses are uncorrelated. In extensive simulations, the new procedure is compared with several previously proposed methods for predicting multiple responses (including partial least squares) and exhibits superior accuracy. One version can be easily implemented in the context of standard statistical packages.
524 citations