Probability Distributions.- Linear Models for Regression.- Linear Models for Classification.- Neural Networks.- Kernel Methods.- Sparse Kernel Machines.- Graphical Models.- Mixture Models and EM.- Approximate Inference.- Sampling Methods.- Continuous Latent Variables.- Sequential Data.- Combining Models.

Pattern Recognition and Machine Learning

https://hal.archives-ouvertes.fr/hal-00755489/file/PRLv4.pdf

Variable selection using random forests

The random forest algorithm, proposed by L. Breiman in 2001, has been extremely successful as a general-purpose classification and regression method. The approach, which combines several randomized decision trees and aggregates their predictions by averaging, has shown excellent performance in settings where the number of variables is much larger than the number of observations. Moreover, it is versatile enough to be applied to large-scale problems, is easily adapted to various ad hoc learning tasks, and returns measures of variable importance. The present article reviews the most recent theoretical and methodological developments for random forests. Emphasis is placed on the mathematical forces driving the algorithm, with special attention given to the selection of parameters, the resampling mechanism, and variable importance measures. This review is intended to provide non-experts easy access to the main ideas.

/pdf/a-random-forest-guided-tour-3vnpr6ncpy.pdf

A random forest guided tour

general and multidisciplinary evaluations of co-management regimes and the conditions for social, economic and ecological success within such regimes are lacking. Here we examine 130 comanaged fisheries in a wide range of countries with different degrees of development, ecosystems, fishing sectors and type of resources. We identified strong leadership as the most important attribute contributing to success, followed by individual or community quotas, social cohesion and protected areas. Less important conditions included enforcement mechanisms, long-term management policies and life history of the resources. Fisheries were most successful when at least eight co-management attributes were present, showing a strong positive relationship between the number of these attributes and success, owing to redundancy in management regulations. Our results demonstrate the critical importance of

/pdf/leadership-social-capital-and-incentives-promote-successful-2so9f72m7y.pdf

Leadership, social capital and incentives promote successful fisheries

The random forest algorithm, proposed by L. Breiman in 2001, has been extremely successful as a general-purpose classification and regression method. The approach, which combines several randomized decision trees and aggregates their predictions by averaging, has shown excellent performance in settings where the number of variables is much larger than the number of observations. Moreover, it is versatile enough to be applied to large-scale problems, is easily adapted to various ad-hoc learning tasks, and returns measures of variable importance. The present article reviews the most recent theoretical and methodological developments for random forests. Emphasis is placed on the mathematical forces driving the algorithm, with special attention given to the selection of parameters, the resampling mechanism, and variable importance measures. This review is intended to provide non-experts easy access to the main ideas.

A Random Forest Guided Tour

This paper describes the R package VSURF. Based on random forests, and for both regression and classification problems, it returns two subsets of variables. The first is a subset of important variables including some redundancy which can be relevant for interpretation, and the second one is a smaller subset corresponding to a model trying to avoid redundancy focusing more closely on prediction objective. The two-stage strategy is based on a preliminary ranking of the explanatory variables using the random forests permutation-based score of importance and proceeds using a stepwise forward strategy for variable introduction. The two proposals can be obtained automatically using data-driven default values, good enough to provide interesting results, but can also be tuned by the user. The algorithm is illustrated on a simulated example and its applications to real datasets are presented.

/pdf/vsurf-an-r-package-for-variable-selection-using-random-5czzpy86ya.pdf

VSURF: An R Package for Variable Selection Using Random Forests

https://hal.archives-ouvertes.fr/hal-01233923/document

Random Forests for Big Data

This paper examines from an experimental perspective random forests, the increasingly used statistical method for classification and regression problems introduced by Leo Breiman in 2001. It first aims at confirming, known but sparse, advice for using random forests and at proposing some complementary remarks for both standard problems as well as high dimensional ones for which the number of variables hugely exceeds the sample size. But the main contribution of this paper is twofold: to provide some insights about the behavior of the variable importance index based on random forests and in addition, to propose to investigate two classical issues of variable selection. The first one is to find important variables for interpretation and the second one is more restrictive and try to design a good prediction model. The strategy involves a ranking of explanatory variables using the random forests score of importance and a stepwise ascending variable introduction strategy.

/pdf/random-forests-some-methodological-insights-cdxwmx41n7.pdf

Random Forests: some methodological insights

Random forests (RFs), introduced by Leo Breiman in 2001, are a very effective statistical method. The complex mechanism of the method makes theoretical analysis difficult. Therefore, simplified versions of RF, called purely RFs (PRF), which can be theoretically handled more easily, have been considered. In this paper, we study the variance of such forests. First, we show a general upper bound which emphasises the fact that a forest reduces the variance. We then introduce a simple variant of PRFs, that we call purely uniformly RFs. For this variant and in the context of regression problems with a one-dimensional predictor space, we show that both random trees and RFs reach minimax rate of convergence. In addition, we prove that compared with random trees, RFs improve accuracy by reducing the estimator variance by a factor of three-fourths.

/pdf/variance-reduction-in-purely-random-forests-1vo06ahxxl.pdf

Robin Genuer

Papers

Variable selection using random forests

VSURF: An R Package for Variable Selection Using Random Forests

Random Forests for Big Data

Random Forests: some methodological insights

Variance reduction in purely random forests