Predictive performance of presence-only species distribution models: a benchmark study with reproducible code

O_LIThe Random Forest (RF) algorithm is an ensemble of classification or regression trees, and is a widely used and high-performing machine learning technique. It is increasingly used for species distribution modelling (SDM). Many researchers use implementations of RF in the R programming language with default parameters to analyse species presence-only data together with background samples. However, there is good evidence that RF with default parameters does not perform well with such species "presence-background" data. This is often attributed to the typical disparity between the number of presence and background samples also known as class imbalance, and several solutions have been proposed. C_LIO_LIHere, we first set the context: the background sample should be large enough to represent all environments in the region. We then aim to understand the drivers of poor performance of RF with presence-background data, and explain, test and evaluate suggested solutions. Using simulated and real species data, we compare performance of default RF with other weighting and sampling approaches. C_LIO_LIWe show that class overlap is an important driver of poor performance, alongside class imbalance. The results demonstrate clear evidence of improvement in the performance of RFs when class imbalance is explicitly managed by sampling methods or when the overfitting commonly associated with overlapping classes is avoided by forcing shallow trees. C_LIO_LIPresence-background data is a particular version of class imbalance in which class overlap is highly likely and extreme imbalance exists. Without compromising the environmental representativeness of the sampled background, we show several approaches to fitting RF that ameliorate the effects of imbalance and overlap, and allow excellent predictive performance. Understanding the problems of RF in presence-background data allows new insights into how best to fit models, and should guide future efforts to best deal with such data. C_LI

Modelling species presence-only data with random forests

Conservation science depends on an accurate understanding of what’s happening in a given ecosystem. How many species live there? What is the makeup of the population? How is that changing over time? Species Distribution Modeling (SDM) seeks to predict the spatial (and sometimes temporal) patterns of species occurrence, i.e. where a species is likely to be found. The last few years have seen a surge of interest in applying powerful machine learning tools to challenging problems in ecology [2, 5, 8]. Despite its considerable importance, SDM has received relatively little attention from the computer science community. Our goal in this work is to provide computer scientists with the necessary background to read the SDM literature and develop ecologically useful ML-based SDM algorithms. In particular, we introduce key SDM concepts and terminology, review standard models, discuss data availability, and highlight technical challenges and pitfalls.

https://dl.acm.org/doi/pdf/10.1145/3460112.3471966

Species Distribution Modeling for Machine Learning Practitioners: A Review

Predicted range shifts of invasive giant hogweed (Heracleum mantegazzianum) in Europe.

https://biorxiv.org/content/10.1101/2020.11.16.384164v1.full.pdf

Species distribution models (SDMs) are widely used to predict and study distributions of species. Many different modeling methods and associated algorithms are used and continue to emerge. It is important to understand how different approaches perform, particularly when applied to species occurrence records that were not gathered in structured surveys (e.g. opportunistic records). This need motivated a large-scale, collaborative effort, published in 2006, that aimed to create objective comparisons of algorithm performance. As a benchmark, and to facilitate future comparisons of approaches, here we publish that dataset: point location records for 226 anonymised species from six regions of the world, with accompanying predictor variables in raster (grid) and point formats. A particularly interesting characteristic of this dataset is that independent presence-absence survey data are available for evaluation alongside the presence-only species occurrence data intended for modeling. The dataset is available on Open Science Framework and as an R package and can be used as a benchmark for modeling approaches and for testing new ways to evaluate the accuracy of SDMs.

/pdf/presence-only-and-presence-absence-data-for-comparing-2jic0qs0ci.pdf

Meinrad Abegg

Papers

Presence-only and presence-absence data for comparing species distribution modeling methods