Quantifying and assessing changes in biological diversity are central aspects of many ecological studies, yet accurate methods of estimating biological diversity from sampling data have been elusive. Hill numbers, or the effective number of species, are increasingly used to characterize the taxonomic, phylogenetic, or functional diversity of an assemblage. However, empirical estimates of Hill numbers, including species richness, tend to be an increasing function of sampling effort and, thus, tend to increase with sample completeness. Integrated curves based on sampling theory that smoothly link rarefaction (interpolation) and prediction (extrapolation) standardize samples on the basis of sample size or sample completeness and facilitate the comparison of biodiversity data. Here we extended previous rarefaction and extrapolation models for species richness (Hill number q D, where q ¼ 0) to measures of taxon diversity incorporating relative abundance (i.e., for any Hill number q D, q . 0) and present a unified approach for both individual-based (abundance) data and sample- based (incidence) data. Using this unified sampling framework, we derive both theoretical formulas and analytic estimators for seamless rarefaction and extrapolation based on Hill numbers. Detailed examples are provided for the first three Hill numbers: q ¼ 0 (species richness), q ¼ 1 (the exponential of Shannon's entropy index), and q ¼ 2 (the inverse of Simpson's concentration index). We developed a bootstrap method for constructing confidence intervals around Hill numbers, facilitating the comparison of multiple assemblages of both rarefied and extrapolated samples. The proposed estimators are accurate for both rarefaction and short-range extrapolation. For long-range extrapolation, the performance of the estimators depends on both the value of q and on the extrapolation range. We tested our methods on simulated data generated from species abundance models and on data from large species inventories. We also illustrate the formulas and estimators using empirical data sets from biodiversity surveys of temperate forest spiders and tropical ants.

/pdf/rarefaction-and-extrapolation-with-hill-numbers-a-framework-1m0gvt1mro.pdf

Rarefaction and extrapolation with Hill numbers: a framework for sampling and estimation in species diversity studies

We propose an integrated sampling, rarefaction, and extrapolation methodology to compare species richness of a set of communities based on samples of equal completeness (as measured by sample coverage) instead of equal size. Traditional rarefaction or extrapolation to equal-sized samples can misrepresent the relationships between the richnesses of the communities being compared because a sample of a given size may be sufficient to fully characterize the lower diversity community, but insufficient to characterize the richer community. Thus, the traditional method systematically biases the degree of differences between community richnesses. We derived a new analytic method for seamless coverage-based rarefaction and extrapolation. We show that this method yields less biased comparisons of richness between communities, and manages this with less total sampling effort. When this approach is integrated with an adaptive coverage-based stopping rule during sampling, samples may be compared directly without rarefaction, so no extra data is taken and none is thrown away. Even if this stopping rule is not used during data collection, coverage-based rarefaction throws away less data than traditional size-based rarefaction, and more efficiently finds the correct ranking of communities according to their true richnesses. Several hypothetical and real examples demonstrate these advantages.

https://esajournals.onlinelibrary.wiley.com/doi/epdf/10.1890/11-1952.1

Coverage‐based rarefaction and extrapolation: standardizing samples by completeness rather than size

SUMMARY A theoretical optimal stopping rule based on the minimization of the testing cost and expected penalty due to the unremoved bugs is derived without the equal failure rate assumption under Nayak's (1988) recapture debugging procedure. Two adaptive stopping rules suitable for practical use are constructed and tested by simulation. The estimation of the number of undetected bugs is also considered via remaining failure rate estimation.

Stopping rules and estimation for recapture debugging with unequal failure rates

Biodiversity sampling is labor intensive, and a substantial fraction of a biota is often represented by species of very low abundance, which typically remain undetected by biodiversity surveys. Statistical methods are widely used to estimate the asymptotic number of species present, including species not yet detected. Additional sampling is required to detect and identify these species, but richness estimators do not indicate how much sampling effort (additional individuals or samples) would be necessary to reach the asymptote of the species accumulation curve. Here we develop the first statistically rigorous nonparametric method for estimating the minimum number of additional individuals, samples, or sampling area required to detect any arbitrary proportion (including 100%) of the estimated asymptotic species richness. The method uses the Chao1 and Chao2 nonparametric estimators of asymptotic richness, which are based on the frequencies of rare species in the original sampling data. To evaluate the performance of the proposed method, we randomly subsampled individuals or quadrats from two large biodiversity inventories (light trap captures of Lepidoptera in Great Britain and censuses of woody plants on Barro Colorado Island [BCI], Panama). The simulation results suggest that the method performs well but is slightly conservative for small sample sizes. Analyses of the BCI results suggest that the method is robust to nonindependence arising from small-scale spatial aggregation of species occurrences. When the method was applied to seven published biodiversity data sets, the additional sampling effort necessary to capture all the estimated species ranged from 1.05 to 10.67 times the original sample (median approximately equal to 2.23). Substantially less effort is needed to detect 90% of the species (0.33-1.10 times the original effort; median approximately equal to 0.80). An Excel spreadsheet tool is provided for calculating necessary sampling effort for either abundance data or replicated incidence data.

https://www.jstor.org/stable/info/25592599

Sufficient sampling for asymptotic minimum species richness estimators

Contrary to common belief, decomposition of diversity into independent richness and evenness components is mathematically impossible. However, richness can be decomposed into independent diversity and evenness or inequality components. The evenness or inequality component derived in this way is connected to most of the common measures of evenness and inequality in ecology and economics. This perspective justifies the derivation of measures of relative evenness, which give the amount of evenness relative to the maximum and minimum possible for a given richness. Pielou’s [1] evenness measure J is shown to be such a measure.

The Relation between Evenness and Diversity

(5) V (a) = Vn O(a) = minn>O Vn(a), denote the minimum risk. On the other hand, in ignorance of ano procedure based on a fixed number n of observations of (1) will minimize (4) simultaneously for all 0 0 and N, (6) LN = AIXN-MA18 + Nt (A > 0), with risk

/pdf/on-the-asymptotic-efficiency-of-a-sequential-procedure-for-3kn6ql89kt.pdf

On the Asymptotic Efficiency of a Sequential Procedure for Estimating the Mean

/pdf/the-performance-of-a-sequential-procedure-for-the-fixed-2o3prrh92l.pdf

The Performance of a Sequential Procedure for the Fixed-Width Interval Estimation of the Mean

A sequential procedure for estimating the mean of an exponential distribution is proposed. It is shown to perform well for large values of the mean, and the results of a Monte Carlo study indicate that it also performs well for moderate values of the mean.

/pdf/further-remarks-on-sequential-estimation-the-exponential-3eh8hc4j5h.pdf

Further Remarks on Sequential Estimation: The Exponential Case

We consider utilizing a sequential experiment for estimating the mean of a normal distribution when the variance of the distribution is unknown. For a suitable choice of the design constants of the experiment and for loss structures of practical interest, it is shown that the difference in the expected loss with optional stopping and the expected loss which would be incurred if σ were known is a bounded function of σ.

Remarks on sequential point estimation

A population is searched by selecting with replacement one of its members at a time and noting the species that it represents. The search may continue indefinitely, but there is a cost of c units for each selection. If the search is terminated with n draws and j species have been discovered, we receive a payoff h(j) - cn, where the reward function h is nondecreasing. For a given class of such reward functions the optimal stopping time is obtained. An adaptive procedure is proposed in the case that the proportions of different species in the population, as well as their number, are unknown; simulations suggest that this procedure performs exceedingly well irrespective of the composition p of the population. Finally, statistical applications are developed for the problem of inference based on randomly occurring data.

https://www.tandfonline.com/doi/pdf/10.1080/01621459.1979.10481667

Norman Starr

Papers

On the Asymptotic Efficiency of a Sequential Procedure for Estimating the Mean

The Performance of a Sequential Procedure for the Fixed-Width Interval Estimation of the Mean

Further Remarks on Sequential Estimation: The Exponential Case

Remarks on sequential point estimation

Optimal and Adaptive Stopping in the Search for New Species