Stability and bias of classification rates in biological applications of discriminant analysis

TLDR: The sampling stability of classification rates in discriminant analysis by using a factorial design with factors for multivariate dimensionality, dispersion structure, configuration of group means, and sample size was assessed by.

Abstract: We assessed the sampling stability of classification rates in discriminant analysis by using a factorial design with factors for multivariate dimensionality, dispersion structure, configuration of group means, and sample size. A total of 32,400 discriminant analyses were conducted, based on data from simulated populations with appropriate underlying statistical distributions. Simulation results indicated strong bias in correct classification rates when group sample sizes were small and when overlap among groups was high. We also found that stability of the correct classification rates was influenced by these factors, indicating that the number of samples required for a given level of precision increases with the amount of overlap among groups. In a review of 60 published studies, we found that 57% of the articles presented results on classification rates, though few of them mentioned potential biases in their results. Wildlife researchers should choose the total number of samples per group to be at least 2 times the number of variables to be measured when overlap among groups is low. Substantially more samples are required as the overlap among groups increases. J. WILDL. MANAGE. 54(2):331-341 Researchers often analyze biological data with discriminant analysis, a statistical method useful for classifying individuals into groups and for highlighting group differences. Necessary features for applications are the separability of samples into distinct groups and the measurement of 1 or more attributes on each sample unit. Typical applications by wildlife biologists involve the identification of groups of distinct animal or plant species and the examination of group differences based on habitat attributes (Edge et al. 1987). Other applications include morphometric analysis of species differences (Troy 1985) and analysis of plant or animal assemblages based on species abundances (Matthews 1979). Williams (1981, 1983) characterized ecological applications of the methodology by their grouping indices and attributes, and Williams and Titus (1988) described more recent applications in terms of sample sizes, system dimensions, and other statistical attributes. In many biological applications of discriminant analysis, the classification functions are characterized by substantial variability associated with small sample sizes and high multivariate dimensionality. There have been several theoretical assessments of the statistical properties of classification rates, especially asymptotic properties under various distributional assumptions (Anderson 1973, Gordon and Olshen 1978). However, the few studies that focused on small sample properties were limited in scope (Van Ness 1979, Page 1985) and generally focused on biases in apparent classification rates (Moran 1975, McLachlan 1976). Because wildlife studies often are hindered by an inability to collect a large number of independent samp es, the sources of variability in classification rates based on small sample sizes need to be examined. We report classification success as part of a simulation study of discriminant analysis (see Williams and Titus [1988] for canonical variates analysis). Our objective was to identify sources of variability in classification rates and to determine minimum sample sizes necessary for es imating these rates. Our study focused on assessment of discriminant analysis in representing system structure known to underlie the data under investigation. Our results compliment the work of Rexstad et al. (1988), who used data with no apparent underlying system structure to document a tendency, under certain conditions, for discriminant analysis to generate spurious relationships. We thank G. W. Pendleton, E. F. Burton, and K. Boone for technical support. D. E. Capen, D. W. Sparling, and J. S. Hatfield provided preliminary reviews and helpful comments. 1 Present address: Office of Migratory Bird Management, U.S. Fish and Wildlife Service, 18th and C Street NW, Washington, DC 20240. 2 Present address: Alaska Department of Fish and Game, Division of Wildlife Conservation, P.O. Box 20, Douglas, AK 99824.