Publisher Summary This chapter discusses the role that the relationship between the number of measurements and the number of training patterns plays at various stages in the design of a pattern recognition system. The designer of a pattern recognition system should make every possible effort to obtain as many samples as possible. As the number of samples increases, not only does the designer have more confidence in the performance of the classifier, but also more measurements can be incorporated in the design of the classifier without the fear of peaking in its performance. However, there are many pattern classification problems where either the number of samples is limited or obtaining a large number of samples is extremely expensive. If the designer chooses to take the optimal Bayesian approach, the average performance of the classifier improves monotonically as the number of measurements is increased. Most practical pattern recognition systems employ a non-Bayesian decision rule because the use of optimal Bayesian approach requires knowledge of prior densities, and besides, their complexity precludes the development of real-time recognition systems. The peaking behavior of practical classifiers is caused principally by their nonoptimal use of measurements.

39 Dimensionality and sample size considerations in pattern recognition practice

Articles, books, and technical reports on the theoretical and experimental estimation of probability of misclassification are listed for the case of correctly labeled or preclassified training data. By way of introduction, the problem of estimating the probability of misclassification is discussed in order to characterize the contributions of the literature.

Bibliography on estimation of misclassification

Abstract We consider the classification problem in the context of two equiprobable multivariate Gaussian densities with a common covariance matrix. The use of Anderson's W statistic for classification results in the existence of an optimum number of features, p opt such that, for a given sample size, the average probability of misclassification decreases at first as the number of features are increased, attains a minimum at p opt and then starts increasing. We have examined this peaking phenomenon for several cases and provide expressions which relate p opt to the number of available training samples and the Mahalanobis distance between the two populations. We also show that to prevent peaking, each additional feature's contribution to the Mahalanobis distance must be a certain proportion of the accumulated Mahalanobis distance.

On The Optimal Number of Features in The Classification Multivariate Gaussian Data

On the optimal number of features in the classification of multivariate Gaussian data

The probabilities of misclassification involved in the use of estimated discriminant functions are subject to chance variations. The author's purpose in this paper is to derive the distribution laws that the probabilities of misclassification follow and to obtain their expected values. The parent populations are assumed to be normal. The first part of the paper considers the univariate case and the second part the multivariate case. The discussion of the multivariate case proceeds in three stages of increasing complexity. When the exact results are complicated, asymptotic results or approximations are given. Finally, the problem of estimating the expected probabilities of misclassification is considered. Interval estimates as well as point estimates are given.

/pdf/errors-in-discrimination-4je0pw7cyl.pdf

Errors in Discrimination

In this paper we shall consider the exact distribution of Wald's classification statistic $V$ in the univariate case, some theoretical approximations in various multivariate cases, and an empirical distribution in a particular multivariate case. We shall also draw some conclusions as to the potential usefulness of the statistic $V$ and the work which remains to be done.

/pdf/on-the-distribution-of-wald-s-classification-statistic-3ydcvobox8.pdf

Harman Leon Harter

Papers

On the Distribution of Wald's Classification Statistic