Showing papers on "Linear discriminant analysis published in 1987"
TL;DR: Discriminant analysis is a technique for the multivariate study of group differences as discussed by the authors, which provides a method of examining the extent to which multiple predictor variables are related to a categorical criterion, that is, group membership.
Abstract: Discriminant analysis is a technique for the multivariate study of group differences. More specifically, it provides a method of examining the extent to which multiple predictor variables are related to a categorical criterion, that is, group membership. Situations in which the technique is particularly useful include those in which the researcher wishes to assess which of a number of continuous variables best differentiates groups of individuals or in which he or she wishes to predict group membership on the basis of the discriminant function (analogous to a multiple regression equation) yielded by the analysis. The method is also useful as a follow-up to a significant analysis of variance. In this article, I describe the method of discriminant analysis, including the concept of discriminant function, discriminant score, group centroid, and discriminant weights and Ioadings. I discuss methods for testing the statistical significance of a function, methods of using the function in classification, and the concept of rotating functions. The use of discriminant analysis in both the two-group case and the multigroup case is illustrated. Finally, I provide a number of illustrative examples of use of the method in the counseling literature. I conclude with cautions regarding the use of the method and with the provision of resources for further study. The technique ofdiscriminant analysis, developed by R. A. Fisher (1936), is one method for the multivariate study of group differences. When used for explanatory purposes, discriminant analysis is particularly appropriate when one wishes (a) to describe, summarize, and understand the differences between or among groups, (b) to determine which of a set of continuous variables best captures or characterizes group differences, (c) to describe the dimensionality of group differences (much like factor analysis describes the dimensionality of a set of continuous variables), (d) to test theories that use stage concepts or taxonomies, and (e) to examine the nature of group differences following a multivariate analysis of variance (MANOVA; Borgen & Seling, 1978). Probably the most frequent applications of discriminant analysis are for predictive purposes, that is, for situations in which it is necessary or desirable to classify subjects into groups or categories. The results of a discriminant analysis allow the prediction of group membership based on the best linear composite or combination of predictor scores. Discriminant analysis is analogous to multiple regression in that both involve prediction from a set of continuous predictor variables (sometimes designated independent variables) to a criterion. The major difference between them is that multiple regression predicts to a continuous criterion variable (sometimes designated the dependent variable), whereas discriminant analysis predicts to a categorical criterion, that is, group membership. Thus, given multiple predictor variables, multiple regression would be the appropriate method of analysis if the dependent variable were continuous, and discriminant analysis would be appropriate if the dependent variable were categorical, with two or more levels.
136 citations
TL;DR: In this paper, a digital image processing system is described to facilitate objective inspection and classification of cereal grains using a charge-coupled device (CCD) video camera interfaced to a custom-built data-acquisition system.
96 citations
TL;DR: In this article, the power of parametric procedures is low, and their results may be in error when applied to non-normal data, and five important advantages of nonparametric methods over commonly used parametric procedure are illustrated.
Abstract: Water quality data are usually analysed with parametric statistical procedures requiring the normality assumption for accuracy of their attained significance levels. However, these data are typically non-normally distributed. When applied to non-normal data, the power of parametric procedures is low, and their results may be in error. Three typical case studies are discussed: differentiation of water quality in streams using analysis of variance; discernment of water quality types using discriminant analysis; and t-tests on differences between two groups which include data below the detection limit. Five important advantages of nonparametric methods over commonly used parametric procedures are illustrated.
89 citations
TL;DR: A simple classification tree is presented and contrasted with a linear discriminant function, which is robust with respect to outlier cases and can use nominal, ordinal, interval, and ratio scaled predictor variables.
Abstract: Classification trees are discriminant models structured as dichtomous keys. A simple classification tree is presented and contrasted with a linear discriminant function. Classification trees have several advantages when compared with linear discriminant analysis. The method is robust with respect to outlier cases. It is nonparametric and can use nominal, ordinal, interval, and ratio scaled predictor variables. Cross-validation is used during tree development to prevent overrating the tree with too many predictor variables. Missing values are handled by using surrogate splits based on nonmissing predictor variables. Classification trees, like linear discriminant analysis, have potential prediction bias and therefore should be validated before being accepted.
78 citations
TL;DR: Logistic ECG and VCG models improve the total accuracy of classification by about 1 to 3% when compared to LDA and reliability of classification represents the improvement, which may enhance the diagnostic utility of theECG andVCG in routine practice.
78 citations
TL;DR: Two expert systems of the rule-building type, TIMM and EX-TRAN, are compared with pattern recognition methods for the classification of olive oils of different origins and TIMM yields slightly better results than nearest neighbors classifiers and linear discriminant analysis.
Abstract: Two expert systems of the rule-building type, TIMM and EX-TRAN, are compared with pattern recognition methods for the classification of olive oils of different origins. Both expert systems are more user-friendly than the pattern recognition programs and TIMM yields slightly better results than nearest neighbors classifiers and linear discriminant analysis.
69 citations
TL;DR: The teaching of logit regression analysis is much neglected in statistics courses within sociology as mentioned in this paper, which is unfortunate since it is well-suited to so many data analysis situations within the discipline.
Abstract: The teaching of logit regression analysis is much neglected in statistics courses within sociology. This is unfortunate since it is well-suited to so many data analysis situations within the discipline. We often run into cases in the real world when the dependent variable is dichotomous. Researchers often deal with such situations by using discriminant analysis, weighted least squares regression, or ordinary least squares regression. These methods can lead to misinterpretations of the results. Logit regression allows the researcher to evaluate the impact of a set of predictor variables on a dichotomous dependent variable without these problems. It is a relatively simple technique to understand for those who already have a grasp on the logic of OLS regression. This paper presents the technique in simple form using both SPSSX and SAS computer output. We contend that the results obtained from logit can be presented to the lay person in a way that is more intuitively understandable than is any other method of presenting data.
55 citations
TL;DR: In this article, the authors compare the performance of normal discriminant analysis and multinomial logistic regression for categorical response data with continuous or categorical explanatory variables, and evaluate the bias and efficiency in parameter estimation.
Abstract: Multinomial logistic regression (also referred to as polychotomous logistic regression) is frequently used for the analysis of categorical response data with continuous or categorical explanatory variables. Parameter estimates are usually obtained through direct maximum likelihood estimation. Normal discriminant analysis has been used as an alternative approach to this methodology, although it is strictly appropriate only when the usual normal discriminant assumptions concerning the explanatory variables are valid. Comparative evaluations of the two procedures, including the investigation of bias and efficiency in parameter estimation, have been almost entirely limited to the case of two response groups for both normal and nonnormal explanatory variables. Bull and Donner (1987) extended the comparison of logistic regression and normal discrimination in the normal case to more than two response groups, deriving the large sample distribution of the slope parameter estimates for each of the two proc...
54 citations
TL;DR: In this article, a new method of multiple discriminant analysis was developed that allows a mixture of continuous and discrete predictors, which can be justified under a wide class of distributional assumptions on the predictor variables.
Abstract: A new method of multiple discriminant analysis was developed that allows a mixture of continuous and discrete predictors. The method can be justified under a wide class of distributional assumptions on the predictor variables. The method can also handle three different sampling situations, conditional, joint and separate. In this method both subjects (cases or any other sampling units) and criterion groups are represented as points in a multidimensional euclidean space. The probability of a particular subject belonging to a particular criterion group is stated as a decreasing function of the distance between the corresponding points. A maximum likelihood estimation procedure was developed and implemented in the form of a FORTRAN program. Detailed analyses of two real data sets were reported to demonstrate various advantages of the proposed method. These advantages mostly derive from model evaluation capabilities based on the Akaike Information Criterion (AIC).
50 citations
TL;DR: In this article, an experimental comparison between a linear programming approach and the well known statistical procedure by Fisher for solving discriminant analysis problems is presented. But the results indicate that both methods are enhanced by the inclusion of the qualitative variables, but that the Fisher approach seems preferable.
44 citations
Book•
01 Jan 1987
TL;DR: Pillai et al. as discussed by the authors proposed a hierarchical hierarchy of relationships between covariance matrices and showed the effect of additional variables in Principal Component Analysis, Discriminant Analysis and Canonical Correlation Analysis.
Abstract: Minimaxity of Empirical Bayes Estimators Derived from Subjective Hyperpriors.- Quasi-Inner Products and Their Applications.- A Hierarchy of Relationships Between Covariance Matrices.- Effect of Additional Variables in Principal Component Analysis, Discriminant Analysis and Canonical Correlation Analysis.- On a Locally Best Invariant and Locally Minimax Test in Symmetrical Multivariate Distributions.- Confidence Intervals for the Slope in a Linear Errors-in-Variables Regression Model.- Likelihood Ratio Test for Multisample Sphericity.- Statistical Selection Procedures in Multivariate Models.- Quadratic Forms to have a Specified Distribution.- Asymptotic Expansions for Errors of Misclassification:Nonnormal Situations.- Transformations of Statistics in Multivariate Analysis.- Error Rate Estimation in Discriminant Analysis: Recent Advances.- Some Simple Optimal Tests in Multivariate Analysis.- Developments in Eigenvalue Estimation.- Asymptotic Non-null Distributions of a Statistic for Testing the Equality of Hermitian Covariance Matrices in the Complex Gaussian Case.- A Model for Interlaboratory Differences.- Bayes Estimators in Lognormal Regression Model.- Multivariate Behrens-Fisher Problem by Heteroscedastic Method.- Tests for Covariance Structure in Familial Data and Principal Component.- Risk of Improved Estimators for Generalized Variance and Precision.- Sampling Distributions of Dependent Quadratic Forms from Normal and Nonnormal Universes.- Bibliography of Works by K. C. S. Pillai.
01 Apr 1987
TL;DR: A Gaussian probabilistic model was developed to screen and select from the large set of features and the significant harmonics of the signature were sorted according to the chi-square value, which is equivalent to the signal-to-noise ratio.
Abstract: Features such as shape, motion and pressure, minutiae details and timing, and transformation methods such as Hadamard and Walsh have been used in signature recognition with various degrees of success. One of the better studies was done by Sato and Kogure using nonlinear warping function. However, it is time consuming in terms of computer time and programming time. In this research, the signatures were normalized for size, orientation, etc. After normalization, the X and Y coordinates of each sampled point of a signature over time (to capture the dynamics of signature writing) were represented as a complex number and the set of complex numbers transformed into the frequency domain via the fast Fourier transform. A Gaussian probabilistic model was developed to screen and select from the large set of features (e.g. amplitude of each harmonics). The significant harmonics of the signature were sorted according to the chi-square value, which is equivalent to the signal-to-noise ratio. Fifteen harmonics with the largest signal-to-noise ratios from the true signatures were used in a discriminant analysis. A total of eight true signatures from a single person and eight each from nineteen forgers were used. It results in an error rate of 2.5%, with the normally more conservative jacknife procedure yielding the same small error rate.
TL;DR: In this paper, the authors compare the sample Euclidean distance classifier (EDC) with the sample linear discriminant function (LDF) when the number of features is large relative to the size of the training samples.
Abstract: The sample linear discriminant function (LDF) is known to perform poorly when the number of features p is large relative to the size of the training samples, A simple and rarely applied alternative to the sample LDF is the sample Euclidean distance classifier (EDC). Raudys and Pikelis (1980) have compared the sample LDF with three other discriminant functions, including thesample EDC, when classifying individuals from two spherical normal populations. They have concluded that the sample EDC outperforms the sample LDF when p is large relative to the training sample size. This paper derives conditions for which the two classifiers are equivalent when all parameters are known and employs a Monte Carlo simulation to compare the sample EDC with the sample LDF no only for the spherical normal case but also for several nonspherical parameter configurations. Fo many practical situations, the sample EDC performs as well as or superior to the sample LDF, even for nonspherical covariance configurations.
01 Jan 1987
TL;DR: The estimation of the error rates associated with a given discriminant rule for allocating an object of unknown origin to one of a finite number, say g, of distinct classes or populations was studied in this article.
Abstract: An important problem in discriminant analysis is the estimation of the error rates associated with a given discriminant rule for allocating an object of unknown origin to one of a finite number, say g, of distinct classes or populations. The rule is based on the observed value of a random vector X of p measurements on the object. Over the years there have been many investigations on this problem; see, for example, Hills (1966), Lachenbruch and Mickey (1968), and McLachlan (1974a, b, c), and the references therein. Toussaint (1974) has compiled an extensive bibliography, which has been updated recently by Hand (1986b). An overview of error rate estimation has been given by McLachlan (1986), while recent work on robust error rate estimation has been summarized by Knoke (1986).
TL;DR: In this article, an alternative method, based on ideal point discriminant analysis (DA), is proposed for analysis of contingency tables, which in a certain sense encompasses the two existing methods, namely loglinear model and dual scaling.
Abstract: Cross-classified data are frequently encountered in behavioral and social science research. The loglinear model and dual scaling (correspondence analysis) are two representative methods of analyzing such data. An alternative method, based on ideal point discriminant analysis (DA), is proposed for analysis of contingency tables, which in a certain sense encompasses the two existing methods. A variety of interesting structures can be imposed on rows and columns of the tables through manipulations of predictor variables and/or as direct constraints on model parameters. This, along with maximum likelihood estimation of the model parameters, allows interesting model comparisons. This is illustrated by the analysis of several data sets.
TL;DR: The authors empirically investigated the performance of four relatively new nonparametric techniques against four different parameteric versions of discriminant analysis using financial data drawn from 232 bankrupt and nonbankrupt companies.
Abstract: This paper empirically investigation the performance of four relatively new nonparametric techniques against four different parameteric versions of discriminant analysis. The models were constructed and analyzed using financial data drawn from 232 bankrupt and nonbankrupt companies. Generally, the nonparametric approaches, with the exception of linear programming, performed as well as or better than the more traditional discriminant analysis.
TL;DR: The estimation of probabilities of correct classification is a primary concern in predictive discriminant analysis and methods of estimating these hit rates include formulas, resubstitution, and external analyses.
Abstract: The estimation of probabilities of correct classification is a primary concern in predictive discriminant analysis. Three such probabilities are: (a) the optimal hit rate, that obtained when the classification rule is based on known parameters; (b) the actual hit rate, that obtained by applying a rule based on a particular sample to future samples; and (c) the expected actual hit rate. Methods of estimating these hit rates include formulas (in the two-group case), resubstitution, and external analyses. The methods are tentatively compared via Monte Carlo sampling from two real data sets.
TL;DR: In this article, frequency-based pattern recognition concepts using linear discriminant functions have been used in analysing acoustic emission signals generated during machining to distinguish between different signal sources, specifically chip formation, tool fracture, and chip noise.
01 Jan 1987
TL;DR: In multivariate methods involving several populations, such as discriminant analysis or MANOVA, equality of all covariance matrices is a frequent assumption and the usual reaction is to estimate the covariance matrix individually in each group as mentioned in this paper.
Abstract: In multivariate methods involving several populations, such as discriminant analysis or MANOVA, equality of all covariance matrices is a frequent assumption. If a test for equality of the covariance matrices suggests that this assumption does not hold, the usual reaction is to estimate the covariance matrices individually in each group. For k populations and p variables this means that the number of parameters estimated increases by (k−1)p(p−1)/2, which is quadratic in p. In many practical applications (as in the example given in section 4), this is not satisfactory, for two reasons: First, the k covariance matrices, although not being identical, may exhibit some common structure. Second, in parametric model fitting, the “principle of parsimony” (Dempster, 1972, p. 157) suggests that parameters should be introduced sparingly and only when the data indicate that they are needed.
01 Apr 1987
TL;DR: A two-stage isolated word speech recognition system that uses a Hidden Markov Model (HMM) recognizer in the first stage and a discriminant analysis system in the second stage, reducing the overall error rate by more than a factor of two.
Abstract: This paper describes a two-stage isolated word speech recognition system that uses a Hidden Markov Model (HMM) recognizer in the first stage and a discriminant analysis system in the second stage. During recognition, when the first-stage recognizer is unable to clearly differentiate between acoustically similar words such as "go" and "no" the second-stage discriminator is used. The second-stage system focuses on those parts of the unknown token which are most effective at discriminating the confused words. The system was tested on a 35 word, 10,710 token stress speech isolated word data base created at Lincoln Laboratory. Adding the second-stage discriminating system produced the best results to date on this data base, reducing the overall error rate by more than a factor of two.
Journal Article•
TL;DR: Results from discriminant analysis and logistic regression were compared using two data sets from a study on predictors of coliform mastitis in dairy cows to find the coefficients from logistic regressions are preferable.
Abstract: Results from discriminant analysis and logistic regression were compared using two data sets from a study on predictors of coliform mastitis in dairy cows. Both techniques selected the same set of variables as important predictors and were of nearly equal value in classifying cows as having, or not having mastitis. The logistic regression model made fewer classification errors. The magnitudes of the effects were considerably different for some variables. Given the failure to meet the underlying assumptions of discriminant analysis, the coefficients from logistic regression are preferable.
TL;DR: Extensions to Fisher's linear discriminant function which allow both differences in class means and covariances to be systematically included in a process for feature reduction are described.
Abstract: This correspondence describes extensions to Fisher's linear discriminant function which allow both differences in class means and covariances to be systematically included in a process for feature reduction. It is shown how the Fukunaga-Koontz transform can be combined with Fisher's method to allow a reduction of feature space from many dimensions to two. Performance is seen to be superior in general to the Foley-Sammon method. The technique is developed to show how a new radius vector (or pair of radius vectors) can be combined with Fisher's vector to produce a classifier with even more power of discrimination. Illustrations of the technique show that good discrimination can be obtained even if there is considerable overlap of classes in any one projection.
01 Jan 1987
TL;DR: In this article, the authors present an introduction to discriminant analysis, which is useful in petrographic problems as well as in taxonomic problems of classifying individuals into various species or subspecies.
Abstract: This chapter presents an introduction to discriminant analysis. Dicriminant analysis can be applied if each sample is described by at least two numerical properties. The basic idea of discriminant analysis is simple. The linear combination is termed a discriminant function. Obviously, the approach is useful, for instance in petrographic problems as well as in taxonomic problems of classifying individuals into various species or subspecies. The simultaneous investigation of more than three properties is rather tedious and difficult without a computer program. In many situations, however, the investigation of certain combinations of two or three properties leads to results that are close to the optimum of the combination of all qualities.
TL;DR: A fast and novel nonoverlapping planar shape recognition scheme with efficiency virtually independent of the number of prototypes is developed, composed of new contour normalization, control point extraction and discriminant analysis algorithms.
TL;DR: The usefulness of multivariate analysis in clinical practice is confirmed, with an overall accuracy of 87.2%, and the rate of correct prediction was 84.6% for cancer recurrence, and 88.2% for survival without recurrence.
Abstract: In order to assess the feasibility of computer-assisted prognostication, step-wise discriminant analysis was applied to data retrospectively obtained from 243 patients who underwent surgery for cancer of the larynx. In all, 62 variables were studied in each patient. Ninety-four out of 243 patients had complete information on the 62 variables studied. With data from these 94 patients a linear discriminant function was obtained, with an overall accuracy of 87.2%. The rate of correct prediction was 84.6% for cancer recurrence, and 88.2% for survival without recurrence. Only 11 variables of the 62 used for analysis were necessary to obtain these results. This paper confirms the usefulness of multivariate analysis in clinical practice.
TL;DR: In this paper, a general expression for the moments of a statistic which contains Anderson's linear discriminant function and the minimum variance unbiased estimator of the log odds ratio as special cases is given in terms of certain invariant polynomials of matrix argument.
Abstract: SUMMARY A general expression is obtained for the moments of a statistic which contains Anderson's linear discriminant function and the minimum variance unbiased estimator of the log odds ratio as special cases. The result is given in terms of certain invariant polynomials of matrix argument, and is used to derive the first four exact central moments, together with asymptotic expansions of the cumulants. This provides an alternative approach to Okamoto's expansion as an Edgeworth series. An asymptotic confidence interval is also obtained for the log odds ratio, using a method of Peers & Iqbal (1985), which allows for the estimation of nuisance parameters. Simulation shows that the interval has quite good properties over a range of parameter values. Some key works: Asymptotic confidence interval; Cumulant; Discriminant analysis; Invariant polynomial; Linear discriminant function; Log odds ratio; Nuisance parameter; Simulation study; Su curve; Zonal polynomial.
TL;DR: In this paper, the authors examined the effects of five univariate scoring techniques for rank order categorical data and the results of analyses using each of the techniques for five and ten-point bi-polar adjective scales.
Abstract: The purpose of this article is to examine the effects of five univariate scoring techniques for rank order categorical data and the results of analyses using each of the techniques for five‐ and ten‐point bi‐polar adjective scales. The effect of scoring method and scale length is assessed for the resultant distance to multivariate normality, inter‐item reliability, discriminant analysis, least squares regression and logistic regression. For these data, the strongest effect of scoring was on distance to multivariate normality and determination of significant variables.
TL;DR: With an increase in sample size, ID3 seemed to break down, producing a large, complex decision tree of dubious generality, whereas discriminant analysis, with a larger sample sizes, used more independent variables but maintained its first set accuracy.
TL;DR: In this article, a linear mixture is defined as a random vector y in which the variable are a (non-negative) weighted average of corresponding variables, assumed to characterize g component groups.
Abstract: This paper proposes an elegant, yet straightforward, model for classifying linear mixtures. A linear mixture is defined as a random vector y in which the variable are a (non-negative) weighted average of corresponding variables, assumed to characterize g component groups. These weights are referred to as ‘mixing proportions’. The model seeks to identify the mixture constituents and estimate the mixing proportions. It is demonstrated within the context of high resolution gas chromatography and the problem of identifying the constituents in polychlorinated biphenyl mixtures.