The Theory of Matrices. By F R. Gantmacher. Two volumes, pp. 374 and 276. 1959. (Translated from the Russian by K. A. Hirsch; Chelsea Publishing Company, New York)

This chapter introduces the subject of statistical pattern recognition (SPR). It starts by considering how features are defined and emphasizes that the nearest neighbor algorithm achieves error rates comparable with those of an ideal Bayes’ classifier. The concepts of an optimal number of features, representativeness of the training data, and the need to avoid overfitting to the training data are stressed. The chapter shows that methods such as the support vector machine and artificial neural networks are subject to these same training limitations, although each has its advantages. For neural networks, the multilayer perceptron architecture and back-propagation algorithm are described. The chapter distinguishes between supervised and unsupervised learning, demonstrating the advantages of the latter and showing how methods such as clustering and principal components analysis fit into the SPR framework. The chapter also defines the receiver operating characteristic, which allows an optimum balance between false positives and false negatives to be achieved.

Statistical Pattern Recognition

Fuzzy c-means clustering (FCM) with spatial constraints (FCM/spl I.bar/S) is an effective algorithm suitable for image segmentation. Its effectiveness contributes not only to the introduction of fuzziness for belongingness of each pixel but also to exploitation of spatial contextual information. Although the contextual information can raise its insensitivity to noise to some extent, FCM/spl I.bar/S still lacks enough robustness to noise and outliers and is not suitable for revealing non-Euclidean structure of the input data due to the use of Euclidean distance (L/sub 2/ norm). In this paper, to overcome the above problems, we first propose two variants, FCM/spl I.bar/S/sub 1/ and FCM/spl I.bar/S/sub 2/, of FCM/spl I.bar/S to aim at simplifying its computation and then extend them, including FCM/spl I.bar/S, to corresponding robust kernelized versions KFCM/spl I.bar/S, KFCM/spl I.bar/S/sub 1/ and KFCM/spl I.bar/S/sub 2/ by the kernel methods. Our main motives of using the kernel methods consist in: inducing a class of robust non-Euclidean distance measures for the original data space to derive new objective functions and thus clustering the non-Euclidean structures in data; enhancing robustness of the original clustering algorithms to noise and outliers, and still retaining computational simplicity. The experiments on the artificial and real-world datasets show that our proposed algorithms, especially with spatial constraints, are more effective.

Robust image segmentation using FCM with spatial constraints based on new kernel-induced distance measure

https://eprints.whiterose.ac.uk/8536/2/Filippone_spectral_methodspr08.pdf

A survey of kernel and spectral methods for clustering

The Fuzzy c-means is one of the most popular ongoing area of research among all types of researchers including Computer science, Mathematics and other areas of engineering, as well as all areas of optimization practices. Several problems from various areas have been effectively solved by using FCM and its different variants. But, for efficient use of the algorithm in various diversified applications, some modifications or hybridization with other algorithms are needed. A comprehensive survey on FCM and its applications in more than one decade has been carried out in this paper to show the efficiency and applicability in a mixture of domains. Also, another intention of this survey is to encourage new researchers to make use of this simple algorithm (which is popularly called soft classification model) in problem solving.

Fuzzy C-Means (FCM) Clustering Algorithm: A Decade Review from 2000 to 2014

Fuzzy c-varieties/elliptotypes clustering in reproducing kernel Hilbert space

Fuzzy clustering can be helpful in finding natural vague boundaries in data. The fuzzy c-means method is one of the most popular clustering methods based on minimization of a criterion function. However, one of the greatest disadvantages of this method is its sensitivity to the presence of noise and outliers in the data. The present paper introduces a new e-insensitive Fuzzy C-Means (eFCM) clustering algorithm. As a special case, this algorithm includes the well-known Fuzzy C-Medians method (FCMED). The performance of the new clustering algorithm is experimentally compared with the Fuzzy C-Means (FCM) method using synthetic data with outliers and heavy-tailed, overlapped groups of the data.

/pdf/an-e-insensitive-approach-to-fuzzy-clustering-5cx98ltf0k.pdf

An ε-insensitive approach to fuzzy clustering

This paper introduces a new classifier design method that is based on a modification of the classical Ho-Kashyap procedure. The proposed method uses the absolute error, rather than the squared error, to design a linear classifier. Additionally, easy control of the generalization ability and robustness to outliers are obtained. Next, an extension to a nonlinear classifier by the mixture-of-experts technique is presented. Each expert is represented by a fuzzy if-then rule in the Takagi-Sugeno-Kang form. Finally, examples are given to demonstrate the validity of the introduced method.

/pdf/a-fuzzy-if-then-rule-based-nonlinear-classifier-3zofuyj76e.pdf

A fuzzy if-then rule-based nonlinear classifier

This paper introduces a new classifier design method based on a kernel extension of the classical Ho-Kashyap procedure. The proposed method uses an approximation of the absolute error rather than the squared error to design a classifier, which leads to robustness against outliers and a better approximation of the misclassification error. Additionally, easy control of the generalization ability is obtained using the structural risk minimization induction principle from statistical learning theory. Finally, examples are given to demonstrate the validity of the introduced method.

/pdf/kernel-ho-kashyap-classifier-with-generalization-control-yi0sd86hlk.pdf

Jacek Łęski

Papers

Fuzzy c-varieties/elliptotypes clustering in reproducing kernel Hilbert space

An ε-insensitive approach to fuzzy clustering

A fuzzy if-then rule-based nonlinear classifier

Kernel Ho-Kashyap classifier with generalization control

A time-domain-constrained fuzzy clustering method and its application to signal analysis