Clustering is the unsupervised classification of patterns (observations, data items, or feature vectors) into groups (clusters). The clustering problem has been addressed in many contexts and by researchers in many disciplines; this reflects its broad appeal and usefulness as one of the steps in exploratory data analysis. However, clustering is a difficult problem combinatorially, and differences in assumptions and contexts in different communities has made the transfer of useful generic concepts and methodologies slow to occur. This paper presents an overview of pattern clustering methods from a statistical pattern recognition perspective, with a goal of providing useful advice and references to fundamental concepts accessible to the broad community of clustering practitioners. We present a taxonomy of clustering techniques, and identify cross-cutting themes and recent advances. We also describe some important applications of clustering algorithms such as image segmentation, object recognition, and information retrieval.

/pdf/data-clustering-a-review-4c452f5cab.pdf

Data clustering: a review

Large datasets are increasingly common and are often difficult to interpret. Principal component analysis (PCA) is a technique for reducing the dimensionality of such datasets, increasing interpretability but at the same time minimizing information loss. It does so by creating new uncorrelated variables that successively maximize variance. Finding such new variables, the principal components, reduces to solving an eigenvalue/eigenvector problem, and the new variables are defined by the dataset at hand, not a priori , hence making PCA an adaptive data analysis technique. It is adaptive in another sense too, since variants of the technique have been developed that are tailored to various different data types and structures. This article will begin by introducing the basic ideas of PCA, discussing what it can and cannot do. It will then describe some variants of PCA and their application.

https://royalsocietypublishing.org/doi/pdf/10.1098/rsta.2015.0202

Principal component analysis: a review and recent developments

Monographs on statistics and applied probability

Feature selection has been the focus of interest for quite some time and much work has been done. With the creation of huge databases and the consequent requirements for good machine learning techniques, new problems arise and novel approaches to feature selection are in demand. This survey is a comprehensive overview of many existing methods from the 1970's to the present. It identifies four steps of a typical feature selection method, and categorizes the different existing methods in terms of generation procedures and evaluation functions, and reveals hitherto unattempted combinations of generation procedures and evaluation functions. Representative methods are chosen from each category for detailed explanation and discussion via example. Benchmark datasets with different characteristics are used for comparative study. The strengths and weaknesses of different methods are explained. Guidelines for applying feature selection methods are given based on data types and domain characteristics. This survey identifies the future research areas in feature selection, introduces newcomers to this field, and paves the way for practitioners who search for suitable methods for solving domain-specific real-world applications.

/pdf/feature-selection-for-classification-1zmcxc1z1q.pdf

Feature Selection for Classification

http://library.utia.cas.cz/separaty/historie/somol-floating%20search%20methods%20in%20feature%20selection.pdf

Floating search methods in feature selection

This paper presents simple and convenient generalized Minkowski metrics on the multidimensional feature space in which coordinate axes are associated with not only quantitative features but also qualitative and structural features. The metrics are defined on a new mathematical model (U/sup (d/),[+], [X]) which is called simply the Cartesian space model, where U/sup (d/) is the feature space which permits mixed feature types, [+] is the Cartesian join operator which yields a generalized description for given descriptions on U/sup (d/), and [X] is the Cartesian meet operator which extracts a common description from given descriptions on U/sup (d/). To illustrate the effectiveness of our generalized Minkowski metrics, we present an approach to the hierarchical conceptual clustering, and a generalization of the principal component analysis for mixed feature data. >

Generalized Minkowski metrics for mixed feature-type data analysis

General distance functions for mixed feature variables are defined based on the Cartesian space (U/sup (d)/, +,x) which is our new mathematical model for pattern recognition. In our metrics, a sample can be described by a d-tuple which is composed of closed intervals and/or finite sets.

General Metrics For Mixed Features The Cartesian Space Theory For Pattern Recognition

The relative neighborhood graph for mixed feature variables

An optimal method for finding a minimum feature subset based on box classifiers is described. Feature selection is represented as a problem of zero-one integer programming. An implicit enumeration method is developed in order to solve this problem. Numerical examples are presented to illustrate the effectiveness of the approach.

Optimum feature selection by zero-one integer programming

http://link.springer.com/content/pdf/10.1007/978-3-642-61159-9_8.pdf

Manabu Ichino

Papers

Generalized Minkowski metrics for mixed feature-type data analysis

General Metrics For Mixed Features The Cartesian Space Theory For Pattern Recognition

The relative neighborhood graph for mixed feature variables

Optimum feature selection by zero-one integer programming

A Fuzzy Symbolic Pattern Classifier