Clustering is the unsupervised classification of patterns (observations, data items, or feature vectors) into groups (clusters). The clustering problem has been addressed in many contexts and by researchers in many disciplines; this reflects its broad appeal and usefulness as one of the steps in exploratory data analysis. However, clustering is a difficult problem combinatorially, and differences in assumptions and contexts in different communities has made the transfer of useful generic concepts and methodologies slow to occur. This paper presents an overview of pattern clustering methods from a statistical pattern recognition perspective, with a goal of providing useful advice and references to fundamental concepts accessible to the broad community of clustering practitioners. We present a taxonomy of clustering techniques, and identify cross-cutting themes and recent advances. We also describe some important applications of clustering algorithms such as image segmentation, object recognition, and information retrieval.

/pdf/data-clustering-a-review-4c452f5cab.pdf

Data clustering: a review

Fuzzy Sets and Fuzzy Logic is a true magnum opus. An enlargement of Fuzzy Sets, Uncertainty,
and Information—an earlier work of Professor Klir and Tina Folger—Fuzzy Sets and Fuzzy Logic
addresses practically every significant topic in the broad expanse of the union of fuzzy set theory
and fuzzy logic. To me Fuzzy Sets and Fuzzy Logic is a remarkable achievement; it covers its vast
territory with impeccable authority, deep insight and a meticulous attention to detail.
To view Fuzzy Sets and Fuzzy Logic in a proper perspective, it is necessary to clarify a point
of semantics which relates to the meanings of fuzzy sets and fuzzy logic.
A frequent source of misunderstanding fias to do with the interpretation of fuzzy logic. The
problem is that the term fuzzy logic has two different meanings. More specifically, in a narrow
sense, fuzzy logic, FLn, is a logical system which may be viewed as an extension and generalization
of classical multivalued logics. But in a wider sense, fuzzy logic, FL^ is almost synonymous
with the theory of fuzzy sets. In this context, what is important to recognize is that: (a) FLW is much
broader than FLn and subsumes FLn as one of its branches; (b) the agenda of FLn is very different
from the agendas of classical multivalued logics; and (c) at this juncture, the term fuzzy logic is
usually used in its wide rather than narrow sense, effectively equating fuzzy logic with FLW
In Fuzzy Sets and Fuzzy Logic, fuzzy logic is interpreted in a sense that is close to FLW. However,
to avoid misunderstanding, the title refers to both fuzzy sets and fuzzy logic.
Underlying the organization of Fuzzy Sets and Fuzzy Logic is a fundamental fact, namely,
that any field X and any theory Y can be fuzzified by replacing the concept of a crisp set in X and Y
by that of a fuzzy set. In application to basic fields such as arithmetic, topology, graph theory, probability
theory and logic, fuzzification leads to fuzzy arithmetic, fuzzy topology, fuzzy graph theory,
fuzzy probability theory and FLn. Similarly, hi application to applied fields such as neural network
theory, stability theory, pattern recognition and mathematical programming, fuzzification leads to
fuzzy neural network theory, fuzzy stability theory, fuzzy pattern recognition and fuzzy mathematical
programming. What is gained through fuzzification is greater generality, higher expressive
power, an enhanced ability to model real-world problems and, most importantly, a methodology for
exploiting the tolerance for imprecision—a methodology which serves to achieve tractability,

Fuzzy Sets and Fuzzy Logic: Theory and Applications

Data analysis plays an indispensable role for understanding various phenomena. Cluster analysis, primitive exploration with little or no prior knowledge, consists of research developed across a wide variety of communities. The diversity, on one hand, equips us with many tools. On the other hand, the profusion of options causes confusion. We survey clustering algorithms for data sets appearing in statistics, computer science, and machine learning, and illustrate their applications in some benchmark data sets, the traveling salesman problem, and bioinformatics, a new field attracting intensive efforts. Several tightly related topics, proximity measure, and cluster validation, are also discussed.

/pdf/survey-of-clustering-algorithms-6zezqhbjyo.pdf

Survey of clustering algorithms

In k-means clustering, we are given a set of n data points in d-dimensional space R/sup d/ and an integer k and the problem is to determine a set of k points in Rd, called centers, so as to minimize the mean squared distance from each data point to its nearest center. A popular heuristic for k-means clustering is Lloyd's (1982) algorithm. We present a simple and efficient implementation of Lloyd's k-means clustering algorithm, which we call the filtering algorithm. This algorithm is easy to implement, requiring a kd-tree as the only major data structure. We establish the practical efficiency of the filtering algorithm in two ways. First, we present a data-sensitive analysis of the algorithm's running time, which shows that the algorithm runs faster as the separation between clusters increases. Second, we present a number of empirical studies both on synthetically generated data and on real data sets from applications in color quantization, data compression, and image segmentation.

/pdf/an-efficient-k-means-clustering-algorithm-analysis-and-1vgn3zr7cg.pdf

An efficient k-means clustering algorithm: analysis and implementation

Monographs on statistics and applied probability

The K-means algorithm is a commonly used technique in cluster analysis. In this paper, several questions about the algorithm are addressed. The clustering problem is first cast as a nonconvex mathematical program. Then, a rigorous proof of the finite convergence of the K-means-type algorithm is given for any metric. It is shown that under certain conditions the algorithm may fail to converge to a local minimum, and that it converges under differentiability conditions to a Kuhn-Tucker point. Finally, a method for obtaining a local-minimum solution is given.

K-Means-Type Algorithms: A Generalized Convergence Theorem and Characterization of Local Optimality

A simulated annealing algorithm for the clustering problem

This paper presents a simulated annealing algorithm (SAA) to solve the unit commitment problem (UCP). New rules for randomly generating feasible solutions are introduced. The problem has two subproblems: a combinatorial optimization problem; and a nonlinear programming problem. The former is solved using the SAA while the latter problem is solved via a quadratic programming routine. Numerical results showed an improvement in the solutions costs compared to previously obtained results.

A simulated annealing algorithm for unit commitment

This paper presents a new algorithm based on integrating genetic algorithms, tabu search and simulated annealing methods to solve the unit commitment problem. The core of the proposed algorithm is based on genetic algorithms. Tabu search is used to generate new population members in the reproduction phase of the genetic algorithm. A simulated annealing method is used to accelerate the convergence of the genetic algorithm by applying the simulated annealing test for all the population members. A new implementation of the genetic algorithm is introduced. The genetic algorithm solution is coded as a mix between binary and decimal representation. The fitness function is constructed from the total operating cost of the generating units without penalty terms. In the tabu search part of the proposed algorithm, a simple short-term memory procedure is used to counter the danger of entrapment at a local optimum, and the premature convergence of the genetic algorithm. A simple cooling schedule has been implemented to apply the simulated annealing test in the algorithm. Numerical results showed the superiority of the solutions obtained compared to genetic algorithms, tabu search and simulated annealing methods, and to two exact algorithms.

Integrating genetic algorithms, tabu search, and simulated annealing for the unit commitment problem

An application of the tabu search (TS) method to solve the unit commitment problem (UCP) is presented. The TS seeks to counter the danger of entrapment at a local optimum by incorporating a memory structure that forbids or penalises certain moves that would return to recently visited solutions. New rules for randomly generating feasible solutions for the UCP are introduced. The problem is divided into two subproblems: a combinatorial optimisation problem and a nonlinear programming problem. The former is solved using the tabu search algorithm (TSA) while the latter problem is solved via a quadratic programming routine. Numerical results show an improvement in the solution cost compared to previously obtained results.

Shokri Z. Selim

Papers

K-Means-Type Algorithms: A Generalized Convergence Theorem and Characterization of Local Optimality

A simulated annealing algorithm for the clustering problem

A simulated annealing algorithm for unit commitment

Integrating genetic algorithms, tabu search, and simulated annealing for the unit commitment problem

Unit commitment by tabu search