K-means is undoubtedly the most widely used partitional clustering algorithm. Unfortunately, due to its gradient descent nature, this algorithm is highly sensitive to the initial placement of the cluster centers. Numerous initialization methods have been proposed to address this problem. In this paper, we first present an overview of these methods with an emphasis on their computational efficiency. We then compare eight commonly used linear time complexity initialization methods on a large and diverse collection of data sets using various performance criteria. Finally, we analyze the experimental results using non-parametric statistical tests and provide recommendations for practitioners. We demonstrate that popular initialization methods often perform poorly and that there are in fact strong alternatives to these methods.

A comparative study of efficient initialization methods for the k-means clustering algorithm

Data clustering is an important technique for exploratory data analysis, and has been studied for several years. It has been shown to be useful in many practical domains such as data classification and image processing. Recently, there has been a growing emphasis on exploratory analysis of very large datasets to discover useful patterns and/or correlations among attributes. This is called data mining, and data clustering is regarded as a particular branch. However existing data clustering methods do not adequately address the problem of processing large datasets with a limited amount of resources (e.g., memory and cpu cycles). So as the dataset size increases, they do not scale up well in terms of memory requirement, running time, and result quality.

In this paper, an efficient and scalable data clustering method is proposed, based on a new in-memory data structure called CF-tree, which serves as an in-memory summary of the data distribution. We have implemented it in a system called BIRCH (Balanced Iterative Reducing and Clustering using Hierarchies), and studied its performance extensively in terms of memory requirements, running time, clustering quality, stability and scalability; we also compare it with other available methods. Finally, BIRCH is applied to solve two real-life problems: one is building an iterative and interactive pixel classification tool, and the other is generating the initial codebook for image compression.

BIRCH: A New Data Clustering Algorithm and Its Applications

This paper surveys current technology and research in the area of digital color imaging. In order to establish the background and lay down terminology, fundamental concepts of color perception and measurement are first presented using vector-space notation and terminology. Present-day color recording and reproduction systems are reviewed along with the common mathematical models used for representing these devices. Algorithms for processing color images for display and communication are surveyed, and a forecast of research trends is attempted. An extensive bibliography is provided.

Digital color imaging

A method for initialising the K-means clustering algorithm using kd-trees

How much can k-means be improved by using better initialization and repeats?

Three fast search routines to be used in the encoding phase of vector quantization (VQ) image compression systems are presented. These routines, which are based on geometric considerations, provide the same results as an exhaustive (or full) search. Examples show that the proposed algorithms need only 3-20% of the number of mathematical operations required by a full search and fewer than 50% of the operations required by recently proposed alternatives. >

Fast full search equivalent encoding algorithms for image compression using vector quantization

A review and a performance comparison of several often-used vector quantization (VQ) codebook generation algorithms are presented. The codebook generation algorithms discussed include the Linde-Buzo-Gray (LBG) binary-splitting algorithm, the pairwise nearest-neighbor algorithm, the simulated annealing algorithm, and the fuzzy c-means clustering analysis algorithm. A new directed-search binary-splitting method which reduces the complexity of the LBG algorithm, is presented. Also, a new initial codebook selection method which can obtain a good initial codebook is presented. By using this initial codebook selection algorithm, the overall LBG codebook generation time can be reduced by a factor of 1.5-2. >

A comparison of several vector quantization codebook generation approaches

This article discusses bit allocation and adaptive search algorithms for mean-residual vector quantization (MRVQ) and multistage vector quantization (MSVQ). The adaptive search algorithm uses a buffer and a distortion threshold function to control the bit rate that is assigned to each input vector. It achieves a constant rate for the entire image but variable bit rate for each vector in the image. For a given codebook and several bit rates, we compare the performance between the optimal bit allocation and adaptive search algorithms. The results show that the performance of the adaptive search algorithm is only 0.20-0.53 dB worse than that of the optimal bit allocation algorithm, but the complexity of the adaptive search algorithm is much less than that of the optimal bit allocation algorithm. >

C.-M. Huang

Papers

Fast full search equivalent encoding algorithms for image compression using vector quantization

A comparison of several vector quantization codebook generation approaches

Comparison between adaptive search and bit allocation algorithms for image compression using vector quantization