The increasing volume of data in modern business and science calls for more complex and sophisticated tools. Although advances in data mining technology have made extensive data collection much easier, it's still always evolving and there is a constant need for new techniques and tools that can help us transform this data into useful information and knowledge. Since the previous edition's publication, great advances have been made in the field of data mining. Not only does the third of edition of Data Mining: Concepts and Techniques continue the tradition of equipping you with an understanding and application of the theory and practice of discovering patterns hidden in large data sets, it also focuses on new, important topics in the field: data warehouses and data cube technology, mining stream, mining social networks, and mining spatial, multimedia and other complex data. Each chapter is a stand-alone guide to a critical topic, presenting proven algorithms and sound implementations ready to be used directly or with strategic modification against live data. This is the resource you need if you want to apply today's most powerful data mining techniques to meet real business challenges. * Presents dozens of algorithms and implementation examples, all in pseudo-code and suitable for use in real-world, large-scale data mining projects. * Addresses advanced topics such as mining object-relational databases, spatial databases, multimedia databases, time-series databases, text databases, the World Wide Web, and applications in several fields. *Provides a comprehensive, practical look at the concepts and techniques you need to get the most out of real business data

/pdf/data-mining-concepts-and-techniques-4dtvdfkvmi.pdf

Data Mining: Concepts and Techniques

Data analysis plays an indispensable role for understanding various phenomena. Cluster analysis, primitive exploration with little or no prior knowledge, consists of research developed across a wide variety of communities. The diversity, on one hand, equips us with many tools. On the other hand, the profusion of options causes confusion. We survey clustering algorithms for data sets appearing in statistics, computer science, and machine learning, and illustrate their applications in some benchmark data sets, the traveling salesman problem, and bioinformatics, a new field attracting intensive efforts. Several tightly related topics, proximity measure, and cluster validation, are also discussed.

/pdf/survey-of-clustering-algorithms-6zezqhbjyo.pdf

Survey of clustering algorithms

We present a novel Locality-Sensitive Hashing scheme for the Approximate Nearest Neighbor Problem under lp norm, based on p-stable distributions.Our scheme improves the running time of the earlier algorithm for the case of the lp norm. It also yields the first known provably efficient approximate NN algorithm for the case p

/pdf/locality-sensitive-hashing-scheme-based-on-p-stable-4wdnkpkjck.pdf

Locality-sensitive hashing scheme based on p-stable distributions

Clustering is the division of data into groups of similar objects. In clustering, some details are disregarded in exchange for data simplification. Clustering can be viewed as a data modeling technique that provides for concise summaries of the data. Clustering is therefore related to many disciplines and plays an important role in a broad range of applications. The applications of clustering usually deal with large datasets and data with many attributes. Exploration of such data is a subject of data mining. This survey concentrates on clustering algorithms from a data mining perspective.

A Survey of Clustering Data Mining Techniques

This paper introduces a product quantization-based approach for approximate nearest neighbor search. The idea is to decompose the space into a Cartesian product of low-dimensional subspaces and to quantize each subspace separately. A vector is represented by a short code composed of its subspace quantization indices. The euclidean distance between two vectors can be efficiently estimated from their codes. An asymmetric version increases precision, as it computes the approximate distance between a vector and a code. Experimental results show that our approach searches for nearest neighbors efficiently, in particular in combination with an inverted file system. Results for SIFT and GIST image descriptors show excellent search accuracy, outperforming three state-of-the-art approaches. The scalability of our approach is validated on a data set of two billion vectors.

/pdf/product-quantization-for-nearest-neighbor-search-4lksffz9k7.pdf

Product Quantization for Nearest Neighbor Search

We explore the effect of dimensionality on the "nearest neighbor" problem. We show that under a broad set of conditions (much broader than independent and identically distributed dimensions), as dimensionality increases, the distance to the nearest data point approaches the distance to the farthest data point. To provide a practical perspective, we present empirical results on both real and synthetic data sets that demonstrate that this effect can occur for as few as 10-15 dimensions. 

These results should not be interpreted to mean that high-dimensional indexing is never meaningful; we illustrate this point by identifying some high-dimensional workloads for which this effect does not occur. However, our results do emphasize that the methodology used almost universally in the database literature to evaluate high-dimensional indexing techniques is flawed, and should be modified. In particular, most such techniques proposed in the literature are not evaluated versus simple linear scan, and are evaluated over workloads for which nearest neighbor is not meaningful. Often, even the reported experiments, when analyzed carefully, show that linear scan would outperform the techniques being proposed on the workloads studied in high (10-15) dimensionality!

/pdf/when-is-nearest-neighbor-meaningful-4fyd5cwwqn.pdf

When Is ''Nearest Neighbor'' Meaningful?

We explore the effect of dimensionality on the nearest neighbor problem. We show that under a broad set of conditions (much broader than independent and identically distributed dimensions), as dimensionality increases, the distance to the nearest data point approaches the distance to the farthest data point. To provide a practical perspective, we present empirical results on both real and synthetic data sets that demonstrate that this effect can occur for as few as 10-15 dimensions. These results should not be interpreted to mean that high-dimensional indexing is never meaningful; we illustrate this point by identifying some high-dimensional workloads for which this effect does not occur. However, our results do emphasize that the methodology used almost universally in the database literature to evaluate high-dimensional indexing techniques is flawed, and should be modified. In particular, most such techniques proposed in the literature are not evaluated versus simple linear scan, and are evaluated over workloads for which nearest neighbor is not meaningful. Often, even the reported experiments, when analyzed carefully, show that linear scan would outperform the techniques being proposed on the workloads studied in high (10-15) dimensionality!.

When is nearest neighbor meaningful

We propose a new compression algorithm that is tailored to database applications. It can be applied to a collection of records, and is especially effective for records with many low to medium cardinality fields and numeric fields. In addition, this new technique supports very fast decompression. Promising application domains include decision support systems (DSS), since fact tables, which are by far the largest tables in these applications, contain many low and medium cardinality fields and typically no text fields. Further, our decompression rates are faster than typical disk throughputs for sequential scans; in contrast, gzip is slower. This is important in DSS applications, which often scan large ranges of records. An important distinguishing characteristic of our algorithm, in contrast to compression algorithms proposed earlier, is that we can decompress individual tuples (even individual fields), rather than a full page (or an entire relation) at a time. Also, all the information needed for tuple decompression resides on the same page with the tuple. This means that a page can be stored in the buffer pool and used in compressed form, simplifying the job of the buffer manager and improving memory utilization. Our compression algorithm also improves index structures such as B-trees and R-trees significantly by reducing the number of leaf pages and compressing index entries, which greatly increases the fan-out. We can also use lossy compression on the internal nodes of an index.

/pdf/compressing-relations-and-indexes-43mij6n5j0.pdf

Compressing relations and indexes

The emergence of several new application domains for databases has introduced the need for more efficient complex query handling than databases currently support. These application domains include On-Line AnalyticaIProcessing(OLAP), Geographical Information Systems (GIS), and scientific databases. This paper focuses attention on a form of selection query, expressible in SQL but not evaluated efficiently by current DBMSs, with wide applicability in these new problem domains. We introduce a processing strategy for this class of queries, which we call queries by linear constraints (QBLC). This processing strategy can be implemented with a wide variety of multidimensional indexing structures that include the R-?&ee variants, the k-d-B-pee, the Buddy-Tree, and many more. Note that any processing strategy meant for general database use must guarantee that all correct auswers are returned. Therefore, all numerical techniques we employ uphold &is guarantee. Thus the most distinguishing characteristic of this processing strategy is its safe handling of numerical error (which can result in the dismissal of valid answers). This paper presents several theoretical results about our processing strategy, and the results of several experiments which show that the processing cost of selection queries by linear constraints can be reduced dramatically by using our processing strategy. ‘This work was partially supported by a David and Lucile Packard Foundation Fellowship in Science and Engineering, a Presidential Young Investigator Award, NASA Research Grant NAGW-3921, and OFtD grant 144ET33. . . ., Permission to make digital/hard copies of aIt or part of this mate&l for persons1 or classmom use is grsnted without fee provided thst the copies are not made or distributed for profit or commerckd advulbge, &e copyd&t notice, the title ofthe publication and its d&e appear, and notice is given that WPyright is by permission ofthe ACM, Inc. To copy othe-rwise, to republish tc~ post on servers or to redistribute to I&, requires specific pen&ion and/or fee PODS ‘ 97 Tucson Arizona IJSA ~pyri&t 1997 ACM O-89791-910-6/97/05 ..$xso Raghu Ramakrishnan Computer Sciences Department University of Wisconsin-Madison 1210 W. Dayton St., Madison, WI 53706

Processing queries by linear constraints

This thesis presents new research on two topics: compression in relational database systems, and nearest neighbor processing techniques. 
With the increasing speed of CPUs relative to disks, using compression as a means of improving disk information throughput is becoming very attractive. While it might at first seem reasonable to use traditional compression algorithms such as Lempel-Ziv, these algorithms are unsuitable because they require uncompressing a large portion of the file to retrieve a small piece of data. 
Motivated by this observation, we have developed a compression algorithm for fixed length data that overcomes this problem. The algorithm is simple, and can easily be added to the file management layer of a DBMS since it supports the usual technique of identifying a record by a 〈pageid, slotid 〉 pair (tuple ID). In addition, this algorithm compresses hyper-rectangle based indexing structures such as R-Trees. 
This thesis also describes the relationship between sort orders, multidimensional bulk loading, and compression ratios. Also included in this thesis is an extensive suite of experiments that show the applicability of our compression technique, especially for data warehousing workloads. 
In recent years, many researchers have focused on finding efficient solutions to the nearest neighbor problem. While there have been many efforts to find faster than linear scan processing strategies for these tasks, there has been no success at solving the general problem. 
While previous work shows that the problem can't be solved efficiently in general, we present a technique for either in memory or secondary storage that is guaranteed to perform well in the “good” situations previously described. Furthermore, it is the first strategy that allows a database administrator to trade off space for execution time. It is also the first nearest neighbor strategy whose behavior is completely understood. Finally, there are variants of the problem that perform far better than alternative techniques on “hard” cases.

Jonathan Goldstein

Papers

When Is ''Nearest Neighbor'' Meaningful?

When is nearest neighbor meaningful

Compressing relations and indexes

Processing queries by linear constraints

Improved query processing and data representation techniques