The increasing volume of data in modern business and science calls for more complex and sophisticated tools. Although advances in data mining technology have made extensive data collection much easier, it's still always evolving and there is a constant need for new techniques and tools that can help us transform this data into useful information and knowledge. Since the previous edition's publication, great advances have been made in the field of data mining. Not only does the third of edition of Data Mining: Concepts and Techniques continue the tradition of equipping you with an understanding and application of the theory and practice of discovering patterns hidden in large data sets, it also focuses on new, important topics in the field: data warehouses and data cube technology, mining stream, mining social networks, and mining spatial, multimedia and other complex data. Each chapter is a stand-alone guide to a critical topic, presenting proven algorithms and sound implementations ready to be used directly or with strategic modification against live data. This is the resource you need if you want to apply today's most powerful data mining techniques to meet real business challenges. * Presents dozens of algorithms and implementation examples, all in pseudo-code and suitable for use in real-world, large-scale data mining projects. * Addresses advanced topics such as mining object-relational databases, spatial databases, multimedia databases, time-series databases, text databases, the World Wide Web, and applications in several fields. *Provides a comprehensive, practical look at the concepts and techniques you need to get the most out of real business data

/pdf/data-mining-concepts-and-techniques-4dtvdfkvmi.pdf

Data Mining: Concepts and Techniques

We present a full-sky 100 μm map that is a reprocessed composite of the COBE/DIRBE and IRAS/ISSA maps, with the zodiacal foreground and confirmed point sources removed. Before using the ISSA maps, we remove the remaining artifacts from the IRAS scan pattern. Using the DIRBE 100 and 240 μm data, we have constructed a map of the dust temperature so that the 100 μm map may be converted to a map proportional to dust column density. The dust temperature varies from 17 to 21 K, which is modest but does modify the estimate of the dust column by a factor of 5. The result of these manipulations is a map with DIRBE quality calibration and IRAS resolution. A wealth of filamentary detail is apparent on many different scales at all Galactic latitudes. In high-latitude regions, the dust map correlates well with maps of H I emission, but deviations are coherent in the sky and are especially conspicuous in regions of saturation of H I emission toward denser clouds and of formation of H2 in molecular clouds. In contrast, high-velocity H I clouds are deficient in dust emission, as expected. To generate the full-sky dust maps, we must first remove zodiacal light contamination, as well as a possible cosmic infrared background (CIB). This is done via a regression analysis of the 100 μm DIRBE map against the Leiden-Dwingeloo map of H I emission, with corrections for the zodiacal light via a suitable expansion of the DIRBE 25 μm flux. This procedure removes virtually all traces of the zodiacal foreground. For the 100 μm map no significant CIB is detected. At longer wavelengths, where the zodiacal contamination is weaker, we detect the CIB at surprisingly high flux levels of 32 ± 13 nW m-2 sr-1 at 140 μm and of 17 ± 4 nW m-2 sr-1 at 240 μm (95% confidence). This integrated flux ~2 times that extrapolated from optical galaxies in the Hubble Deep Field. The primary use of these maps is likely to be as a new estimator of Galactic extinction. To calibrate our maps, we assume a standard reddening law and use the colors of elliptical galaxies to measure the reddening per unit flux density of 100 μm emission. We find consistent calibration using the B-R color distribution of a sample of the 106 brightest cluster ellipticals, as well as a sample of 384 ellipticals with B-V and Mg line strength measurements. For the latter sample, we use the correlation of intrinsic B-V versus Mg2 index to tighten the power of the test greatly. We demonstrate that the new maps are twice as accurate as the older Burstein-Heiles reddening estimates in regions of low and moderate reddening. The maps are expected to be significantly more accurate in regions of high reddening. These dust maps will also be useful for estimating millimeter emission that contaminates cosmic microwave background radiation experiments and for estimating soft X-ray absorption. We describe how to access our maps readily for general use.

/pdf/maps-of-dust-infrared-emission-for-use-in-estimation-of-207gvl50wo.pdf

Maps of Dust Infrared Emission for Use in Estimation of Reddening and Cosmic Microwave Background Radiation Foregrounds

We consider the problem of discovering association rules between items in a large database of sales transactions. We present two new algorithms for solving thii problem that are fundamentally different from the known algorithms. Empirical evaluation shows that these algorithms outperform the known algorithms by factors ranging from three for small problems to more than an order of magnitude for large problems. We also show how the best features of the two proposed algorithms can be combined into a hybrid algorithm, called AprioriHybrid. Scale-up experiments show that AprioriHybrid scales linearly with the number of transactions. AprioriHybrid also has excellent scale-up properties with respect to the transaction size and the number of items in the database.

/pdf/fast-algorithms-for-mining-association-rules-3yuau93ycd.pdf

Fast algorithms for mining association rules

Current computing and storage technology is rapidly outstripping society's ability to make meaningful use of the torrent of available data. Without a concerted effort to develop knowledge discovery techniques, organizations stand to forfeit much of the value from the data they currently collect and store.

Data mining and knowledge discovery: making sense out of data

■ Data mining and knowledge discovery in databases have been attracting a significant amount of research, industry, and media attention of late. What is all the excitement about? This article provides an overview of this emerging field, clarifying how data mining and knowledge discovery in databases are related both to each other and to related fields, such as machine learning, statistics, and databases. The article mentions particular real-world applications, specific data-mining techniques, challenges involved in real-world applications of knowledge discovery, and current and future research directions in the field.

/pdf/from-data-mining-to-knowledge-discovery-in-databases-4fznxuujb9.pdf

From Data Mining to Knowledge Discovery in Databases

Automating the analysis and cataloging of sky surveys

Automated Star/Galaxy Classification for Digitized Poss-II

We describe the application of decision tree based classification techniques to the development of an automated tool for the reduction of a large scientific data set. The 2nd Palomar Observatory Sky Survey (POSS-II) will provide a comprehensive coverage of the northern celestial hemisphere in the form of digitized photographic plates containing on the order of 0.5 billion objects. It is necessary to classify the detected objects as stars, galaxies, or instrumental artifacts. The size of this data set precludes manual analysis. Image processing routines are used to identify sky objects and to extract a set of basic features for each object that are used to select a useful and robust set of attributes for classifying sky objects. The GID3 * and O-BTree decision tree learning algorithms, in conjunction with the RULER system for statistically pruning and merging multiple trees, are used to classify the detected objects. The learning algorithms produced classifiers that can classify objects in the survey that are too faint for visual classification with an accuracy level of about 94%. This increases the number of classified objects in the final catalog by three-fold relative to the best techniques previously employed by extending at least one magnitude fainter than objects classified in photographic sky surveys to date. SKICAT represents a system in which machine learning played a powerful and enabling role, and solved a difficult, scientifically significant problem.

SKICAT: a machine learning system for automated cataloging of large scale sky surveys

Initial Galaxy Counts from Digitized Poss-II

This paper reports on the discovery of a compact, unresolved infrared nucleus, coincident with the radio core, in the prototypical powerful radio galaxy Cygnus A (3C 405). The infrared colors and magnitudes of the nucleus can be explained as a highly reddened extension of the radio continuum. The implied restframe extinction is A(V) equal to about 50 + or {minus} 30 magnitudes. The extinction-corrected luminosity of the object is in the quasar range. This discovery gives some support to the unification models for quasars and powerful radio galaxies. 35 refs.

N. Weir

Papers

Automating the analysis and cataloging of sky surveys

Automated Star/Galaxy Classification for Digitized Poss-II

SKICAT: a machine learning system for automated cataloging of large scale sky surveys

Initial Galaxy Counts from Digitized Poss-II

Discovery of an Infrared Nucleus in Cygnus A: an Obscured Quasar Revealed?