The increasing volume of data in modern business and science calls for more complex and sophisticated tools. Although advances in data mining technology have made extensive data collection much easier, it's still always evolving and there is a constant need for new techniques and tools that can help us transform this data into useful information and knowledge. Since the previous edition's publication, great advances have been made in the field of data mining. Not only does the third of edition of Data Mining: Concepts and Techniques continue the tradition of equipping you with an understanding and application of the theory and practice of discovering patterns hidden in large data sets, it also focuses on new, important topics in the field: data warehouses and data cube technology, mining stream, mining social networks, and mining spatial, multimedia and other complex data. Each chapter is a stand-alone guide to a critical topic, presenting proven algorithms and sound implementations ready to be used directly or with strategic modification against live data. This is the resource you need if you want to apply today's most powerful data mining techniques to meet real business challenges. * Presents dozens of algorithms and implementation examples, all in pseudo-code and suitable for use in real-world, large-scale data mining projects. * Addresses advanced topics such as mining object-relational databases, spatial databases, multimedia databases, time-series databases, text databases, the World Wide Web, and applications in several fields. *Provides a comprehensive, practical look at the concepts and techniques you need to get the most out of real business data

/pdf/data-mining-concepts-and-techniques-4dtvdfkvmi.pdf

Data Mining: Concepts and Techniques

We are given a large database of customer transactions. Each transaction consists of items purchased by a customer in a visit. We present an efficient algorithm that generates all significant association rules between items in the database. The algorithm incorporates buffer management and novel estimation and pruning techniques. We also present results of applying this algorithm to sales data obtained from a large retailing company, which shows the effectiveness of the algorithm.

/pdf/mining-association-rules-between-sets-of-items-in-large-13fvg5fahi.pdf

Mining association rules between sets of items in large databases

Background on genetic algorithms, LISP, and genetic programming hierarchical problem-solving introduction to automatically-defined functions - the two-boxes problem problems that straddle the breakeven point for computational effort Boolean parity functions determining the architecture of the program the lawnmower problem the bumblebee problem the increasing benefits of ADFs as problems are scaled up finding an impulse response function artificial ant on the San Mateo trail obstacle-avoiding robot the minesweeper problem automatic discovery of detectors for letter recognition flushes and four-of-a-kinds in a pinochle deck introduction to biochemistry and molecular biology prediction of transmembrane domains in proteins prediction of omega loops in proteins lookahead version of the transmembrane problem evolutionary selection of the architecture of the program evolution of primitives and sufficiency evolutionary selection of terminals evolution of closure simultaneous evolution of architecture, primitive functions, terminals, sufficiency, and closure the role of representation and the lens effect Appendices: list of special symbols list of special functions list of type fonts default parameters computer implementation annotated bibliography of genetic programming electronic mailing list and public repository

Genetic Programming: On the Programming of Computers by Means of Natural Selection

We consider the problem of discovering association rules between items in a large database of sales transactions. We present two new algorithms for solving thii problem that are fundamentally different from the known algorithms. Empirical evaluation shows that these algorithms outperform the known algorithms by factors ranging from three for small problems to more than an order of magnitude for large problems. We also show how the best features of the two proposed algorithms can be combined into a hybrid algorithm, called AprioriHybrid. Scale-up experiments show that AprioriHybrid scales linearly with the number of transactions. AprioriHybrid also has excellent scale-up properties with respect to the transaction size and the number of items in the database.

/pdf/fast-algorithms-for-mining-association-rules-3yuau93ycd.pdf

Fast algorithms for mining association rules

/pdf/fast-algorithms-for-mining-association-rules-in-large-uffixqgjfa.pdf

Fast Algorithms for Mining Association Rules in Large Databases

Scientific discovery is often regarded as romantic and creative -- and hence unanalyzable -- whereas the everyday process of verifying discoveries is sober and more suited to analysis. Yet this fascinating exploration of how scientific work proceeds argues that however sudden the moment of discovery may seem, the discovery process can be described and modeled. Using the methods and concepts of contemporary information-processing psychology (or cognitive science) the authors develop a series of artificial-intelligence programs that can simulate the human thought processes used to discover scientific laws. The programs -- BACON, DALTON, GLAUBER, and STAHL -- are all largely data-driven, that is, when presented with series of chemical or physical measurements they search for uniformities and linking elements, generating and checking hypotheses and creating new concepts as they go along. Scientific Discovery examines the nature of scientific research and reviews the arguments for and against a normative theory of discovery; describes the evolution of the BACON programs, which discover quantitative empirical laws and invent new concepts; presents programs that discover laws in qualitative and quantitative data; and ties the results together, suggesting how a combined and extended program might find research problems, invent new instruments, and invent appropriate problem representations. Numerous prominent historical examples of discoveries from physics and chemistry are used as tests for the programs and anchor the discussion concretely in the history of science.

Scientific Discovery: Computational Explorations of the Creative Processes

Part A: Data mining and knowledge discovery Part B: Fundamental Concepts Part C: The process of knowledge discovery in databases Part D: Discovery Systems Part E: Interdisciplinary links of KDD Part F: Business problems Part G: Industry sectors Part H: KDD in practice: case studies

Handbook of Data Mining and Knowledge Discovery

Scientific discovery

: Scientific discovery is a complex activity involving many different components. Our interest in discovery has led us to construct four artificial intelligence systems that address different facets of this process. BACON.6 focuses on discovering empirical laws that summarize numerical data. This program searches a space of data and a space of numerical laws, and includes methods for postulating intrinsic properties and noting common divisors. GLAUBER is concerned with discovering laws of qualitative structure, such as the hypothesis that acids react with alkalis to form salts. It searches the space of qualitative laws, using evaluation functions to focus attention on laws covering the greatest number of observed facts. STAHL attempts to determine the components of substances involved in reactions, and has been used to model the reasoning that led to the phlogiston theory. This system searches through the space of componential models, using heuristics to make plausible inferences. The final system, DALTON, is concerned with formulating structural models of chemical reactions. It searches the space of possible models, considering simple models before more complex ones and using a conservation assumption to constrain possibilities. While each of these discovery systems is interesting in its own right, we are also exploring ways in which the systems can interact to help direct each other's search processes. (Author)

/pdf/the-search-for-regularity-four-aspects-of-scientific-4h7iqavscy.pdf

The Search for Regularity: Four Aspects of Scientific Discovery

: In the last decade a few artificial intelligence researchers have turned their attention to a domain often considered the realm of genius - scientific discovery. The vast majority of this work has focused on empirical discovery, and much of the effort has been concerned with the discovery of numeric laws. This paper traces one evolutionary chain of research on discovery in particular the development of data-driven heuristic methods relating to numeric discovery. The authors examine four systems - Gerwin's function induction system, Langley, Bradshaw, and Simon's BACON, Zytkow's FAHRENHEIT, and Nordhausen and Langley's IDS - and describe how each program introduces abilities lacking in earlier systems. The conceptual advances involve three different but interrelated aspects of discovery: the form of laws and theoretical terms discovered; the ability to determine the scope and context of laws; and the ability to design experiments. This document evaluates each of the systems, but focuses on their theoretical contributions rather than on reporting their behavior in specific domains. It closes the paper by reviewing the work on machine discovery from the views of the history and philosophy of science. (KR)

/pdf/data-driven-approaches-to-empirical-discovery-2com4891mo.pdf

Jan M. Zytkow

Papers

Scientific Discovery: Computational Explorations of the Creative Processes

Handbook of Data Mining and Knowledge Discovery

Scientific discovery

The Search for Regularity: Four Aspects of Scientific Discovery

Data-driven approaches to empirical discovery