Reinforcement learning, one of the most active research areas in artificial intelligence, is a computational approach to learning whereby an agent tries to maximize the total amount of reward it receives when interacting with a complex, uncertain environment. In Reinforcement Learning, Richard Sutton and Andrew Barto provide a clear and simple account of the key ideas and algorithms of reinforcement learning. Their discussion ranges from the history of the field's intellectual foundations to the most recent developments and applications. The only necessary mathematical background is familiarity with elementary concepts of probability. The book is divided into three parts. Part I defines the reinforcement learning problem in terms of Markov decision processes. Part II provides basic solution methods: dynamic programming, Monte Carlo methods, and temporal-difference learning. Part III presents a unified view of the solution methods and incorporates artificial neural networks, eligibility traces, and planning; the two final chapters present case studies and consider the future of reinforcement learning.

/pdf/reinforcement-learning-an-introduction-rzxgej9p17.pdf

Reinforcement Learning: An Introduction

There is, I think, something ethereal about i —the square root of minus one. I remember first hearing about it at school. It seemed an odd beast at that time—an intruder hovering on the edge of reality.

Usually familiarity dulls this sense of the bizarre, but in the case of i it was the reverse: over the years the sense of its surreal nature intensified. It seemed that it was impossible to write mathematics that described the real world in …

I and i

The increasing volume of data in modern business and science calls for more complex and sophisticated tools. Although advances in data mining technology have made extensive data collection much easier, it's still always evolving and there is a constant need for new techniques and tools that can help us transform this data into useful information and knowledge. Since the previous edition's publication, great advances have been made in the field of data mining. Not only does the third of edition of Data Mining: Concepts and Techniques continue the tradition of equipping you with an understanding and application of the theory and practice of discovering patterns hidden in large data sets, it also focuses on new, important topics in the field: data warehouses and data cube technology, mining stream, mining social networks, and mining spatial, multimedia and other complex data. Each chapter is a stand-alone guide to a critical topic, presenting proven algorithms and sound implementations ready to be used directly or with strategic modification against live data. This is the resource you need if you want to apply today's most powerful data mining techniques to meet real business challenges. * Presents dozens of algorithms and implementation examples, all in pseudo-code and suitable for use in real-world, large-scale data mining projects. * Addresses advanced topics such as mining object-relational databases, spatial databases, multimedia databases, time-series databases, text databases, the World Wide Web, and applications in several fields. *Provides a comprehensive, practical look at the concepts and techniques you need to get the most out of real business data

/pdf/data-mining-concepts-and-techniques-4dtvdfkvmi.pdf

Data Mining: Concepts and Techniques

Data Mining: Practical Machine Learning Tools and Techniques offers a thorough grounding in machine learning concepts as well as practical advice on applying machine learning tools and techniques in real-world data mining situations. This highly anticipated third edition of the most acclaimed work on data mining and machine learning will teach you everything you need to know about preparing inputs, interpreting outputs, evaluating results, and the algorithmic methods at the heart of successful data mining. Thorough updates reflect the technical changes and modernizations that have taken place in the field since the last edition, including new material on Data Transformations, Ensemble Learning, Massive Data Sets, Multi-instance Learning, plus a new version of the popular Weka machine learning software developed by the authors. Witten, Frank, and Hall include both tried-and-true techniques of today as well as methods at the leading edge of contemporary research. *Provides a thorough grounding in machine learning concepts as well as practical advice on applying the tools and techniques to your data mining projects *Offers concrete tips and techniques for performance improvement that work by transforming the input or output in machine learning methods *Includes downloadable Weka software toolkit, a collection of machine learning algorithms for data mining tasks-in an updated, interactive interface. Algorithms in toolkit cover: data pre-processing, classification, regression, clustering, association rules, visualization

Data Mining: Practical Machine Learning Tools and Techniques

https://www2.econ.iastate.edu/tesfatsi/DeepLearningInNeuralNetworksOverview.JSchmidhuber2015.pdf

Deep learning in neural networks

This article presents an incremental algorithm for inducing decision trees equivalent to those formed by Quinlan's nonincremental ID3 algorithm, given the same training instances. The new algorithm, named ID5R, lets one apply the ID3 induction process to learning tasks in which training instances are presented serially. Although the basic tree-building algorithms differ only in how the decision trees are constructed, experiments show that incremental training makes it possible to select training instances more carefully, which can result in smaller decision trees. The ID3 algorithm and its variants are compared in terms of theoretical complexity and empirical behavior.

/pdf/incremental-induction-of-decision-trees-4e79lnj66o.pdf

Incremental Induction of Decision Trees

The ability to restructure a decision tree efficiently enables a variety of approaches to decision tree induction that would otherwise be prohibitively expensive. Two such approaches are described here, one being incremental tree induction (ITI), and the other being non-incremental tree induction using a measure of tree quality instead of test quality (DMTI). These approaches and several variants offer new computational and classifier characteristics that lend themselves to particular applications.

/pdf/decision-tree-induction-based-on-efficient-tree-3pwjqsh6iu.pdf

Decision Tree Induction Based on Efficient Tree Restructuring

Unlike a univariate decision tree, a multivariate decision tree is not restricted to splits of the instance space that are orthogonal to the features' axes. This article addresses several issues for constructing multivariate decision trees: representing a multivariate test, including symbolic and numeric features, learning the coefficients of a multivariate test, selecting the features to include in a test, and pruning of multivariate decision trees. We present several new methods for forming multivariate decision trees and compare them with several well-known methods. We compare the different methods across a variety of learning tasks, in order to assess each method's ability to find concise, accurate decision trees. The results demonstrate that some multivariate methods are in general more effective than others (in the context of our experimental assumptions). In addition, the experiments confirm that allowing multivariate tests generally improves the accuracy of the resulting decision tree over a univariate tree.

/pdf/multivariate-decision-trees-500ll42ehz.pdf

Multivariate Decision Trees

This chapter concerns learning heuristic problem-solving strategies through experience. In particular, we focus on the issue of learning heuristics to guide a forward-search problem solver, and describe a computer program called lex, which acquires problem-solving heuristics in the domain of symbolic integration. lex acquires and modifies heuristics by iteratively applying the following process: (i) generate a practice problem; (ii) use available heuristics to solve this problem; (iii) analyze the search steps performed in obtaining the solution; and (iv) propose and refine new domain-specific heuristics to improve performance on subsequent problems. We describe the methods currently used by lex, analyze strengths and weaknesses of these methods, and discuss our current research toward more powerful approaches to learning heuristics.

Learning by experimentation: acquiring and refining problem-solving heuristics

We identify and examine the fundamental role that bias plays in inductive concept learning. Bias is the set of all influences, procedural or declarative, that causes a concept learner to prefer one hypothesis to another. Much of the success of concept learning programs to date results from the program's author having provided the learning program with appropriate bias. To date there has been no good mechanical method for shifting from one bias to another that is better. Instead, the author of a learning program has himself had to search for a better bias. The program author manually generates a bias, from scratch or by revising a previous bias, and then tests it in his program. If the author is not satisfied with the induced concepts, then he repeats the manual-generate and program-test cycle. If the author is satisfied, then he deems his program successful. Too often, he does not recognize his own role in the learning process.
Our thesis is that search for appropriate bias is itself a major part of the learning task, and that we can create mechanical procedures for conducting a well-directed search for an appropriate bias. We would like to understand better how a program author goes about doing his search for appropriate bias. What insights does he have? What does he learn when he observes that a particular bias produces poor performance? What domain knowledge does he apply?
We explore the problem of mechanizing the search for appropriate bias. To that end, we develop a framework for a procedure that shifts bias. We then build two instantiations of the procedure in a program called STABB, which we then incorporate in the LEX learning program. One, called "least disjunction", uses simple syntactic manipulation, and the other, called "constraint back propagation" uses analytic deduction. We report experiments with the implementations that both demonstrate the usefulness of the framework, and uncover important issues for this kind of learning.

Paul E. Utgoff

Papers

Incremental Induction of Decision Trees

Decision Tree Induction Based on Efficient Tree Restructuring

Multivariate Decision Trees

Learning by experimentation: acquiring and refining problem-solving heuristics

Shift of bias for inductive concept learning