Reinforcement learning, one of the most active research areas in artificial intelligence, is a computational approach to learning whereby an agent tries to maximize the total amount of reward it receives when interacting with a complex, uncertain environment. In Reinforcement Learning, Richard Sutton and Andrew Barto provide a clear and simple account of the key ideas and algorithms of reinforcement learning. Their discussion ranges from the history of the field's intellectual foundations to the most recent developments and applications. The only necessary mathematical background is familiarity with elementary concepts of probability. The book is divided into three parts. Part I defines the reinforcement learning problem in terms of Markov decision processes. Part II provides basic solution methods: dynamic programming, Monte Carlo methods, and temporal-difference learning. Part III presents a unified view of the solution methods and incorporates artificial neural networks, eligibility traces, and planning; the two final chapters present case studies and consider the future of reinforcement learning.

/pdf/reinforcement-learning-an-introduction-rzxgej9p17.pdf

Reinforcement Learning: An Introduction

The increasing volume of data in modern business and science calls for more complex and sophisticated tools. Although advances in data mining technology have made extensive data collection much easier, it's still always evolving and there is a constant need for new techniques and tools that can help us transform this data into useful information and knowledge. Since the previous edition's publication, great advances have been made in the field of data mining. Not only does the third of edition of Data Mining: Concepts and Techniques continue the tradition of equipping you with an understanding and application of the theory and practice of discovering patterns hidden in large data sets, it also focuses on new, important topics in the field: data warehouses and data cube technology, mining stream, mining social networks, and mining spatial, multimedia and other complex data. Each chapter is a stand-alone guide to a critical topic, presenting proven algorithms and sound implementations ready to be used directly or with strategic modification against live data. This is the resource you need if you want to apply today's most powerful data mining techniques to meet real business challenges. * Presents dozens of algorithms and implementation examples, all in pseudo-code and suitable for use in real-world, large-scale data mining projects. * Addresses advanced topics such as mining object-relational databases, spatial databases, multimedia databases, time-series databases, text databases, the World Wide Web, and applications in several fields. *Provides a comprehensive, practical look at the concepts and techniques you need to get the most out of real business data

/pdf/data-mining-concepts-and-techniques-4dtvdfkvmi.pdf

Data Mining: Concepts and Techniques

From the Publisher:
This is the first comprehensive treatment of feed-forward neural networks from the perspective of statistical pattern recognition. After introducing the basic concepts, the book examines techniques for modelling probability density functions and the properties and merits of the multi-layer perceptron and radial basis function network models. Also covered are various forms of error functions, principal algorithms for error function minimalization, learning and generalization in neural networks, and Bayesian techniques and their applications. Designed as a text, with over 100 exercises, this fully up-to-date work will benefit anyone involved in the fields of neural computation and pattern recognition.

/pdf/neural-networks-for-pattern-recognition-34puigiau1.pdf

Neural networks for pattern recognition

A snake is an energy-minimizing spline guided by external constraint forces and influenced by image forces that pull it toward features such as lines and edges. Snakes are active contour models: they lock onto nearby edges, localizing them accurately. Scale-space continuation can be used to enlarge the capture region surrounding a feature. Snakes provide a unified account of a number of visual problems, including detection of edges, lines, and subjective contours; motion tracking; and stereo matching. We have used snakes successfully for interactive interpretation, in which user-imposed constraint forces guide the snake near features of interest.

/pdf/snakes-active-contour-models-2cuxgoykah.pdf

Snakes : Active Contour Models

This chapter contains sections titled: The Problem, The Generalized Delta Rule, Simulation Results, Some Further Generalizations, Conclusion

/pdf/learning-internal-representations-by-error-propagation-100fypr7vm.pdf

Learning internal representations by error propagation

GENERAL INTRODUCTION. Adaptive Systems. The Adaptive Linear Combiner. THEORY OF ADAPTATION WITH STATIONARY SIGNALS. Properties of the Quadratic Performance Surface. Searching the Performance Surface. Gradient Estimation and Its Effects on Adaptation. ADAPTIVE ALGORITHMS AND STRUCTURES. The LMS Algorithm. The Z-Transform in Adaptive Signal Processing. Other Adaptive Algorithms and Structures. Adaptive Lattice Filters. APPLICATIONS. Adaptive Modeling and System Identification. Inverse Adaptive Modeling, Deconvolution, and Equalization. Adaptive Control Systems. Adaptive Interference Cancelling. Introduction to Adaptive Arrays and Adaptive Beamforming. Analysis of Adaptive Beamformers.

Adaptive Signal Processing

This paper describes the concept of adaptive noise cancelling, an alternative method of estimating signals corrupted by additive noise or interference. The method uses a "primary" input containing the corrupted signal and a "reference" input containing noise correlated in some unknown way with the primary noise. The reference input is adaptively filtered and subtracted from the primary input to obtain the signal estimate. Adaptive filtering before subtraction allows the treatment of inputs that are deterministic or stochastic, stationary or time variable. Wiener solutions are developed to describe asymptotic adaptive performance and output signal-to-noise ratio for stationary stochastic inputs, including single and multiple reference inputs. These solutions show that when the reference input is free of signal and certain other conditions are met noise in the primary input can be essentiany eliminated without signal distortion. It is further shown that in treating periodic interference the adaptive noise canceller acts as a notch filter with narrow bandwidth, infinite null, and the capability of tracking the exact frequency of the interference; in this case the canceller behaves as a linear, time-invariant system, with the adaptive filter converging on a dynamic rather than a static solution. Experimental results are presented that illustrate the usefulness of the adaptive noise cancelling technique in a variety of practical applications. These applications include the cancelling of various forms of periodic interference in electrocardiography, the cancelling of periodic interference in speech signals, and the cancelling of broad-band interference in the side-lobes of an antenna array. In further experiments it is shown that a sine wave and Gaussian noise can be separated by using a reference input that is a delayed version of the primary input. Suggested applications include the elimination of tape hum or turntable rumble during the playback of recorded broad-band signals and the automatic detection of very-low-level periodic signals masked by broad-band noise.

/pdf/adaptive-noise-cancelling-principles-and-applications-1kc4rr7avw.pdf

Adaptive noise cancelling: Principles and applications

/pdf/adaptive-switching-circuits-30n2svf0c7.pdf

Adaptive switching circuits

Fundamental developments in feedforward artificial neural networks from the past thirty years are reviewed. The history, origination, operating characteristics, and basic theory of several supervised neural-network training algorithms (including the perceptron rule, the least-mean-square algorithm, three Madaline rules, and the backpropagation technique) are described. The concept underlying these iterative adaptation algorithms is the minimal disturbance principle, which suggests that during training it is advisable to inject new information into a network in a manner that disturbs stored information to the smallest extent possible. The two principal kinds of online rules that have developed for altering the weights of a network are examined for both single-threshold elements and multielement networks. They are error-correction rules, which alter the weights of a network to correct error in the output response to the present input pattern, and gradient rules, which alter the weights of a network during each pattern presentation by gradient descent with the objective of reducing mean-square error (averaged over all training patterns). >

/pdf/30-years-of-adaptive-neural-networks-perceptron-madaline-and-3p7ko3fnas.pdf

30 years of adaptive neural networks: perceptron, Madaline, and backpropagation

The authors describe how a two-layer neural network can approximate any nonlinear function by forming a union of piecewise linear segments. A method is given for picking initial weights for the network to decrease training time. The authors have used the method to initialize adaptive weights over a large number of different training problems and have achieved major improvements in learning speed in every case. The improvement is best when a large number of hidden units is used with a complicated desired response. The authors have used the method to train the truck-backer-upper and were able to decrease the training time from about two days to four hours

/pdf/improving-the-learning-speed-of-2-layer-neural-networks-by-3gxzyazgm2.pdf

Bernard Widrow

Papers

Adaptive Signal Processing

Adaptive noise cancelling: Principles and applications

Adaptive switching circuits

30 years of adaptive neural networks: perceptron, Madaline, and backpropagation

Improving the learning speed of 2-layer neural networks by choosing initial values of the adaptive weights