Reinforcement learning, one of the most active research areas in artificial intelligence, is a computational approach to learning whereby an agent tries to maximize the total amount of reward it receives when interacting with a complex, uncertain environment. In Reinforcement Learning, Richard Sutton and Andrew Barto provide a clear and simple account of the key ideas and algorithms of reinforcement learning. Their discussion ranges from the history of the field's intellectual foundations to the most recent developments and applications. The only necessary mathematical background is familiarity with elementary concepts of probability. The book is divided into three parts. Part I defines the reinforcement learning problem in terms of Markov decision processes. Part II provides basic solution methods: dynamic programming, Monte Carlo methods, and temporal-difference learning. Part III presents a unified view of the solution methods and incorporates artificial neural networks, eligibility traces, and planning; the two final chapters present case studies and consider the future of reinforcement learning.

/pdf/reinforcement-learning-an-introduction-rzxgej9p17.pdf

Reinforcement Learning: An Introduction

https://www2.econ.iastate.edu/tesfatsi/DeepLearningInNeuralNetworksOverview.JSchmidhuber2015.pdf

Deep learning in neural networks

This article presents a general class of associative reinforcement learning algorithms for connectionist networks containing stochastic units. These algorithms, called REINFORCE algorithms, are shown to make weight adjustments in a direction that lies along the gradient of expected reinforcement in both immediate-reinforcement tasks and certain limited forms of delayed-reinforcement tasks, and they do this without explicitly computing gradient estimates or even storing information from which such estimates could be computed. Specific examples of such algorithms are presented, some of which bear a close relationship to certain existing algorithms while others are novel but potentially interesting in their own right. Also given are results that show how such algorithms can be naturally integrated with backpropagation. We close with a brief discussion of a number of additional issues surrounding the use of such algorithms, including what is known about their limiting behaviors as well as further considerations that might be used to help develop similar but potentially more powerful reinforcement learning algorithms.

/pdf/simple-statistical-gradient-following-algorithms-for-39sidawxep.pdf

Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning

This paper surveys the field of reinforcement learning from a computer-science perspective. It is written to be accessible to researchers familiar with machine learning. Both the historical basis of the field and a broad selection of current work are summarized. Reinforcement learning is the problem faced by an agent that learns behavior through trial-and-error interactions with a dynamic environment. The work described here has a resemblance to work in psychology, but differs considerably in the details and in the use of the word "reinforcement." The paper discusses central issues of reinforcement learning, including trading off exploration and exploitation, establishing the foundations of the field via Markov decision theory, learning from delayed reinforcement, constructing empirical models to accelerate learning, making use of generalization and hierarchy, and coping with hidden state. It concludes with a survey of some implemented systems and an assessment of the practical utility of current methods for reinforcement learning.

/pdf/reinforcement-learning-a-survey-38lmyylg2y.pdf

Reinforcement learning: a survey

This paper surveys the field of reinforcement learning from a computer-science perspective. It is written to be accessible to researchers familiar with machine learning. Both the historical basis of the field and a broad selection of current work are summarized. Reinforcement learning is the problem faced by an agent that learns behavior through trial-and-error interactions with a dynamic environment. The work described here has a resemblance to work in psychology, but differs considerably in the details and in the use of the word ``reinforcement.'' The paper discusses central issues of reinforcement learning, including trading off exploration and exploitation, establishing the foundations of the field via Markov decision theory, learning from delayed reinforcement, constructing empirical models to accelerate learning, making use of generalization and hierarchy, and coping with hidden state. It concludes with a survey of some implemented systems and an assessment of the practical utility of current methods for reinforcement learning.

Reinforcement Learning: A Survey

A parallel stochastic algorithm for relaxation labeling is analyzed. For the case of symmetric compatibility functions, it is proved that time algorithm will always converge to a consistent labeling. >

/pdf/analysis-of-stochastic-automata-algorithm-for-relaxation-58bapbfui4.pdf

Analysis of stochastic automata algorithm for relaxation labelling

This paper considers a multi-person discrete game with random payoffs. The distribution of the random payoff is unknown to the players and further none of the players know the strategies or the actual moves of other players. A class of absolutely expedient learning algorithms for the game based on a decentralised team of Learning Automata is presented. These algorithms correspond, in some sense, to rational behaviour on the part of the players. All stable stationary points of the algorithm are shown to be Nash equilibria for the game. It is also shown that under some additional constraints on the game, the team will always converge to a Nash equilibrium.

/pdf/absolutely-expedient-algorithms-for-learning-nash-equilibria-29qhba6zx8.pdf

Absolutely expedient algorithms for learning Nash equilibria

A hypothesis is proposed that predicts regions where fixations are most likely to occur on a visual pattern viewed by a human test subject. A scheme is developed for evaluating a fixation probability density function for any given pattern, based on an assumed fixation probability density function for a pattern consisting of a single dot. Predictions of the hypothesis agree with experimental data available in the literature. When the hypothesis is used in conjunction with some assumptions, it also provides interpretations for some well-known geometrical illusions, such as the Muller-Lyer illusion. It is hoped that a study of this kind will aid in understanding some aspects of the mechanisms subserving human visual pattern recognition.

A Probabilistic Hypothesis for the Prediction of Visual Fixations

A frequency-domain criterion for the asymptotic stability-in-the-large of systems containing many non-linearities is derived in terms of the positive realness of the product of a diagonal multiplier matrix and the transfer function matrix of the linear part Several sub-classes of monotonically increasing non-linear functions are considered and it is shown that the elements of the multiplier matrix can be permitted to have complex conjugate poles and zeros whon the non-linearities possess at least a restricted odd asymmetry A Lyapunov function of the quadratic plus multi integral type and a matrix version of the Meyer—Kalman-Yakubovich lemma are used in deriving the results

Absolute Stability of Systems with Multiple Non-linearities†

The paper deals with the basic problem of adjusting a matrix gain in a discrete-time linear multivariable system The object is to obtain a global convergence criterion, ie conditions under which a specified error signal asymptotically approaches zero and other signals in the system remain bounded for arbitrary initial conditions and for any bounded input to the system It is shown that for a class of up-dating algorithms for the adjustable gain matrix, global convergence is crucially dependent on a transfer matrix G(z) which has a simple block diagram interpretation When w(z)G(z) is strictly discrete positive real for a scalar w(z) such that w-1(z) is strictly proper with poles and zeros within the unit circle, an augmented error scheme is suggested and is proved to result in global convergence The solution avoids feeding back a quadratic term as recommended in other schemes for single-input single-output systems

M. A. L. Thathachar

Papers

Analysis of stochastic automata algorithm for relaxation labelling

Absolutely expedient algorithms for learning Nash equilibria

A Probabilistic Hypothesis for the Prediction of Visual Fixations

Absolute Stability of Systems with Multiple Non-linearities†

A global convergence criterion for discrete-time multivariable adaptive systems