Reinforcement learning, one of the most active research areas in artificial intelligence, is a computational approach to learning whereby an agent tries to maximize the total amount of reward it receives when interacting with a complex, uncertain environment. In Reinforcement Learning, Richard Sutton and Andrew Barto provide a clear and simple account of the key ideas and algorithms of reinforcement learning. Their discussion ranges from the history of the field's intellectual foundations to the most recent developments and applications. The only necessary mathematical background is familiarity with elementary concepts of probability. The book is divided into three parts. Part I defines the reinforcement learning problem in terms of Markov decision processes. Part II provides basic solution methods: dynamic programming, Monte Carlo methods, and temporal-difference learning. Part III presents a unified view of the solution methods and incorporates artificial neural networks, eligibility traces, and planning; the two final chapters present case studies and consider the future of reinforcement learning.

/pdf/reinforcement-learning-an-introduction-rzxgej9p17.pdf

Reinforcement Learning: An Introduction

https://www2.econ.iastate.edu/tesfatsi/DeepLearningInNeuralNetworksOverview.JSchmidhuber2015.pdf

Deep learning in neural networks

This article presents a general class of associative reinforcement learning algorithms for connectionist networks containing stochastic units. These algorithms, called REINFORCE algorithms, are shown to make weight adjustments in a direction that lies along the gradient of expected reinforcement in both immediate-reinforcement tasks and certain limited forms of delayed-reinforcement tasks, and they do this without explicitly computing gradient estimates or even storing information from which such estimates could be computed. Specific examples of such algorithms are presented, some of which bear a close relationship to certain existing algorithms while others are novel but potentially interesting in their own right. Also given are results that show how such algorithms can be naturally integrated with backpropagation. We close with a brief discussion of a number of additional issues surrounding the use of such algorithms, including what is known about their limiting behaviors as well as further considerations that might be used to help develop similar but potentially more powerful reinforcement learning algorithms.

/pdf/simple-statistical-gradient-following-algorithms-for-39sidawxep.pdf

Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning

This paper surveys the field of reinforcement learning from a computer-science perspective. It is written to be accessible to researchers familiar with machine learning. Both the historical basis of the field and a broad selection of current work are summarized. Reinforcement learning is the problem faced by an agent that learns behavior through trial-and-error interactions with a dynamic environment. The work described here has a resemblance to work in psychology, but differs considerably in the details and in the use of the word "reinforcement." The paper discusses central issues of reinforcement learning, including trading off exploration and exploitation, establishing the foundations of the field via Markov decision theory, learning from delayed reinforcement, constructing empirical models to accelerate learning, making use of generalization and hierarchy, and coping with hidden state. It concludes with a survey of some implemented systems and an assessment of the practical utility of current methods for reinforcement learning.

/pdf/reinforcement-learning-a-survey-38lmyylg2y.pdf

Reinforcement learning: a survey

This paper surveys the field of reinforcement learning from a computer-science perspective. It is written to be accessible to researchers familiar with machine learning. Both the historical basis of the field and a broad selection of current work are summarized. Reinforcement learning is the problem faced by an agent that learns behavior through trial-and-error interactions with a dynamic environment. The work described here has a resemblance to work in psychology, but differs considerably in the details and in the use of the word ``reinforcement.'' The paper discusses central issues of reinforcement learning, including trading off exploration and exploitation, establishing the foundations of the field via Markov decision theory, learning from delayed reinforcement, constructing empirical models to accelerate learning, making use of generalization and hierarchy, and coping with hidden state. It concludes with a survey of some implemented systems and an assessment of the practical utility of current methods for reinforcement learning.

Reinforcement Learning: A Survey

From the combination of knowledge and actions, someone can improve their skill and ability. It will lead them to live and work much better. This is why, the students, workers, or even employers should have reading habit for books. Any book will give certain knowledge to take all benefits. This is what this learning automata an introduction tells you. It will add more knowledge of you to life and work better. Try it and prove it.

Learning Automata: An Introduction

Stochastic automata operating in an unknown random environment have been proposed earlier as models of learning. These automata update their action probabilities in accordance with the inputs received from the environment and can improve their own performance during operation. In this context they are referred to as learning automata. A survey of the available results in the area of learning automata has been attempted in this paper. Attention has been focused on the norms of behavior of learning automata, issues in the design of updating schemes, convergence of the action probabilities, and interaction of several automata. Utilization of learning automata in parameter optimization and hypothesis testing is discussed, and potential areas of application are suggested.

/pdf/learning-automata-a-survey-48tr2mi6qw.pdf

Learning Automata - A Survey

Automata models of learning systems introduced in the 1960s were popularized as learning automata (LA) in a survey paper by Narendra and Thathachar (1974). Since then, there have been many fundamental advances in the theory as well as applications of these learning models. In the past few years, the structure of LA, has been modified in several directions to suit different applications. Concepts such as parameterized learning automata (PLA), generalized learning,automata (GLA), and continuous action-set learning automata (CALA) have been proposed, analyzed, and applied to solve many significant learning problems. Furthermore, groups of LA forming teams and feedforward networks have been shown to converge to desired solutions under appropriate learning algorithms. Modules of LA have been used for parallel operation with consequent increase in speed of convergence. All of these concepts and results are relatively new and are scattered in technical literature. An attempt has been made in this paper to bring together the main ideas involved in a unified framework and provide pointers to relevant references.

/pdf/varieties-of-learning-automata-an-overview-1004lapuer.pdf

Varieties of learning automata: an overview

1 Introduction- 11 Machine Intelligence and Learning- 12 Learning Automata- 13 The Finite Action Learning Automaton (FALA)- 131 The Automaton- 132 The Random Environment- 133 Operation of FALA- 14 Some Classical Learning Algorithms- 141 Linear Reward-Inaction (LR?I) Algorithm- 142 Other Linear Algorithms- 143 Estimator Algorithms- 144 Simulation Results- 15 The Discretized Probability FALA- 151 DLR?I Algorithm- 152 Discretized Pursuit Algorithm- 16 The Continuous Action Learning Automaton (CALA)- 161 Analysis of the Algorithm- 162 Simulation Results- 163 Another Continuous Action Automaton- 17 The Generalized Learning Automaton (GLA)- 171 Learning Algorithm- 172 An Example- 18 The Parameterized Learning Automaton (PLA)- 181 Learning Algorithm- 19 Multiautomata Systems- 110 Supplementary Remarks- 2 Games of Learning Automata- 21 Introduction- 22 A Multiple Payoff Stochastic Game of Automata- 221 The Learning Algorithm- 23 Analysis of the Automata Game Algorithm- 231 Analysis of the Approximating ODE- 24 Game with Common Payoff- 25 Games of FALA- 251 Common Payoff Games of FALA- 252 Pursuit Algorithm for a Team of FALA- 253 Other Types of Games- 26 Common Payoff Games of CALA- 261 Stochastic Approximation Algorithms and CALA- 27 Applications- 271 System Identification- 272 Learning Conjunctive Concepts- 28 Discussion- 29 Supplementary Remarks- 3 Feedforward Networks- 31 Introduction- 32 Networks of FALA- 33 The Learning Model- 331 G-Environment- 332 The Network- 333 Network Operation- 34 The Learning Algorithm- 35 Analysis- 36 Extensions- 361 Other Network Structures- 362 Other Learning Algorithms- 37 Convergence to the Global Maximum- 371 The Network- 372 The Global Learning Algorithm- 373 Analysis of the Global Algorithm- 38 Networks of GLA- 39 Discussion- 310 Supplementary Remarks- 4 Learning Automata for Pattern Classification- 41 Introduction- 42 Pattern Recognition- 43 Common Payoff Game of Automata for PR- 431 Pattern Classification with FALA- 432 Pattern Classification with CALA- 433 Simulations- 44 Automata Network for Pattern Recognition- 441 Simulations- 442 Network of Automata for Learning Global Maximum- 45 Decision Tree Classifiers- 451 Learning Decision Trees using GLA and CALA- 452 Learning Piece-wise Linear Functions- 46 Discussion- 47 Supplementary Remarks- 5 Parallel Operation of Learning Automata- 51 Introduction- 52 Parallel Operation of FALA- 521 Analysis- 522 ?-optimality- 523 Speed of Convergence and Module Size- 524 Simulation Studies- 53 Parallel Operation of CALA- 54 Parallel Pursuit Algorithm- 541 Simulation Studies- 55 General Procedure- 56 Parallel Operation of Games of FALA- 561 Analysis- 562 Common Payoff Game- 57 Parallel Operation of Networks of FALA- 571 Analysis- 572 Modules of Parameterized Learning Automata (PLA)- 573 Modules of Generalized Learning Automata (GLA)- 574 Pattern Classification Example- 58 Discussion- 59 Supplementary Remarks- 6 Some Recent Applications- 61 Introduction- 62 Supervised Learning of Perceptual Organization in Computer Vision- 63 Distributed Control of Broadcast Communication Networks- 64O ther Applications- 65 Discussion- Epilogue- Appendices- A The ODE Approach to Analysis of Learning Algorithms- AI Introduction- A2 Derivation of the ODE Approximation- A21 Assumptions- A22 Analysis- A3 Approximating ODEs for Some Automata Algorithms- A32 The CALA Algorithm- A33 Automata Team Algorithms- A4 Relaxing the Assumptions- B Proofs of Convergence for Pursuit Algorithm- B1 Proof of Theorem 11- B2 Proof of Theorem 57- C Weak Convergence and SDE Approximations- CI Introduction- C2 Weak Convergence- C3 Convergence to SDE- C31 Application to Global Algorithms- C4 Convergence to ODE- References

Networks of Learning Automata: Techniques for Online Stochastic Optimization

A multi-person discrete game where the payoff after each play is stochastic is considered. The distribution of the random payoff is unknown to the players and further none of the players know the strategies or the actual moves of other players. A learning algorithm for the game based on a decentralized team of learning automata is presented. It is proved that all stable stationary points of the algorithm are Nash equilibria for the game. Two special cases of the game are also discussed, namely, game with common payoff and the relaxation labelling problem. The former has applications such as pattern recognition and the latter is a problem widely studied in computer vision. For the two special cases it is shown that the algorithm always converges to a desirable solution. >

/pdf/decentralized-learning-of-nash-equilibria-in-multi-person-2gv2m3qu3c.pdf

M. A. L. Thathachar

Papers

Learning Automata: An Introduction

Learning Automata - A Survey

Varieties of learning automata: an overview

Networks of Learning Automata: Techniques for Online Stochastic Optimization

Decentralized learning of Nash equilibria in multi-person stochastic games with incomplete information