Multilayer neural networks trained with the back-propagation algorithm constitute the best example of a successful gradient based learning technique. Given an appropriate network architecture, gradient-based learning algorithms can be used to synthesize a complex decision surface that can classify high-dimensional patterns, such as handwritten characters, with minimal preprocessing. This paper reviews various methods applied to handwritten character recognition and compares them on a standard handwritten digit recognition task. Convolutional neural networks, which are specifically designed to deal with the variability of 2D shapes, are shown to outperform all other techniques. Real-life document recognition systems are composed of multiple modules including field extraction, segmentation recognition, and language modeling. A new learning paradigm, called graph transformer networks (GTN), allows such multimodule systems to be trained globally using gradient-based methods so as to minimize an overall performance measure. Two systems for online handwriting recognition are described. Experiments demonstrate the advantage of global training, and the flexibility of graph transformer networks. A graph transformer network for reading a bank cheque is also described. It uses convolutional neural network character recognizers combined with global training techniques to provide record accuracy on business and personal cheques. It is deployed commercially and reads several million cheques per day.

Gradient-based learning applied to document recognition

Reinforcement learning, one of the most active research areas in artificial intelligence, is a computational approach to learning whereby an agent tries to maximize the total amount of reward it receives when interacting with a complex, uncertain environment. In Reinforcement Learning, Richard Sutton and Andrew Barto provide a clear and simple account of the key ideas and algorithms of reinforcement learning. Their discussion ranges from the history of the field's intellectual foundations to the most recent developments and applications. The only necessary mathematical background is familiarity with elementary concepts of probability. The book is divided into three parts. Part I defines the reinforcement learning problem in terms of Markov decision processes. Part II provides basic solution methods: dynamic programming, Monte Carlo methods, and temporal-difference learning. Part III presents a unified view of the solution methods and incorporates artificial neural networks, eligibility traces, and planning; the two final chapters present case studies and consider the future of reinforcement learning.

/pdf/reinforcement-learning-an-introduction-rzxgej9p17.pdf

Reinforcement Learning: An Introduction

There is, I think, something ethereal about i —the square root of minus one. I remember first hearing about it at school. It seemed an odd beast at that time—an intruder hovering on the edge of reality.

Usually familiarity dulls this sense of the bizarre, but in the case of i it was the reverse: over the years the sense of its surreal nature intensified. It seemed that it was impossible to write mathematics that described the real world in …

I and i

The architecture and learning procedure underlying ANFIS (adaptive-network-based fuzzy inference system) is presented, which is a fuzzy inference system implemented in the framework of adaptive networks. By using a hybrid learning procedure, the proposed ANFIS can construct an input-output mapping based on both human knowledge (in the form of fuzzy if-then rules) and stipulated input-output data pairs. In the simulation, the ANFIS architecture is employed to model nonlinear functions, identify nonlinear components on-line in a control system, and predict a chaotic time series, all yielding remarkable results. Comparisons with artificial neural networks and earlier work on fuzzy modeling are listed and discussed. Other extensions of the proposed ANFIS and promising applications to automatic control and signal processing are also suggested. >

ANFIS: adaptive-network-based fuzzy inference system

https://www2.econ.iastate.edu/tesfatsi/DeepLearningInNeuralNetworksOverview.JSchmidhuber2015.pdf

Deep learning in neural networks

Recently, a new approach to the adaptive control of linear time-invariant plants with unknown parameters (referred to as second level adaptation), was introduced by Han and Narendra in [1]. Based on $N (\geq m+1)$ fixed or adaptive models of the plant, where $m$ is the dimension of the unknown parameter vector, an unknown parameter vector $\alpha\in R^{N}$ is estimated in the new approach, and in turn, is used to control the overall system. Simulation studies were presented in [1] to demonstrate that the new method is significantly better than those that are currently in use. 
In this paper, we undertake a more detailed examination of the theoretical and practical advantages claimed for the new method. In particular, the need for many models, the proof of stability, and the improvement in performance and robustness are explored in depth both theoretically and experimentally.

The Rationale for Second Level Adaptation

The paper surveys the major developments in learning automata theory and applications in the last two decades. Since a survey article on the subject appeared in 1977, the emphasis is on subsequent developments. Of particular importance is some recent work on the use of many automata interacting in a decentralized manner. This framework provides a conceptual focus and an analytical basis for future research on modeling and control of complex systems. Applications of the theory are also reviewed, with special attention devoted to the problem of traffic routing in telecommunication networks.

Recent Developments in Learning Automata

The problem of determining whether or not a stable LTI dynamical system has a Lyapunov function V (x) = xTPx, with a diagonal matrix P > 0, referred to as a diagonal Lyapunov function, has attracted a great deal of attention in the past. The results obtained thus far are, for the most part, algebraic in nature. In this paper, using a geometric approach, and the concept of common Lyapunov functions, necessary and sufficient conditions for the existence of a diagonal Lyapunov function are given in terms of strict positive realness of a transfer function matrix.

Strict positive realness and the existence of diagonal Lyapunov functions

The paper discusses the stable adaptive control of linear discrete-time systems using multiple models. The principal contributions of the paper are the proofs of convergence of both the deterministic (noise free) and stochastic adaptive control problems.

Adaptive control of discrete-time systems using multiple models

A stochastic automaton with a variable structure (SAVS) may be described by the sextuple {X, Φ, α, p, U, G} where X={x0, x1,⋯,xk} is the input set, Φ = {Φ1, Φ2,⋯,Φs} is the internal state set, α = {α1, α1,⋯,αr} with r ≤ s is the output or action set, p is the state probability vector (i.e., at stage or time instant n, p(n) = (p1(n), p2(n),⋯,ps(n)) governing the choice of the state, U is an updating scheme which generates p(n + 1) from p(n) and G:Φ → α is the output function. In general, G may be a stochastic function. In this paper it is assumed that G is deterministic and one-to-one (i.e., r = s), k = 1 and s < ∞.

Kumpati S. Narendra

Papers

The Rationale for Second Level Adaptation

Recent Developments in Learning Automata

Strict positive realness and the existence of diagonal Lyapunov functions

Adaptive control of discrete-time systems using multiple models

On Variable-Structure Stochastic Automata