Multilayer neural networks trained with the back-propagation algorithm constitute the best example of a successful gradient based learning technique. Given an appropriate network architecture, gradient-based learning algorithms can be used to synthesize a complex decision surface that can classify high-dimensional patterns, such as handwritten characters, with minimal preprocessing. This paper reviews various methods applied to handwritten character recognition and compares them on a standard handwritten digit recognition task. Convolutional neural networks, which are specifically designed to deal with the variability of 2D shapes, are shown to outperform all other techniques. Real-life document recognition systems are composed of multiple modules including field extraction, segmentation recognition, and language modeling. A new learning paradigm, called graph transformer networks (GTN), allows such multimodule systems to be trained globally using gradient-based methods so as to minimize an overall performance measure. Two systems for online handwriting recognition are described. Experiments demonstrate the advantage of global training, and the flexibility of graph transformer networks. A graph transformer network for reading a bank cheque is also described. It uses convolutional neural network character recognizers combined with global training techniques to provide record accuracy on business and personal cheques. It is deployed commercially and reads several million cheques per day.

Gradient-based learning applied to document recognition

Reinforcement learning, one of the most active research areas in artificial intelligence, is a computational approach to learning whereby an agent tries to maximize the total amount of reward it receives when interacting with a complex, uncertain environment. In Reinforcement Learning, Richard Sutton and Andrew Barto provide a clear and simple account of the key ideas and algorithms of reinforcement learning. Their discussion ranges from the history of the field's intellectual foundations to the most recent developments and applications. The only necessary mathematical background is familiarity with elementary concepts of probability. The book is divided into three parts. Part I defines the reinforcement learning problem in terms of Markov decision processes. Part II provides basic solution methods: dynamic programming, Monte Carlo methods, and temporal-difference learning. Part III presents a unified view of the solution methods and incorporates artificial neural networks, eligibility traces, and planning; the two final chapters present case studies and consider the future of reinforcement learning.

/pdf/reinforcement-learning-an-introduction-rzxgej9p17.pdf

Reinforcement Learning: An Introduction

There is, I think, something ethereal about i —the square root of minus one. I remember first hearing about it at school. It seemed an odd beast at that time—an intruder hovering on the edge of reality.

Usually familiarity dulls this sense of the bizarre, but in the case of i it was the reverse: over the years the sense of its surreal nature intensified. It seemed that it was impossible to write mathematics that described the real world in …

I and i

The architecture and learning procedure underlying ANFIS (adaptive-network-based fuzzy inference system) is presented, which is a fuzzy inference system implemented in the framework of adaptive networks. By using a hybrid learning procedure, the proposed ANFIS can construct an input-output mapping based on both human knowledge (in the form of fuzzy if-then rules) and stipulated input-output data pairs. In the simulation, the ANFIS architecture is employed to model nonlinear functions, identify nonlinear components on-line in a control system, and predict a chaotic time series, all yielding remarkable results. Comparisons with artificial neural networks and earlier work on fuzzy modeling are listed and discussed. Other extensions of the proposed ANFIS and promising applications to automatic control and signal processing are also suggested. >

ANFIS: adaptive-network-based fuzzy inference system

https://www2.econ.iastate.edu/tesfatsi/DeepLearningInNeuralNetworksOverview.JSchmidhuber2015.pdf

Deep learning in neural networks

The main contribution of this paper is the introduction of a new canonical form for an adaptive observer for a linear multivariable system. The adaptive observer simultaneously estimates the state and the parameters of the unknown plant and is shown to be globally asymptotically stable. The extension of the results to discrete systems and the application of the adaptive observer in the control of helicopter dynamics is also considered in detail.

Stable adaptive schemes for state estimation and identification of linear systems

This paper presents a unified theoretical framework for the identification and control of a nonlinear discrete-time dynamical system, in which the nonlinear system is represented explicitly as a sum of its linearized component and the residual nonlinear component referred to as a "higher order function." This representation substantially simplifies the procedure of applying the implicit function theorem to derive local properties of the nonlinear system, and reveals the role played by the linearized system in a more transparent form. Under the assumption that the linearized system is controllable and observable, it is shown that: 1) the nonlinear system is also controllable and observable in a local domain; 2) a feedback law exists to stabilize the nonlinear system locally; and 3) the nonlinear system can exactly track a constant or a periodic sequence locally, if its linearized system can do so. With some additional assumptions, the nonlinear system is shown to have a well-defined relative degree (delay) and zero-dynamics. If the zero-dynamics of the linearized system is asymptotically stable, so is that of the nonlinear one, and in such a case, a control law exists for the nonlinear system to asymptotically track an arbitrary reference signal exactly, in a neighborhood of the equilibrium state. The tracking can be achieved by using the state vector for feedback, or by using only the input and the output, in which case the nonlinear autoregressive moving-average (NARMA) model is established and utilized. These results are important for understanding the use of neural networks as identifiers and controllers for general nonlinear discrete-time dynamical systems.

Identification and control of a nonlinear discrete-time system based on its linearization: a unified framework

Neural networks with different architectures have been successfully used for the identification and control of a wide class of nonlinear systems. The problem of rejection of input disturbances, when such networks are used in practical problems is considered. A large class of disturbances, which can be modeled as the outputs of unforced linear or nonlinear dynamic systems, is treated. The objective is to determine the identification model and the control law to minimize the effect of the disturbance at the output. In all cases, the method used involves expansion of the state space of the disturbance-free plant in an attempt to eliminate the effect of the disturbance. Several stages of increasing complexity of the problem are discussed in detail. Theoretical justification is provided for the existence of solutions to the problem of complete rejection of the disturbance in special cases. This provides the rationale for using similar techniques in situations where such theoretical analysis is not available. >

Disturbance rejection in nonlinear systems using neural networks

The aim of this paper is to develop a theory of adaptive routing in telephone networks using learning methods. A mathematical model of the network with slow-learning algorithms distributed at various nodes is presented. The algorithms update the routing probabilities on the basis of network feedback information (like call blocking or completion) only. Convergence of the routing strategies is established. Two linear updating algorithms, under certain conditions, are shown to have desirable equilibrium behavior like load equalization and minimum blocking probability for the entire network.

A learning model for routing in telephone networks

A new procedure is presented for the identification of discrete linear multivariable systems. Using Lyapunov's direct method the asymptotic stability of the overall system is established when the inputs are sufficiently general. It is shown that the procedure may also be extended for the identification of a certain class of nonlinear systems.

Kumpati S. Narendra

Papers

Stable adaptive schemes for state estimation and identification of linear systems

Identification and control of a nonlinear discrete-time system based on its linearization: a unified framework

Disturbance rejection in nonlinear systems using neural networks

A learning model for routing in telephone networks

An identification procedure for discrete multivariable systems