Multilayer neural networks trained with the back-propagation algorithm constitute the best example of a successful gradient based learning technique. Given an appropriate network architecture, gradient-based learning algorithms can be used to synthesize a complex decision surface that can classify high-dimensional patterns, such as handwritten characters, with minimal preprocessing. This paper reviews various methods applied to handwritten character recognition and compares them on a standard handwritten digit recognition task. Convolutional neural networks, which are specifically designed to deal with the variability of 2D shapes, are shown to outperform all other techniques. Real-life document recognition systems are composed of multiple modules including field extraction, segmentation recognition, and language modeling. A new learning paradigm, called graph transformer networks (GTN), allows such multimodule systems to be trained globally using gradient-based methods so as to minimize an overall performance measure. Two systems for online handwriting recognition are described. Experiments demonstrate the advantage of global training, and the flexibility of graph transformer networks. A graph transformer network for reading a bank cheque is also described. It uses convolutional neural network character recognizers combined with global training techniques to provide record accuracy on business and personal cheques. It is deployed commercially and reads several million cheques per day.

Gradient-based learning applied to document recognition

Reinforcement learning, one of the most active research areas in artificial intelligence, is a computational approach to learning whereby an agent tries to maximize the total amount of reward it receives when interacting with a complex, uncertain environment. In Reinforcement Learning, Richard Sutton and Andrew Barto provide a clear and simple account of the key ideas and algorithms of reinforcement learning. Their discussion ranges from the history of the field's intellectual foundations to the most recent developments and applications. The only necessary mathematical background is familiarity with elementary concepts of probability. The book is divided into three parts. Part I defines the reinforcement learning problem in terms of Markov decision processes. Part II provides basic solution methods: dynamic programming, Monte Carlo methods, and temporal-difference learning. Part III presents a unified view of the solution methods and incorporates artificial neural networks, eligibility traces, and planning; the two final chapters present case studies and consider the future of reinforcement learning.

/pdf/reinforcement-learning-an-introduction-rzxgej9p17.pdf

Reinforcement Learning: An Introduction

There is, I think, something ethereal about i —the square root of minus one. I remember first hearing about it at school. It seemed an odd beast at that time—an intruder hovering on the edge of reality.

Usually familiarity dulls this sense of the bizarre, but in the case of i it was the reverse: over the years the sense of its surreal nature intensified. It seemed that it was impossible to write mathematics that described the real world in …

I and i

The architecture and learning procedure underlying ANFIS (adaptive-network-based fuzzy inference system) is presented, which is a fuzzy inference system implemented in the framework of adaptive networks. By using a hybrid learning procedure, the proposed ANFIS can construct an input-output mapping based on both human knowledge (in the form of fuzzy if-then rules) and stipulated input-output data pairs. In the simulation, the ANFIS architecture is employed to model nonlinear functions, identify nonlinear components on-line in a control system, and predict a chaotic time series, all yielding remarkable results. Comparisons with artificial neural networks and earlier work on fuzzy modeling are listed and discussed. Other extensions of the proposed ANFIS and promising applications to automatic control and signal processing are also suggested. >

ANFIS: adaptive-network-based fuzzy inference system

https://www2.econ.iastate.edu/tesfatsi/DeepLearningInNeuralNetworksOverview.JSchmidhuber2015.pdf

Deep learning in neural networks

SUMMARY The paper discusses in detail a new method for indirect model reference adaptive control (MRAC) of linear time-invariant continuous-time plants with unknown parameters. The method involves not only dynamic adjustment of plant parameter estimates but also those of the controller parameters. Hence the overall system can be described by a set of non-linear differential equations as in the case of direct control. Many of the difficulties encountered in the conventional indirect approach, where an algebraic equation is solved to determine the control parameters, are consequently bypassed in this method. The proof of stability of the equilibrium state of the overall system is found to be different from that used in direct control. Using Lyapunov’s theory, it is first shown that the parameter errors between the parameter estimates of the identifier and the true parameters of the plant, as well as those between the actual parameters of the controller and their desired values, are bounded. Following this, using growth rates of signals in the adaptive loop as well as order arguments, it is shown that the error equations are globally uniformly stable and that the tracking (control) error tends to zero asymptotically. This in turn establishes the fact that both direct and indirect model reference adaptive schemes require the same amount of prior information to achieve stable adaptive control.

/pdf/indirect-model-reference-adaptive-control-with-dynamic-22k8liaflz.pdf

Indirect model reference adaptive control with dynamic adjustment of parameters

Direct methods and indirect methods for the adaptive control of linear plants with unknown parameters are currently well known. Recently methods for combining direct and indirect methods have also been suggested. In this paper the latter method is combined with the variable structure method, which is also well established. New adaptive laws are suggested which overcome the principal drawbacks of the variable structure method. While the direct and indirect components of the new method assure asymptotic stability, the variable structure component improves the transient response of the system. Simulation results are presented to show that the overall system is asymptotically stable in the ideal case, and exhibits robustness and improved performance in the presence of different classes of perturbations.

A Combined Direct, Indirect and Variable Structure Method For Robust Adaptive Control

In this paper we examine the problem of control of multivariable systems using neural networks. The problem is discussed assuming different amounts of prior information concerning the plant and hence different levels of complexity. In the first stage it is assumed that the state equations describing the plant are known and that the state of the system is accessible. Following this the same problem is considered when the state equations are unknown. In the last stage the adaptive control of the multivariable system using only input-output data is discussed in detail. The objective of the paper is to demonstrate that results from nonlinear control theory and linear adaptive control theory can be used to design practically viable controllers for unknown nonlinear multivariable systems using neural networks. The different assumptions that have to be made, the choice of identifier and controller architectures and the generation of adaptive laws for the adjustment of the parameters of the neural networks form the core of the paper. >

Adaptive control of nonlinear multivariable systems using neural networks

In this paper, a general framework for the identification of discrete-time time-varying system is proposed. To make the problem tractable, the time variation is approximated by a piecewise constant function assuming finite number N of unknown values. The algorithms proposed would be applicable for switched system with linear and nonlinear subsystems. It also facilitates the derivation of both offline and online parameter identification algorithms. Extensive simulation studies show that our algorithm can indeed provide accurate estimates of the plant parameters even in noisy cases. A preliminary convergence analysis is also available, which we would further develop in the near future.

A general framework for least-squares based identification of time-varying system using multiple models

: The main aim of the report is to make transparent most of the interesting ideas involved in the design of adaptive systems using the direct method of Lyapunov, by considering primarily systems described by first-order differential equations. A study is made of three adaptive designs for such systems. (Author)

Kumpati S. Narendra

Papers

Indirect model reference adaptive control with dynamic adjustment of parameters

A Combined Direct, Indirect and Variable Structure Method For Robust Adaptive Control

Adaptive control of nonlinear multivariable systems using neural networks

A general framework for least-squares based identification of time-varying system using multiple models

Adaptive Control Using Lyapunov's Direct Method.