Multilayer neural networks trained with the back-propagation algorithm constitute the best example of a successful gradient based learning technique. Given an appropriate network architecture, gradient-based learning algorithms can be used to synthesize a complex decision surface that can classify high-dimensional patterns, such as handwritten characters, with minimal preprocessing. This paper reviews various methods applied to handwritten character recognition and compares them on a standard handwritten digit recognition task. Convolutional neural networks, which are specifically designed to deal with the variability of 2D shapes, are shown to outperform all other techniques. Real-life document recognition systems are composed of multiple modules including field extraction, segmentation recognition, and language modeling. A new learning paradigm, called graph transformer networks (GTN), allows such multimodule systems to be trained globally using gradient-based methods so as to minimize an overall performance measure. Two systems for online handwriting recognition are described. Experiments demonstrate the advantage of global training, and the flexibility of graph transformer networks. A graph transformer network for reading a bank cheque is also described. It uses convolutional neural network character recognizers combined with global training techniques to provide record accuracy on business and personal cheques. It is deployed commercially and reads several million cheques per day.

Gradient-based learning applied to document recognition

Reinforcement learning, one of the most active research areas in artificial intelligence, is a computational approach to learning whereby an agent tries to maximize the total amount of reward it receives when interacting with a complex, uncertain environment. In Reinforcement Learning, Richard Sutton and Andrew Barto provide a clear and simple account of the key ideas and algorithms of reinforcement learning. Their discussion ranges from the history of the field's intellectual foundations to the most recent developments and applications. The only necessary mathematical background is familiarity with elementary concepts of probability. The book is divided into three parts. Part I defines the reinforcement learning problem in terms of Markov decision processes. Part II provides basic solution methods: dynamic programming, Monte Carlo methods, and temporal-difference learning. Part III presents a unified view of the solution methods and incorporates artificial neural networks, eligibility traces, and planning; the two final chapters present case studies and consider the future of reinforcement learning.

/pdf/reinforcement-learning-an-introduction-rzxgej9p17.pdf

Reinforcement Learning: An Introduction

There is, I think, something ethereal about i —the square root of minus one. I remember first hearing about it at school. It seemed an odd beast at that time—an intruder hovering on the edge of reality.

Usually familiarity dulls this sense of the bizarre, but in the case of i it was the reverse: over the years the sense of its surreal nature intensified. It seemed that it was impossible to write mathematics that described the real world in …

I and i

The architecture and learning procedure underlying ANFIS (adaptive-network-based fuzzy inference system) is presented, which is a fuzzy inference system implemented in the framework of adaptive networks. By using a hybrid learning procedure, the proposed ANFIS can construct an input-output mapping based on both human knowledge (in the form of fuzzy if-then rules) and stipulated input-output data pairs. In the simulation, the ANFIS architecture is employed to model nonlinear functions, identify nonlinear components on-line in a control system, and predict a chaotic time series, all yielding remarkable results. Comparisons with artificial neural networks and earlier work on fuzzy modeling are listed and discussed. Other extensions of the proposed ANFIS and promising applications to automatic control and signal processing are also suggested. >

ANFIS: adaptive-network-based fuzzy inference system

https://www2.econ.iastate.edu/tesfatsi/DeepLearningInNeuralNetworksOverview.JSchmidhuber2015.pdf

Deep learning in neural networks

Neural networks with different architectures have been used for the identification and control of a wide class of nonlinear systems. In the present work, the authors consider the problem of input disturbance rejection, when such networks are used in practical problems. A large class of disturbances, which can be modeled as the outputs of unforced linear or nonlinear dynamic systems, are treated. The objective is to determine the identification model and the control law to minimize the effect of the disturbance at the output. In all cases, the method used involves expansion of the state space of the disturbance-free plant in an attempt to eliminate the effect of the disturbance. Several stages of increasing complexity of the problem are discussed in detail. Two simulation studies based on the results discussed are presented. >

Disturbance rejection in nonlinear systems using neural networks

It was recently shown that adaptive laws, for the adjustment of control parameters in strictly decentralized control systems with unknown parameters, can be developed to assure stability, provided prior information is available concerning the objectives of the various subsystems. However, while this can be theoretically justified, practical simulation studies have shown that the transient responses of the subsystems will be generally prohibitively large. To overcome this problem, the authors have been studying the possibility of using limited communication between subsystems to improve the responses significantly. This paper discusses in detail the stability and performance questions in such systems, and demonstrates that substantial improvement in performance may be possible with very little communication at critical instants. First, the paper discusses the communication problem as a switching problem, in which the controllers switch between two different adaptive laws, when there is no communication and over intervals when communication takes place. It is shown using a common quadratic Lyapunov function, that the stability and performance issues can be decoupled. Next, by attaching a cost for communication between subsystems as well as the errors at the outputs, the problem is cast as an optimization problem. Extensive simulation studies on a network containing six subsystems are included to indicate the effect of intermittent communication on performance.

To communicate or not to communicate: A decision-theoretic approach to decentralized adaptive control

It is shown that in strictly decentralized adaptive control systems, it is theoretically possible to asymptotically track desired states (outputs) with zero error. The controller of each subsystem needs to know only the desired states (or outputs) of the other subsystems i.e. all the controllers share their prior information and cooperate implicitly.

Decentralized adaptive control

http://www.nt.ntnu.no/users/skoge/prost/proceedings/ifac11-proceedings/data/html/papers/2237.pdf

Adaptive Control Using Collective Information Obtained from Multiple Models

A new approach, described as second level adaptation, was introduced in [1] for the control of unknown linear time-invariant plants using multiple identification models. If θ
 p 
∈ R
 n 
, the unknown parameter vector of an LTI system lies in the convex hull P(t
 0 
) of (n+1) vectors θ
 i 
(t
 0 
) (initial values of adaptive vectors) in parameter space, it was shown that it lies also in the convex hull of θ
 i 
(t)(i = 1, 2, ..., n+1), of the adaptive parameters of the identification models. If the representations of the plants and models are in companion form, and all the state variables are accessible, simulation results were presented to demonstrate that the new method would result in much better performance than conventional adaptive control. In this paper, all aspects of second level adaptation are critically reviewed. Following this, an analysis of the stability and robustness of the approach is undertaken, and detailed reasons are provided for the observed improvement in performance. Due to space limitations, most of the results described in the paper pertain to plants in companion form, with all state variables accessible. Towards the end of the paper, an effort is made to indicate how the same concepts can be extended to more general cases. These include plants whose matrices are not in companion form, and systems in which only the input and the output of the plant are accessible. The details of the latter problems will be included in a forthcoming paper [7]. The authors believe that the two papers, together, will make a convincing case for the use of multiple models in adaptive control.

Kumpati S. Narendra

Papers

Disturbance rejection in nonlinear systems using neural networks

To communicate or not to communicate: A decision-theoretic approach to decentralized adaptive control

Decentralized adaptive control

Adaptive Control Using Collective Information Obtained from Multiple Models

Stability, robustness, and performance issues in second level adaptation