Multilayer neural networks trained with the back-propagation algorithm constitute the best example of a successful gradient based learning technique. Given an appropriate network architecture, gradient-based learning algorithms can be used to synthesize a complex decision surface that can classify high-dimensional patterns, such as handwritten characters, with minimal preprocessing. This paper reviews various methods applied to handwritten character recognition and compares them on a standard handwritten digit recognition task. Convolutional neural networks, which are specifically designed to deal with the variability of 2D shapes, are shown to outperform all other techniques. Real-life document recognition systems are composed of multiple modules including field extraction, segmentation recognition, and language modeling. A new learning paradigm, called graph transformer networks (GTN), allows such multimodule systems to be trained globally using gradient-based methods so as to minimize an overall performance measure. Two systems for online handwriting recognition are described. Experiments demonstrate the advantage of global training, and the flexibility of graph transformer networks. A graph transformer network for reading a bank cheque is also described. It uses convolutional neural network character recognizers combined with global training techniques to provide record accuracy on business and personal cheques. It is deployed commercially and reads several million cheques per day.

Gradient-based learning applied to document recognition

Reinforcement learning, one of the most active research areas in artificial intelligence, is a computational approach to learning whereby an agent tries to maximize the total amount of reward it receives when interacting with a complex, uncertain environment. In Reinforcement Learning, Richard Sutton and Andrew Barto provide a clear and simple account of the key ideas and algorithms of reinforcement learning. Their discussion ranges from the history of the field's intellectual foundations to the most recent developments and applications. The only necessary mathematical background is familiarity with elementary concepts of probability. The book is divided into three parts. Part I defines the reinforcement learning problem in terms of Markov decision processes. Part II provides basic solution methods: dynamic programming, Monte Carlo methods, and temporal-difference learning. Part III presents a unified view of the solution methods and incorporates artificial neural networks, eligibility traces, and planning; the two final chapters present case studies and consider the future of reinforcement learning.

/pdf/reinforcement-learning-an-introduction-rzxgej9p17.pdf

Reinforcement Learning: An Introduction

There is, I think, something ethereal about i —the square root of minus one. I remember first hearing about it at school. It seemed an odd beast at that time—an intruder hovering on the edge of reality.

Usually familiarity dulls this sense of the bizarre, but in the case of i it was the reverse: over the years the sense of its surreal nature intensified. It seemed that it was impossible to write mathematics that described the real world in …

I and i

The architecture and learning procedure underlying ANFIS (adaptive-network-based fuzzy inference system) is presented, which is a fuzzy inference system implemented in the framework of adaptive networks. By using a hybrid learning procedure, the proposed ANFIS can construct an input-output mapping based on both human knowledge (in the form of fuzzy if-then rules) and stipulated input-output data pairs. In the simulation, the ANFIS architecture is employed to model nonlinear functions, identify nonlinear components on-line in a control system, and predict a chaotic time series, all yielding remarkable results. Comparisons with artificial neural networks and earlier work on fuzzy modeling are listed and discussed. Other extensions of the proposed ANFIS and promising applications to automatic control and signal processing are also suggested. >

ANFIS: adaptive-network-based fuzzy inference system

https://www2.econ.iastate.edu/tesfatsi/DeepLearningInNeuralNetworksOverview.JSchmidhuber2015.pdf

Deep learning in neural networks

In the above paper,1 an optimal method of self-optimization of certain system parameters using noisy binary-valued performance feedback is extended, without losing optimality, to situations with many-valued performance feedback. The effect of time-varying feedback mechanisms is briefly considered.

Comments on "Use of Stochastic Automata for Parameter Self-Optimization with Multimodal Performance Criteria"

: The paper presents a unified approach to the design of stable adaptive algorithms for hybrid, discrete and continuous systems which have two time scales. Hybrid error models are first analyzed in detail using two distinct approaches and the results are extended to discrete systems in which the parameters are adjusted at rates slower than those at which the systems operate. The algorithms developed when applied to the adaptive control problem are shown to result in global stability. Simulation results are included and reveal the dependence of the convergence rates of the various algorithms on the two time-scales.

Error Models for Stable Hybrid Adaptive Systems. Part II

The concepts related to multilevel control are investigated in the context of a greatly simplified hypothetical aircraft control problem. A general fault detection and control problem is considered. The problem deals with the case where a nonlinear plant is suitably parameterized and each of the different configurations contains an unknown parameter which lies in a compact interval. In this case, it is assumed that a nominal plant in each configuration can be identified off-line using neural networks. A stabilizing controller is assumed to exist for all the plants belonging to a configuration and was designed off-line using neural networks. The fault detection problem was carried out at the higher level by a neural network used as a pattern recognizer. The simulation results shown illustrate the operation of the entire system when a fault is assumed to occur. An adaptive controller was used online to compensate for the uncertainty in the parameters.<<ETX>>

http://ieeexplore.ieee.org/iel2/607/6597/00261280.pdf

Multilevel Control of Dynamical Systems Using Neural Networks

The paper briefly reviews the status of control theory and practice using neural networks, and discusses some of the questions that have to be addressed if neural networks are to be used in real-time control.

Neural networks for real-time control

In this paper, we present a globally stable adaptive control algorithm applicable to discrete-time single-input single-output linear time-invariant systems. Apriori knowledge of neither the sign of the system's high frequency gain nor an upper bound on its magnitude is needed to ensure global stability.

Kumpati S. Narendra

Papers

Comments on "Use of Stochastic Automata for Parameter Self-Optimization with Multimodal Performance Criteria"

Error Models for Stable Hybrid Adaptive Systems. Part II

Multilevel Control of Dynamical Systems Using Neural Networks

Neural networks for real-time control

Removing the high-frequency gain assumption in discrete adaptive control