Multilayer neural networks trained with the back-propagation algorithm constitute the best example of a successful gradient based learning technique. Given an appropriate network architecture, gradient-based learning algorithms can be used to synthesize a complex decision surface that can classify high-dimensional patterns, such as handwritten characters, with minimal preprocessing. This paper reviews various methods applied to handwritten character recognition and compares them on a standard handwritten digit recognition task. Convolutional neural networks, which are specifically designed to deal with the variability of 2D shapes, are shown to outperform all other techniques. Real-life document recognition systems are composed of multiple modules including field extraction, segmentation recognition, and language modeling. A new learning paradigm, called graph transformer networks (GTN), allows such multimodule systems to be trained globally using gradient-based methods so as to minimize an overall performance measure. Two systems for online handwriting recognition are described. Experiments demonstrate the advantage of global training, and the flexibility of graph transformer networks. A graph transformer network for reading a bank cheque is also described. It uses convolutional neural network character recognizers combined with global training techniques to provide record accuracy on business and personal cheques. It is deployed commercially and reads several million cheques per day.

Gradient-based learning applied to document recognition

Reinforcement learning, one of the most active research areas in artificial intelligence, is a computational approach to learning whereby an agent tries to maximize the total amount of reward it receives when interacting with a complex, uncertain environment. In Reinforcement Learning, Richard Sutton and Andrew Barto provide a clear and simple account of the key ideas and algorithms of reinforcement learning. Their discussion ranges from the history of the field's intellectual foundations to the most recent developments and applications. The only necessary mathematical background is familiarity with elementary concepts of probability. The book is divided into three parts. Part I defines the reinforcement learning problem in terms of Markov decision processes. Part II provides basic solution methods: dynamic programming, Monte Carlo methods, and temporal-difference learning. Part III presents a unified view of the solution methods and incorporates artificial neural networks, eligibility traces, and planning; the two final chapters present case studies and consider the future of reinforcement learning.

/pdf/reinforcement-learning-an-introduction-rzxgej9p17.pdf

Reinforcement Learning: An Introduction

There is, I think, something ethereal about i —the square root of minus one. I remember first hearing about it at school. It seemed an odd beast at that time—an intruder hovering on the edge of reality.

Usually familiarity dulls this sense of the bizarre, but in the case of i it was the reverse: over the years the sense of its surreal nature intensified. It seemed that it was impossible to write mathematics that described the real world in …

I and i

The architecture and learning procedure underlying ANFIS (adaptive-network-based fuzzy inference system) is presented, which is a fuzzy inference system implemented in the framework of adaptive networks. By using a hybrid learning procedure, the proposed ANFIS can construct an input-output mapping based on both human knowledge (in the form of fuzzy if-then rules) and stipulated input-output data pairs. In the simulation, the ANFIS architecture is employed to model nonlinear functions, identify nonlinear components on-line in a control system, and predict a chaotic time series, all yielding remarkable results. Comparisons with artificial neural networks and earlier work on fuzzy modeling are listed and discussed. Other extensions of the proposed ANFIS and promising applications to automatic control and signal processing are also suggested. >

ANFIS: adaptive-network-based fuzzy inference system

https://www2.econ.iastate.edu/tesfatsi/DeepLearningInNeuralNetworksOverview.JSchmidhuber2015.pdf

Deep learning in neural networks

The paper addresses two complementary problems. The first one is to derive a rigorous formulation of adaptive control when the parameters of the plant are time-varying. To make the problem tractable, the time-variation is approximated by a piecewise-constant function assuming a finite number N of unknown values. The instants at which the parameters switch are also unknown. A proof of stability is presented assuming that an algorithm for simultaneous identification of N values exists. In the second part of the paper, an algorithm is designed which causes N adaptive models to converge to N different locations in parameter space in a self-organizing fashion. Work has been in progress but the proof of convergence has defied solution for over three years. The paper discusses the theoretical difficulties encountered, solutions obtained in special cases, new mathematical tools that are being investigated, and potential applications of the approach.

Simultaneous Identification and Control of Time-Varying Systems

In a recent paper [1] the authors proposed a new methodology for the adaptive control of a linear time-invariant plant using multiple models which is significantly different from the “switching” and “switching and tuning” methods which have been in use for over a decade. Extensive simulation studies have also revealed that the performance using the new method is far superior to earlier methods. In this paper an attempt is made to extend the same concepts to the adaptive control of discrete-time systems. It is well known that the control of discrete-time systems is simpler than the control of their continuous-time counterparts, that they find wider application in practice, and that the proofs of stability are substantially simpler. Also, in many cases (e.g. periodic systems) discrete-time control may be possible when even the formulation of tractable problems in continuous-time is impossible. The objective of this paper is to examine how the methodology proposed differs in the two cases with regard to transparency of the principal concepts, and effectiveness in practical applications.

/pdf/discrete-time-adaptive-control-using-multiple-models-46nxnbke1c.pdf

Discrete-time adaptive control using multiple models

An algebraic characterization of necessary and sufficient conditions for the existence of a diagonal solution P>0 to the Lyapunov equation for third order matrices has been derived. In this paper we show an alternative way of obtaining the same conditions. This is based on necessary and sufficient conditions that we derive for the existence of a common diagonal solution to the Lyapunov equation for two stable matrices in R/sup 2/spl times/2/. The importance of the new approach is that it appears to be possible to extend it to determine a common diagonal solution P>0 to the Lyapunov equation for two matrices A,B /spl isin/ R/sup 3/spl times/3/ and even to more general cases.

Kumpati S. Narendra

Papers

Simultaneous Identification and Control of Time-Varying Systems

Discrete-time adaptive control using multiple models

On the existence of diagonal solutions to the Lyapunov equation for a third order system

Optimization of Dynamical Systems Containing Neural Networks

Stability Analysis of Nonlinear and Time-Varying Discrete Systems