Multilayer neural networks trained with the back-propagation algorithm constitute the best example of a successful gradient based learning technique. Given an appropriate network architecture, gradient-based learning algorithms can be used to synthesize a complex decision surface that can classify high-dimensional patterns, such as handwritten characters, with minimal preprocessing. This paper reviews various methods applied to handwritten character recognition and compares them on a standard handwritten digit recognition task. Convolutional neural networks, which are specifically designed to deal with the variability of 2D shapes, are shown to outperform all other techniques. Real-life document recognition systems are composed of multiple modules including field extraction, segmentation recognition, and language modeling. A new learning paradigm, called graph transformer networks (GTN), allows such multimodule systems to be trained globally using gradient-based methods so as to minimize an overall performance measure. Two systems for online handwriting recognition are described. Experiments demonstrate the advantage of global training, and the flexibility of graph transformer networks. A graph transformer network for reading a bank cheque is also described. It uses convolutional neural network character recognizers combined with global training techniques to provide record accuracy on business and personal cheques. It is deployed commercially and reads several million cheques per day.

Gradient-based learning applied to document recognition

Reinforcement learning, one of the most active research areas in artificial intelligence, is a computational approach to learning whereby an agent tries to maximize the total amount of reward it receives when interacting with a complex, uncertain environment. In Reinforcement Learning, Richard Sutton and Andrew Barto provide a clear and simple account of the key ideas and algorithms of reinforcement learning. Their discussion ranges from the history of the field's intellectual foundations to the most recent developments and applications. The only necessary mathematical background is familiarity with elementary concepts of probability. The book is divided into three parts. Part I defines the reinforcement learning problem in terms of Markov decision processes. Part II provides basic solution methods: dynamic programming, Monte Carlo methods, and temporal-difference learning. Part III presents a unified view of the solution methods and incorporates artificial neural networks, eligibility traces, and planning; the two final chapters present case studies and consider the future of reinforcement learning.

/pdf/reinforcement-learning-an-introduction-rzxgej9p17.pdf

Reinforcement Learning: An Introduction

There is, I think, something ethereal about i —the square root of minus one. I remember first hearing about it at school. It seemed an odd beast at that time—an intruder hovering on the edge of reality.

Usually familiarity dulls this sense of the bizarre, but in the case of i it was the reverse: over the years the sense of its surreal nature intensified. It seemed that it was impossible to write mathematics that described the real world in …

I and i

The architecture and learning procedure underlying ANFIS (adaptive-network-based fuzzy inference system) is presented, which is a fuzzy inference system implemented in the framework of adaptive networks. By using a hybrid learning procedure, the proposed ANFIS can construct an input-output mapping based on both human knowledge (in the form of fuzzy if-then rules) and stipulated input-output data pairs. In the simulation, the ANFIS architecture is employed to model nonlinear functions, identify nonlinear components on-line in a control system, and predict a chaotic time series, all yielding remarkable results. Comparisons with artificial neural networks and earlier work on fuzzy modeling are listed and discussed. Other extensions of the proposed ANFIS and promising applications to automatic control and signal processing are also suggested. >

ANFIS: adaptive-network-based fuzzy inference system

https://www2.econ.iastate.edu/tesfatsi/DeepLearningInNeuralNetworksOverview.JSchmidhuber2015.pdf

Deep learning in neural networks

An asymptotically stable reference model plays a crucial role in the entire literature on adaptive systems. Given a plant with unknown parameters, the objective is to adapt the parameters of a controller so that the behavior of the controlled plant emulates that of the reference model in some sense. This paper addresses the following question, which is markedly different from that encountered in conventional adaptive control: “Can two or more unstable plants adaptively stabilize each other? ”. This is because neither system has a stable model to emulate, and each depends upon the other to stabilize itself. It is not surprising that this seemingly innocuous question has far reaching implications in widely different fields such as biology, psychology, economics and robotics, where it is found to arise frequently. If simple rules of adaptation were adequate to answer the above question, it would not merit much attention. However, preliminary investigations have revealed that such adaptation often does lead to instability. In fact the answer to the question is far from simple, and depends upon the assumptions made regarding the adaptive subsystems, and the manner in which they interact with each other. This has given rise to a vast spectrum of interesting questions. The objective of this paper is consequently not to provide an exhaustive set of answers, but merely to pose several problems in what the authors hope will be a new area of research, and to present preliminary results concerning some of them.

Unstable systems stabilizing each other through adaptation

The concept of using multiple models to cope with transients which arise in adaptive systems with large parametric uncertainties was introduced in the 1990s. Both switching between multiple fixed models, and switching and tuning between fixed and adaptive models was proposed, and the stability of the resulting schemes was established. In all cases, the number of models needed is generally large, and the models used do not cooperate in any real sense. It was recently shown by the authors that if it is known a priori that the unknown plant parameter vector lies in the convex hull of a set of adaptive model parameter vectors at the initial time, it will remain in the convex hull of the parameters even as they evolve with time [1]. Later, a stability result was derived in [2] which decouples the stability and performance issues. In this paper, a new concept of second level adaptation is introduced to develop different stable strategies which improve the performance of the overall system. Simulation results are provided to illustrate the effectiveness of the proposed scheme in a rapidly time-varying environment, and are shown to be far superior to existing schemes.

Second level adaptation using multiple models

The paper deals with the classical results of Routh and Hurwitz, Biehler and Kharitonov, concerning the stability of a linear time invariant differential equation. It is shown that these results follow directly from a Schwarz matrix representation of stable systems.

Classical Results on the Stability of Linear Time-Invariant Systems, and the Schwarz Form

Previous extensions of the stability criterion of Popov [1] for systems with a stable linear time-invariant plant $G(s)$ followed by a nonlinear time-varying gain $k(t)f( \cdot )$ entailed the definition of nonlinearity classes $[ f( \cdot ) \in N]$ and corresponding frequency domain multipliers $Z_N (s)$ Then, defining a measure of nonlinearity \[ F_{\min } \equiv \mathop {\min }\limits_x \{ \frac{{xf( x)}}{{\int_0^x {f( z )dz} }} \} \] stability is ensured if $Z(s) \in Z_N (s)$ exists, such that (i) $G(s)Z(s)$ is strictly positive real, (ii) $Z(s - \Lambda ) \in Z_N (s)$ and (iii)\[\frac{1}{k}\frac{{dk}}{{dt}}\leqq \Lambda F_{\min } \quad ( {\text{see [ 5 ]}})\]In this paper, the point by point requirement of (iii) is replaced by an integral criterion In many cases the constraints on $k(t)$ predicated by the integral inequality are substantially less strict than those of (iii) above As (iii) is a special case of the integral relation, the results are never more strictThe development here will be li

Kumpati S. Narendra

Papers

Unstable systems stabilizing each other through adaptation

Second level adaptation using multiple models

Classical Results on the Stability of Linear Time-Invariant Systems, and the Schwarz Form

The Corduneanu-Popov Approach to the Stability of Nonlinear Time-Varying Systems

A New Iterative Procedure for the Minimization of a Quadratic Form on a Convex Set