Multilayer neural networks trained with the back-propagation algorithm constitute the best example of a successful gradient based learning technique. Given an appropriate network architecture, gradient-based learning algorithms can be used to synthesize a complex decision surface that can classify high-dimensional patterns, such as handwritten characters, with minimal preprocessing. This paper reviews various methods applied to handwritten character recognition and compares them on a standard handwritten digit recognition task. Convolutional neural networks, which are specifically designed to deal with the variability of 2D shapes, are shown to outperform all other techniques. Real-life document recognition systems are composed of multiple modules including field extraction, segmentation recognition, and language modeling. A new learning paradigm, called graph transformer networks (GTN), allows such multimodule systems to be trained globally using gradient-based methods so as to minimize an overall performance measure. Two systems for online handwriting recognition are described. Experiments demonstrate the advantage of global training, and the flexibility of graph transformer networks. A graph transformer network for reading a bank cheque is also described. It uses convolutional neural network character recognizers combined with global training techniques to provide record accuracy on business and personal cheques. It is deployed commercially and reads several million cheques per day.

Gradient-based learning applied to document recognition

Background on genetic algorithms, LISP, and genetic programming hierarchical problem-solving introduction to automatically-defined functions - the two-boxes problem problems that straddle the breakeven point for computational effort Boolean parity functions determining the architecture of the program the lawnmower problem the bumblebee problem the increasing benefits of ADFs as problems are scaled up finding an impulse response function artificial ant on the San Mateo trail obstacle-avoiding robot the minesweeper problem automatic discovery of detectors for letter recognition flushes and four-of-a-kinds in a pinochle deck introduction to biochemistry and molecular biology prediction of transmembrane domains in proteins prediction of omega loops in proteins lookahead version of the transmembrane problem evolutionary selection of the architecture of the program evolution of primitives and sufficiency evolutionary selection of terminals evolution of closure simultaneous evolution of architecture, primitive functions, terminals, sufficiency, and closure the role of representation and the lens effect Appendices: list of special symbols list of special functions list of type fonts default parameters computer implementation annotated bibliography of genetic programming electronic mailing list and public repository

Genetic Programming: On the Programming of Computers by Means of Natural Selection

Planning algorithms are impacting technical disciplines and industries around the world, including robotics, computer-aided design, manufacturing, computer graphics, aerospace applications, drug design, and protein folding. This coherent and comprehensive book unifies material from several sources, including robotics, control theory, artificial intelligence, and algorithms. The treatment is centered on robot motion planning but integrates material on planning in discrete spaces. A major part of the book is devoted to planning under uncertainty, including decision theory, Markov decision processes, and information spaces, which are the “configuration spaces” of all sensor-based planning problems. The last part of the book delves into planning under differential constraints that arise when automating the motions of virtually any mechanical system. Developed from courses taught by the author, the book is intended for students, engineers, and researchers in robotics, artificial intelligence, and control theory as well as computer graphics, algorithms, and computational biology.

Planning Algorithms: Introductory Material

Computational techniques involving contrast enhancement and noise filtering on two-dimensional image arrays are developed based on their local mean and variance. These algorithms are nonrecursive and do not require the use of any kind of transform. They share the same characteristics in that each pixel is processed independently. Consequently, this approach has an obvious advantage when used in real-time digital image processing applications and where a parallel processor can be used. For both the additive and multiplicative cases, the a priori mean and variance of each pixel is derived from its local mean and variance. Then, the minimum mean-square error estimator in its simplest form is applied to obtain the noise filtering algorithms. For multiplicative noise a statistical optimal linear approximation is made. Experimental results show that such an assumption yields a very effective filtering algorithm. Examples on images containing 256 × 256 pixels are given. Results show that in most cases the techniques developed in this paper are readily adaptable to real-time image processing.

Digital Image Enhancement and Noise Filtering by Use of Local Statistics

This paper presents a practical design perspective on multivariable feedback control problems. It reviews the basic issue-feedback design in the face of uncertainties-and generalizes known single-input, single-output (SISO) statements and constraints of the design problem to multiinput, multioutput (MIMO) cases. Two major MIMO design approaches are then evaluated in the context of these results.

/pdf/multivariable-feedback-design-concepts-for-a-classical-3ynzpf9dn5.pdf

Multivariable feedback design: Concepts for a classical/modern synthesis

Applied optimal control

A canonical parameter space is introduced for linear time-invariant, discrete-time systems design. All observable and controllable linear systems of a given order are shown to share one stability domain in this space. This invariance of the stability domain is shown to be fundamental to the solution of the gain output-feedback stabilizability problem and other significant linear systems design problems. The geometric properties of the stability domain are investigated. Its convex hull is shown to be a simplex. Our approach is compared to the D -decomposition, the root-locus, and other methods. Its advantages over these methods in attacking the problem of stabilizability by gain output-feedback and other important problems are discussed.

A canonical parameter space for linear systems design

A theory of observers for discrete-time linear systems with inaccessible inputs is developed utilizing a dynamical inverse/state observer structure. It is shown that without any assumption on the form of the inputs, an observer can be designed that asymptotically will track the state for a certain class of systems. Furthermore, a considerable saving in input sensors (hardware) can be accomplished by using software to implement the dynamical inverse. An illustrative example and recommendations for future research are included.

Observers for multivariable systems with inaccessible inputs

A predictive bandwidth management scheme and network architecture for real-time VBR traffic

It is widely recognized that the multi-hop strategy is the preferred scheme in WDM networks employing slow tuning optical transmitters (and/or receivers) and having relatively static traffic matrices. The essential idea is to tune the transmitters and/or receivers in the network so that an optimal configuration, which minimizes the number of hops that most of the traffic needs to make, is obtained. Earlier work in this area has concentrated on “skewed” traffic matrices consisting of point-to-point connections. We show that the problem is different when multipoint traffic, which is becoming increasingly important, is introduced. We present a heuristic algorithm for the optimal placement of nodes to satisfy traffic matrices that have multipoint connections. Two logical topologies, the bidirectional linear chain and the bidirectional ring, are the target topologies considered. They are evaluated with respect to three traffic models based on fixed, uniform, and geometric distribution of the number of destinations. Monte-Carlo simulation results for this algorithm are obtained for a large number of connections and are compared with the performance of the random placement algorithm. For the latter case, both simulation and analytical results, based on the theory of order statistics, are shown. The results demonstrate that the algorithm leads to substantial savings for node placement in both topologies.

James S. Meditch

Papers

Applied optimal control

A canonical parameter space for linear systems design

Observers for multivariable systems with inaccessible inputs

A predictive bandwidth management scheme and network architecture for real-time VBR traffic

Supporting Multipoint Connections in Multi-Hop WDM Optical Networks