From the Publisher:
This is the first comprehensive treatment of feed-forward neural networks from the perspective of statistical pattern recognition. After introducing the basic concepts, the book examines techniques for modelling probability density functions and the properties and merits of the multi-layer perceptron and radial basis function network models. Also covered are various forms of error functions, principal algorithms for error function minimalization, learning and generalization in neural networks, and Bayesian techniques and their applications. Designed as a text, with over 100 exercises, this fully up-to-date work will benefit anyone involved in the fields of neural computation and pattern recognition.

/pdf/neural-networks-for-pattern-recognition-34puigiau1.pdf

Neural networks for pattern recognition

Publisher Summary This chapter provides an account of different neural network architectures for pattern recognition. A neural network consists of several simple processing elements called neurons. Each neuron is connected to some other neurons and possibly to the input nodes. Neural networks provide a simple computing paradigm to perform complex recognition tasks in real time. The chapter categorizes neural networks into three types: single-layer networks, multilayer feedforward networks, and feedback networks. It discusses the gradient descent and the relaxation method as the two underlying mathematical themes for deriving learning algorithms. A lot of research activity is centered on learning algorithms because of their fundamental importance in neural networks. The chapter discusses two important directions of research to improve learning algorithms: the dynamic node generation, which is used by the cascade correlation algorithm; and designing learning algorithms where the choice of parameters is not an issue. It closes with the discussion of performance and implementation issues.

Neural Networks for Pattern Recognition

Artificial neural net models have been studied for many years in the hope of achieving human-like performance in the fields of speech and image recognition. These models are composed of many nonlinear computational elements operating in parallel and arranged in patterns reminiscent of biological neural nets. Computational elements or nodes are connected via weights that are typically adapted during use to improve performance. There has been a recent resurgence in the field of artificial neural nets caused by new net topologies and algorithms, analog VLSI implementation techniques, and the belief that massive parallelism is essential for high performance speech and image recognition. This paper provides an introduction to the field of artificial neural nets by reviewing six important neural net models that can be used for pattern classification. These nets are highly parallel building blocks that illustrate neural net components and design principles and can be used to construct more complex systems. In addition to describing these nets, a major emphasis is placed on exploring how some existing classification and clustering algorithms can be performed using simple neuron-like components. Single-layer nets can implement algorithms required by Gaussian maximum-likelihood classifiers and optimum minimum-error classifiers for binary patterns corrupted by noise. More generally, the decision regions required by any classification algorithm can be generated in a straightforward manner by three-layer feed-forward nets.

An introduction to computing with neural nets

Multitask Learning is an approach to inductive transfer that improves generalization by using the domain information contained in the training signals of related tasks as an inductive bias. It does this by learning tasks in parallel while using a shared representation; what is learned for each task can help other tasks be learned better. This paper reviews prior work on MTL, presents new evidence that MTL in backprop nets discovers task relatedness without the need of supervisory signals, and presents new results for MTL with k-nearest neighbor and kernel regression. In this paper we demonstrate multitask learning in three domains. We explain how multitask learning works, and show that there are many opportunities for multitask learning in real domains. We present an algorithm and results for multitask learning with case-based methods like k-nearest neighbor and kernel regression, and sketch an algorithm for multitask learning in decision trees. Because multitask learning works, can be applied to many different kinds of domains, and can be used with different learning algorithms, we conjecture there will be many opportunities for its use on real-world problems.

Multitask Learning

1. Introduction and Overview. 2. Detecting Influential Observations and Outliers. 3. Detecting and Assessing Collinearity. 4. Applications and Remedies. 5. Research Issues and Directions for Extensions. Bibliography. Author Index. Subject Index.

Regression Diagnostics: Identifying Influential Data and Sources of Collinearity

Moment invariants are evaluated as a feature space for pattern recognition in terms of discrimination power and noise tolerance The notion of complex moments is introduced as a simple and straightforward way to derive moment invariants Through this relation, properties of complex moments are used to characterize moment invariants Aspects of information loss, suppression, and redundancy encountered in moment invariants are investigated and significant results are derived The behavior of moment invariants in the presence of additive noise is also described

/pdf/recognitive-aspects-of-moment-invariants-2x3dyvvr3z.pdf

Recognitive Aspects of Moment Invariants

This paper provides a brief introduction to forecasting in financial markets with emphasis on commodity futures and foreign exchange. We describe the basic approaches to forecasting, and discuss the noisy nature of financial data. Using neural networks as a learning paradigm, we describe different techniques for choosing the inputs, outputs, and error function. We also describe the learning from hints technique that augments the standard learning from examples method. We demonstrate the use of hints in foreign-exchange trading of the U.S. Dollar versus the British Pound, the German Mark, the Japanese Yen, and the Swiss Franc, over a period of 32 months. The paper does not assume a background in financial markets.

Introduction to financial forecasting

The information capacity of general forms of memory is formalized. The number of bits of information that can be stored in the Hopfield model of associative memory is estimated. It is found that the asymptotic information capacity of a Hopfield network of N neurons is of the order N^{3} b. The number of arbitrary state vectors that can be made stable in a Hopfield network of N neurons is proved to be bounded above by N .

/pdf/information-capacity-of-the-hopfield-model-jp4q25441n.pdf

Information capacity of the Hopfield model

Reference EPFL-ARTICLE-158542doi:10.1038/scientificamerican0387-88View record in Web of Science Record created on 2010-11-25, modified on 2017-05-10

Optical neural computers

The role of moments in image normalization and invariant pattern recognition is addressed. The classical idea of the principal axes is analyzed and extended to a more general definition. The relationship between moment-based normalization, moment invariants, and circular harmonics is established. Invariance properties of moments, as opposed to their recognition properties, are identified using a new class of normalization procedures. The application of moment-based normalization in pattern recognition is demonstrated by experiment.

/pdf/image-normalization-by-complex-moments-4i18lbrn5f.pdf

Yaser S. Abu-Mostafa

Papers

Recognitive Aspects of Moment Invariants

Introduction to financial forecasting

Information capacity of the Hopfield model

Optical neural computers

Image Normalization by Complex Moments