From the Publisher:
This is the first comprehensive treatment of feed-forward neural networks from the perspective of statistical pattern recognition. After introducing the basic concepts, the book examines techniques for modelling probability density functions and the properties and merits of the multi-layer perceptron and radial basis function network models. Also covered are various forms of error functions, principal algorithms for error function minimalization, learning and generalization in neural networks, and Bayesian techniques and their applications. Designed as a text, with over 100 exercises, this fully up-to-date work will benefit anyone involved in the fields of neural computation and pattern recognition.

/pdf/neural-networks-for-pattern-recognition-34puigiau1.pdf

Neural networks for pattern recognition

High-throughput sequencing assays such as RNA-Seq, ChIP-Seq or barcode counting provide quantitative readouts in the form of count data. To infer differential signal in such data correctly and with good statistical power, estimation of data variability throughout the dynamic range and a suitable error model are required. We propose a method based on the negative binomial distribution, with variance and mean linked by local regression and present an implementation, DESeq, as an R/Bioconductor package.

/pdf/differential-expression-analysis-for-sequence-count-data-1txizymire.pdf

Differential expression analysis for sequence count data.

Machine Learning is the study of methods for programming computers to learn. Computers are applied to a wide range of tasks, and for most of these it is relatively easy for programmers to design and implement the necessary software. However, there are many tasks for which this is difficult or impossible. These can be divided into four general categories. First, there are problems for which there exist no human experts. For example, in modern automated manufacturing facilities, there is a need to predict machine failures before they occur by analyzing sensor readings. Because the machines are new, there are no human experts who can be interviewed by a programmer to provide the knowledge necessary to build a computer system. A machine learning system can study recorded data and subsequent machine failures and learn prediction rules. Second, there are problems where human experts exist, but where they are unable to explain their expertise. This is the case in many perceptual tasks, such as speech recognition, hand-writing recognition, and natural language understanding. Virtually all humans exhibit expert-level abilities on these tasks, but none of them can describe the detailed steps that they follow as they perform them. Fortunately, humans can provide machines with examples of the inputs and correct outputs for these tasks, so machine learning algorithms can learn to map the inputs to the outputs. Third, there are problems where phenomena are changing rapidly. In finance, for example, people would like to predict the future behavior of the stock market, of consumer purchases, or of exchange rates. These behaviors change frequently, so that even if a programmer could construct a good predictive computer program, it would need to be rewritten frequently. A learning program can relieve the programmer of this burden by constantly modifying and tuning a set of learned prediction rules. Fourth, there are applications that need to be customized for each computer user separately. Consider, for example, a program to filter unwanted electronic mail messages. Different users will need different filters. It is unreasonable to expect each user to program his or her own rules, and it is infeasible to provide every user with a software engineer to keep the rules up-to-date. A machine learning system can learn which mail messages the user rejects and maintain the filtering rules automatically. Machine learning addresses many of the same research questions as the fields of statistics, data mining, and psychology, but with differences of emphasis. Statistics focuses on understanding the phenomena that have generated the data, often with the goal of testing different hypotheses about those phenomena. Data mining seeks to find patterns in the data that are understandable by people. Psychological studies of human learning aspire to understand the mechanisms underlying the various learning behaviors exhibited by people (concept learning, skill acquisition, strategy change, etc.).

Machine learning

A general non-parametric technique is proposed for the analysis of a complex multimodal feature space and to delineate arbitrarily shaped clusters in it. The basic computational module of the technique is an old pattern recognition procedure: the mean shift. For discrete data, we prove the convergence of a recursive mean shift procedure to the nearest stationary point of the underlying density function and, thus, its utility in detecting the modes of the density. The relation of the mean shift procedure to the Nadaraya-Watson estimator from kernel regression and the robust M-estimators; of location is also established. Algorithms for two low-level vision tasks discontinuity-preserving smoothing and image segmentation - are described as applications. In these algorithms, the only user-set parameter is the resolution of the analysis, and either gray-level or color images are accepted as input. Extensive experimental results illustrate their excellent performance.

Mean shift: a robust approach toward feature space analysis

Anomaly detection is an important problem that has been researched within diverse research areas and application domains. Many anomaly detection techniques have been specifically developed for certain application domains, while others are more generic. This survey tries to provide a structured and comprehensive overview of the research on anomaly detection. We have grouped existing techniques into different categories based on the underlying approach adopted by each technique. For each category we have identified key assumptions, which are used by the techniques to differentiate between normal and anomalous behavior. When applying a given technique to a particular domain, these assumptions can be used as guidelines to assess the effectiveness of the technique in that domain. For each category, we provide a basic anomaly detection technique, and then show how the different existing techniques in that category are variants of the basic technique. This template provides an easier and more succinct understanding of the techniques belonging to each category. Further, for each category, we identify the advantages and disadvantages of the techniques in that category. We also provide a discussion on the computational complexity of the techniques since it is an important issue in real application domains. We hope that this survey will provide a better understanding of the different directions in which research has been done on this topic, and how techniques developed in one area can be applied in domains for which they were not intended to begin with.

/pdf/anomaly-detection-a-survey-1x33g9m0a3.pdf

Anomaly detection: A survey

Automatic classification of seismic signals has been typically carried out on feature-based representations. Recent research works have shown that constructing classifiers on dissimilarity representations is a more practical and, sometimes, a more accurate solution for some pattern recognition problems. In this paper, we consider Bayesian classifiers constructed on dissimilarity representations. We show that such classifiers are a feasible and reliable alternative for automatic classification of seismic signals. Our experiments were conducted on a dataset containing seismic signals recorded by two selected stations of the monitoring network at Nevado del Ruiz Volcano. Dissimilarity representations were constructed by calculating pairwise Euclidean distances and a non-Euclidean measure on the normalized spectra, which is based on the difference in area between spectral curves. Results show that even though Euclidean dissimilarities have advantageous properties, non-Euclidean measures can be beneficial for matching spectra of seismic signals.

/pdf/dissimilarity-based-classification-of-seismic-signals-at-1hvsh6xntv.pdf

Dissimilarity-based classification of seismic signals at nevado del ruiz volcano

This article studies the possibility of detecting dementia in an early stage, using nonrigid registration of MR brain scans in combination with dissimilarity-based pattern recognition techniques. Instead of focussing on the shape of a single brain structure, we take into account the shape differences within the entire brain. Imaging data was obtained from a longitudinal, population based study of the elderly. A set of 29 subjects was identified, who were asymptomatic at the time of scanning, but were diagnosed as having dementia within 0.7 to 5 years after the scan, and a set of 29 age and gender matched healthy controls were selected. Each subject was registered to all other subjects, using a nonrigid registration algorithm. Based on statistics of the deformation field in the brain, a dissimilarity measure was calculated between each pair of subjects, yielding a 58×58 dissimilarity matrix. A kNN classifier was trained on the dissimilarity matrix and the performance was tested in a leave-one-out experiment. A classification accuracy of 81% was attained (spec. 83%, sens. 79%). This demonstrates the potential of whole-brain intersubject dissimilarities to aid in early diagnosis of dementia.

/pdf/early-diagnosis-of-dementia-based-on-intersubject-whole-42scwho7kd.pdf

Early diagnosis of dementia based on intersubject whole-brain dissimilarities

A flexible description of images is offered by a cloud of points in a feature space. In the context of image retrieval such clouds can be represented in a number of ways. Two approaches are here considered. The first approach is based on the assumption of a normal distribution, hence homogeneous clouds, while the second one focuses on the boundary description, which is more suitable for multimodal clouds. The images are then compared either by using the Mahalanobis distance or by the support vector data description (SVDD), respectively. The paper investigates some possibilities of combining the image clouds based on the idea that responses of several cloud descriptions may convey a pattern, specific for semantically similar images. A ranking of image dissimilarities is used as a comparison for two image databases targeting image classification and retrieval problems. We show that combining of the SVDD descriptions improves the retrieval performance with respect to ranking, on the contrary to the Mahalanobis case. Surprisingly, it turns out that the ranking of the Mahalanobis distances works well also for inhomogeneous images.

/pdf/a-study-on-combining-image-representations-for-image-1l2a6gtbki.pdf

A study on combining image representations for image classification and retrieval

Selective sampling, a part of the active learning method, reduces the cost of labeling supplementary training data by asking for the labels only of the most informative, unlabeled examples. This additional information added to an initial, randomly chosen training set is expected to improve the generalization performance of a learning machine. We investigate some methods for a selection of the most informative examples in the context of one-class classification problems (OCC) i.e. problems where only (or nearly only) the examples of the so-called target class are available. We applied selective sampling algorithms to a variety of domains, including realworld problems: mine detection and texture segmentation. The goal of this paper is to show why the best or most often used selective sampling methods for two- or multi-class problems are not necessarily the best ones for the one-class classification problem. By modifying the sampling methods, we present a way of selecting a small subset from the unlabeled data to be presented to an expert for labeling such that the performance of the retrained one-class classifier is significantly improved.

/pdf/uncertainty-sampling-methods-for-one-class-classifiers-ghpz8pejza.pdf

Uncertainty sampling methods for one-class classifiers

Automatic pattern recognition is usually considered as an engineering area which focusses on the development and evaluation of systems that imitate or assist humans in their ability of recognizing patterns. It may, however, also be considered as a science that studies the faculty of human beings (and possibly other biological systems) to discover, distinguish, characterize patterns in their environment and accordingly identify new observations. The engineering approach to pattern recognition is in this view an attempt to build systems that simulate this phenomenon. By doing that, scientific understanding is gained of what is needed in order to recognize patterns, in general.

/pdf/the-science-of-pattern-recognition-achievements-and-2ihm2g5cgu.pdf

Robert P. W. Duin

Papers

Dissimilarity-based classification of seismic signals at nevado del ruiz volcano

Early diagnosis of dementia based on intersubject whole-brain dissimilarities

A study on combining image representations for image classification and retrieval

Uncertainty sampling methods for one-class classifiers

The Science of Pattern Recognition. Achievements and Perspectives