From the Publisher:
This is the first comprehensive treatment of feed-forward neural networks from the perspective of statistical pattern recognition. After introducing the basic concepts, the book examines techniques for modelling probability density functions and the properties and merits of the multi-layer perceptron and radial basis function network models. Also covered are various forms of error functions, principal algorithms for error function minimalization, learning and generalization in neural networks, and Bayesian techniques and their applications. Designed as a text, with over 100 exercises, this fully up-to-date work will benefit anyone involved in the fields of neural computation and pattern recognition.

/pdf/neural-networks-for-pattern-recognition-34puigiau1.pdf

Neural networks for pattern recognition

High-throughput sequencing assays such as RNA-Seq, ChIP-Seq or barcode counting provide quantitative readouts in the form of count data. To infer differential signal in such data correctly and with good statistical power, estimation of data variability throughout the dynamic range and a suitable error model are required. We propose a method based on the negative binomial distribution, with variance and mean linked by local regression and present an implementation, DESeq, as an R/Bioconductor package.

/pdf/differential-expression-analysis-for-sequence-count-data-1txizymire.pdf

Differential expression analysis for sequence count data.

Machine Learning is the study of methods for programming computers to learn. Computers are applied to a wide range of tasks, and for most of these it is relatively easy for programmers to design and implement the necessary software. However, there are many tasks for which this is difficult or impossible. These can be divided into four general categories. First, there are problems for which there exist no human experts. For example, in modern automated manufacturing facilities, there is a need to predict machine failures before they occur by analyzing sensor readings. Because the machines are new, there are no human experts who can be interviewed by a programmer to provide the knowledge necessary to build a computer system. A machine learning system can study recorded data and subsequent machine failures and learn prediction rules. Second, there are problems where human experts exist, but where they are unable to explain their expertise. This is the case in many perceptual tasks, such as speech recognition, hand-writing recognition, and natural language understanding. Virtually all humans exhibit expert-level abilities on these tasks, but none of them can describe the detailed steps that they follow as they perform them. Fortunately, humans can provide machines with examples of the inputs and correct outputs for these tasks, so machine learning algorithms can learn to map the inputs to the outputs. Third, there are problems where phenomena are changing rapidly. In finance, for example, people would like to predict the future behavior of the stock market, of consumer purchases, or of exchange rates. These behaviors change frequently, so that even if a programmer could construct a good predictive computer program, it would need to be rewritten frequently. A learning program can relieve the programmer of this burden by constantly modifying and tuning a set of learned prediction rules. Fourth, there are applications that need to be customized for each computer user separately. Consider, for example, a program to filter unwanted electronic mail messages. Different users will need different filters. It is unreasonable to expect each user to program his or her own rules, and it is infeasible to provide every user with a software engineer to keep the rules up-to-date. A machine learning system can learn which mail messages the user rejects and maintain the filtering rules automatically. Machine learning addresses many of the same research questions as the fields of statistics, data mining, and psychology, but with differences of emphasis. Statistics focuses on understanding the phenomena that have generated the data, often with the goal of testing different hypotheses about those phenomena. Data mining seeks to find patterns in the data that are understandable by people. Psychological studies of human learning aspire to understand the mechanisms underlying the various learning behaviors exhibited by people (concept learning, skill acquisition, strategy change, etc.).

Machine learning

A general non-parametric technique is proposed for the analysis of a complex multimodal feature space and to delineate arbitrarily shaped clusters in it. The basic computational module of the technique is an old pattern recognition procedure: the mean shift. For discrete data, we prove the convergence of a recursive mean shift procedure to the nearest stationary point of the underlying density function and, thus, its utility in detecting the modes of the density. The relation of the mean shift procedure to the Nadaraya-Watson estimator from kernel regression and the robust M-estimators; of location is also established. Algorithms for two low-level vision tasks discontinuity-preserving smoothing and image segmentation - are described as applications. In these algorithms, the only user-set parameter is the resolution of the analysis, and either gray-level or color images are accepted as input. Extensive experimental results illustrate their excellent performance.

Mean shift: a robust approach toward feature space analysis

Anomaly detection is an important problem that has been researched within diverse research areas and application domains. Many anomaly detection techniques have been specifically developed for certain application domains, while others are more generic. This survey tries to provide a structured and comprehensive overview of the research on anomaly detection. We have grouped existing techniques into different categories based on the underlying approach adopted by each technique. For each category we have identified key assumptions, which are used by the techniques to differentiate between normal and anomalous behavior. When applying a given technique to a particular domain, these assumptions can be used as guidelines to assess the effectiveness of the technique in that domain. For each category, we provide a basic anomaly detection technique, and then show how the different existing techniques in that category are variants of the basic technique. This template provides an easier and more succinct understanding of the techniques belonging to each category. Further, for each category, we identify the advantages and disadvantages of the techniques in that category. We also provide a discussion on the computational complexity of the techniques since it is an important issue in real application domains. We hope that this survey will provide a better understanding of the different directions in which research has been done on this topic, and how techniques developed in one area can be applied in domains for which they were not intended to begin with.

/pdf/anomaly-detection-a-survey-1x33g9m0a3.pdf

Anomaly detection: A survey

http://www.rduin.nl/papers/sp_11_3way.pdf

Classification of three-way data by the dissimilarity representation

A survey of the field of pattern recognition is presented in a manner broad enough not to be limited to the progress in recent years alone. In this review the emphasis is on the principal problems, methods and applications, and hence the omission of many details is unavoidable. Pattern recognition is discussed both intuitively and, more formally, as a many-to-one mapping. Several sources of variability, leading to many different possible representations of a certain pattern, are mentioned. All of these representations have to be classified as one pattern indicated by a certain class. An essential aspect of pattern recognition, relevant for this classification, is the selection of features by means of preprocessing the input data. Another essential aspect is the decision process based upon these features and leading to the classification. Several types of preprocessing are treated. Global and local transformations suitable for different types of filtering, decomposition and segmentation are indicated. A discussion of the three techniques used in pattern recognition, namely statistical, fuzzy and linguistic, is given.

https://iopscience.iop.org/article/10.1088/0034-4885/43/6/002/pdf

Progress report on pattern recognition

An industrial rock classification system is constructed and studied The local texture information in many image patches is extracted and classified The decisions made at the local level are fused to form the high-level decision on the image/rock as a whole The main difficulties of this application lay in significant variability and inhomogeneity of local textures caused by uneven rock surfaces and intrusions Therefore, an emphasis is paid to the derivation of informative representation of local texture and to robust classification algorithms The study focuses on the co-occurrence representation of texture comparing the two frequently used strategies, namely the approach based on Haralick features and methods utilizing directly the co-occurrence likelihoods Apart of maximum-likelihood (ML) classifiers also an alternative method is studied considering the likelihoods to prototypes as feature of a new space Unlike the ML methods, a classifier built in this space may leverage all training examples It is experimentally illustrated, that in the rock classification setup the methods directly using the co-occurrence estimates outperform the feature-based techniques

/pdf/improving-the-maximum-likelihood-co-occurrence-classifier-a-3terlp108z.pdf

Improving the maximum-likelihood co-occurrence classifier: a study on classification of inhomogeneous rock images

General dissimilarity-based learning approaches have been proposed for dissimilarity data sets (Pekalska et al., 2002). They arise in problems in which direct comparisons of objects are made, e.g. by computing pairwise distances between images, spectra, graphs or strings. In this paper, we study under which circumstances such dissimilarity-based techniques can be used for deriving classifiers in feature vector spaces. We show that such classifiers perform comparably or better than the nearest neighbor rule based either on the entire or condensed training set. Moreover, they can be beneficial for highly-overlapping classes and for non-normally distributed data sets, with categorical, mixed or otherwise difficult features

/pdf/dissimilarity-based-classification-for-vectorial-owyutavo9u.pdf

Dissimilarity-based classification for vectorial representations

Edge detection is well developed area of image analysis. Many various kinds of techniques were designed for one-channel images. Also, a considerable attention was paid to edge detection in color, multispectral, and hyperspectral images. However, there are still many open issues in edge detection in multichannel images. For example, even the definition of multichannel edge is rather empirical and is not well established. In this paper statistical pattern recognition methodology is used to approach the problem of edge detection by considering image pixels as points in a multidimensional feature space. Appropriate multivariate techniques are used to retrieve information which can be useful for edge detection. The proposed approaches were tested on the real-world data.

/pdf/edge-detection-in-hyperspectral-imaging-multivariate-xoo107j65m.pdf

Robert P. W. Duin

Papers

Classification of three-way data by the dissimilarity representation

Progress report on pattern recognition

Improving the maximum-likelihood co-occurrence classifier: a study on classification of inhomogeneous rock images

Dissimilarity-based classification for vectorial representations

Edge detection in hyperspectral imaging: multivariate statistical approaches