Machine Learning is the study of methods for programming computers to learn. Computers are applied to a wide range of tasks, and for most of these it is relatively easy for programmers to design and implement the necessary software. However, there are many tasks for which this is difficult or impossible. These can be divided into four general categories. First, there are problems for which there exist no human experts. For example, in modern automated manufacturing facilities, there is a need to predict machine failures before they occur by analyzing sensor readings. Because the machines are new, there are no human experts who can be interviewed by a programmer to provide the knowledge necessary to build a computer system. A machine learning system can study recorded data and subsequent machine failures and learn prediction rules. Second, there are problems where human experts exist, but where they are unable to explain their expertise. This is the case in many perceptual tasks, such as speech recognition, hand-writing recognition, and natural language understanding. Virtually all humans exhibit expert-level abilities on these tasks, but none of them can describe the detailed steps that they follow as they perform them. Fortunately, humans can provide machines with examples of the inputs and correct outputs for these tasks, so machine learning algorithms can learn to map the inputs to the outputs. Third, there are problems where phenomena are changing rapidly. In finance, for example, people would like to predict the future behavior of the stock market, of consumer purchases, or of exchange rates. These behaviors change frequently, so that even if a programmer could construct a good predictive computer program, it would need to be rewritten frequently. A learning program can relieve the programmer of this burden by constantly modifying and tuning a set of learned prediction rules. Fourth, there are applications that need to be customized for each computer user separately. Consider, for example, a program to filter unwanted electronic mail messages. Different users will need different filters. It is unreasonable to expect each user to program his or her own rules, and it is infeasible to provide every user with a software engineer to keep the rules up-to-date. A machine learning system can learn which mail messages the user rejects and maintain the filtering rules automatically. Machine learning addresses many of the same research questions as the fields of statistics, data mining, and psychology, but with differences of emphasis. Statistics focuses on understanding the phenomena that have generated the data, often with the goal of testing different hypotheses about those phenomena. Data mining seeks to find patterns in the data that are understandable by people. Psychological studies of human learning aspire to understand the mechanisms underlying the various learning behaviors exhibited by people (concept learning, skill acquisition, strategy change, etc.).

Machine learning

The primary goal of pattern recognition is supervised or unsupervised classification. Among the various frameworks in which pattern recognition has been traditionally formulated, the statistical approach has been most intensively studied and used in practice. More recently, neural network techniques and methods imported from statistical learning theory have been receiving increasing attention. The design of a recognition system requires careful attention to the following issues: definition of pattern classes, sensing environment, pattern representation, feature extraction and selection, cluster analysis, classifier design and learning, selection of training and test samples, and performance evaluation. In spite of almost 50 years of research and development in this field, the general problem of recognizing complex patterns with arbitrary orientation, location, and scale remains unsolved. New and emerging applications, such as data mining, web searching, retrieval of multimedia data, face recognition, and cursive handwriting recognition, require robust and efficient pattern recognition techniques. The objective of this review paper is to summarize and compare some of the well-known methods used in various stages of a pattern recognition system and identify research topics and applications which are at the forefront of this exciting and challenging field.

/pdf/statistical-pattern-recognition-a-review-1kz1b0tte4.pdf

Statistical pattern recognition: a review

We develop a common theoretical framework for combining classifiers which use distinct pattern representations and show that many existing schemes can be considered as special cases of compound classification where all the pattern representations are used jointly to make a decision. An experimental comparison of various classifier combination schemes demonstrates that the combination rule developed under the most restrictive assumptions-the sum rule-outperforms other classifier combinations schemes. A sensitivity analysis of the various schemes to estimation errors is carried out to show that this finding can be justified theoretically.

On combining classifiers

From the Publisher:
Pattern recognition has long been studied in relation to many different (and mainly unrelated) applications, such as remote sensing, computer vision, space research, and medical imaging. In this book Professor Ripley brings together two crucial ideas in pattern recognition; statistical methods and machine learning via neural networks. Unifying principles are brought to the fore, and the author gives an overview of the state of the subject. Many examples are included to illustrate real problems in pattern recognition and how to overcome them.This is a self-contained account, ideal both as an introduction for non-specialists readers, and also as a handbook for the more expert reader.

Pattern recognition and neural networks

This paper proposes a new tree-based ensemble method for supervised classification and regression problems. It essentially consists of randomizing strongly both attribute and cut-point choice while splitting a tree node. In the extreme case, it builds totally randomized trees whose structures are independent of the output values of the learning sample. The strength of the randomization can be tuned to problem specifics by the appropriate choice of a parameter. We evaluate the robustness of the default choice of this parameter, and we also provide insight on how to adjust it in particular situations. Besides accuracy, the main strength of the resulting algorithm is computational efficiency. A bias/variance analysis of the Extra-Trees algorithm is also provided as well as a geometrical and a kernel characterization of the models induced.

/pdf/extremely-randomized-trees-20wkesw97q.pdf

Extremely randomized trees

In this paper we discuss applications of pattern recognition and image processing to automatic processing and analysis of histopathological images. We focus on two applications: counting of red and white blood cells using microscopic images of blood smear samples and breast cancer malignancy grading from slides of fine needle aspiration biopsies. We provide literature survey and point out new challenges.

Application of Pattern Recognition Techniques for the Analysis of Histopathological Images

In this article, we study the fixed-design regression estimation based on real and artificial data, where the artificial data comes from previously undertaken similar experiments. A least-squares estimate that gives different weights to the real and artificial data is introduced. It is investigated under which condition the rate of convergence of this estimate is better than the rate of convergence of an ordinary least-squares estimate applied to the real data only. The results are illustrated using simulated and real data.

Fixed-design regression estimation based on real and artificial data

This paper explores the supervised pattern recognition problem based on feature partitioning. This formulation leads to a new problem in computational geometry. The supervised pattern recognition problem is formulated as an heuristic good clique cover problem satisfying the k-nearest neighbors rule. First it is applied a heuristic algorithm for partitioning a graph into a minimal number of cliques. Next cliques are merged using the k-nearest neighbors rule. An important advantage of this approach is the decomposition of a problem involving l classes into l optimization problems involving a single class. The computational complexity of the method, computational procedures, and classification rules are discussed. A geometrical interpretation of the solution is also given. Using the proposed approach, the geometrical structure of the training set is utilized in the best possible way.

A new geometrical approach for solving the supervised pattern recognition problem

Nonlinear function learning using optimal radial basis function newtworks

Breast Cancer is one of the most deadly cancers affecting middle-aged women. Accurate diagnosis and prognosis
are crucial to reduce the high death rate. Nowadays there are numerous diagnostic tools for breast cancer
diagnosis. In this paper we discuss a role of nuclear segmentation from fine needle aspiration biopsy (FNA) slides
and its influence on malignancy classification. Classification of malignancy plays a very important role during
the diagnosis process of breast cancer. Out of all cancer diagnostic tools, FNA slides provide the most valuable
information about the cancer malignancy grade which helps to choose an appropriate treatment. This process
involves assessing numerous nuclear features and therefore precise segmentation of nuclei is very important.
In this work we compare three powerful segmentation approaches and test their impact on the classification of
breast cancer malignancy. The studied approaches involve level set segmentation, fuzzy c-means segmentation
and textural segmentation based on co-occurrence matrix.
Segmented nuclei were used to extract nuclear features for malignancy classification. For classification purposes
four different classifiers were trained and tested with previously extracted features. The compared classifiers
are Multilayer Perceptron (MLP), Self-Organizing Maps (SOM), Principal Component-based Neural Network
(PCA) and Support Vector Machines (SVM). The presented results show that level set segmentation yields the
best results over the three compared approaches and leads to a good feature extraction with a lowest average
error rate of 6.51% over four different classifiers. The best performance was recorded for multilayer perceptron
with an error rate of 3.07% using fuzzy c-means segmentation.

/pdf/influence-of-nuclei-segmentation-on-breast-cancer-malignancy-4co8lcyonp.pdf

Adam Krzyżak

Papers

Application of Pattern Recognition Techniques for the Analysis of Histopathological Images

Fixed-design regression estimation based on real and artificial data

A new geometrical approach for solving the supervised pattern recognition problem

Nonlinear function learning using optimal radial basis function newtworks

Influence of nuclei segmentation on breast cancer malignancy classification