There is, I think, something ethereal about i —the square root of minus one. I remember first hearing about it at school. It seemed an odd beast at that time—an intruder hovering on the edge of reality.

Usually familiarity dulls this sense of the bizarre, but in the case of i it was the reverse: over the years the sense of its surreal nature intensified. It seemed that it was impossible to write mathematics that described the real world in …

I and i

Machine Learning is the study of methods for programming computers to learn. Computers are applied to a wide range of tasks, and for most of these it is relatively easy for programmers to design and implement the necessary software. However, there are many tasks for which this is difficult or impossible. These can be divided into four general categories. First, there are problems for which there exist no human experts. For example, in modern automated manufacturing facilities, there is a need to predict machine failures before they occur by analyzing sensor readings. Because the machines are new, there are no human experts who can be interviewed by a programmer to provide the knowledge necessary to build a computer system. A machine learning system can study recorded data and subsequent machine failures and learn prediction rules. Second, there are problems where human experts exist, but where they are unable to explain their expertise. This is the case in many perceptual tasks, such as speech recognition, hand-writing recognition, and natural language understanding. Virtually all humans exhibit expert-level abilities on these tasks, but none of them can describe the detailed steps that they follow as they perform them. Fortunately, humans can provide machines with examples of the inputs and correct outputs for these tasks, so machine learning algorithms can learn to map the inputs to the outputs. Third, there are problems where phenomena are changing rapidly. In finance, for example, people would like to predict the future behavior of the stock market, of consumer purchases, or of exchange rates. These behaviors change frequently, so that even if a programmer could construct a good predictive computer program, it would need to be rewritten frequently. A learning program can relieve the programmer of this burden by constantly modifying and tuning a set of learned prediction rules. Fourth, there are applications that need to be customized for each computer user separately. Consider, for example, a program to filter unwanted electronic mail messages. Different users will need different filters. It is unreasonable to expect each user to program his or her own rules, and it is infeasible to provide every user with a software engineer to keep the rules up-to-date. A machine learning system can learn which mail messages the user rejects and maintain the filtering rules automatically. Machine learning addresses many of the same research questions as the fields of statistics, data mining, and psychology, but with differences of emphasis. Statistics focuses on understanding the phenomena that have generated the data, often with the goal of testing different hypotheses about those phenomena. Data mining seeks to find patterns in the data that are understandable by people. Psychological studies of human learning aspire to understand the mechanisms underlying the various learning behaviors exhibited by people (concept learning, skill acquisition, strategy change, etc.).

Machine learning

Data Mining - Concepts and Techniques.

We present a generative appearance-based method for recognizing human faces under variation in lighting and viewpoint. Our method exploits the fact that the set of images of an object in fixed pose, but under all possible illumination conditions, is a convex cone in the space of images. Using a small number of training images of each face taken with different lighting directions, the shape and albedo of the face can be reconstructed. In turn, this reconstruction serves as a generative model that can be used to render (or synthesize) images of the face under novel poses and illumination conditions. The pose space is then sampled and, for each pose, the corresponding illumination cone is approximated by a low-dimensional linear subspace whose basis vectors are estimated using the generative model. Our recognition algorithm assigns to a test image the identity of the closest approximated illumination cone. Test results show that the method performs almost without error, except on the most extreme lighting directions.

From few to many: illumination cone models for face recognition under variable lighting and pose

We present a neural network-based upright frontal face detection system. A retinally connected neural network examines small windows of an image and decides whether each window contains a face. The system arbitrates between multiple networks to improve performance over a single network. We present a straightforward procedure for aligning positive face examples for training. To collect negative examples, we use a bootstrap algorithm, which adds false detections into the training set as training progresses. This eliminates the difficult task of manually selecting nonface training examples, which must be chosen to span the entire space of nonface images. Simple heuristics, such as using the fact that faces rarely overlap in images, can further improve the accuracy. Comparisons with several other state-of-the-art face detection systems are presented, showing that our system has comparable performance in terms of detection and false-positive rates.

Neural network-based face detection

The pattern classification problem can be defined as one of assigning a label to a pattern of unknown class based on labelled prototype patterns. The method described in this paper is based on the following two ideas which appeal to our common sense: when the correctness of a classifier on a pattern x is in question, it is best to consider the performance of the same classifier on the patterns which are similar to x; and a classifier is usually accurate when the test pattern x falls close to the center of its class in feature space and prone to error when it falls near a class boundary.

http://ieeexplore.ieee.org/iel4/5773/15417/00715556.pdf

Improving classifier accuracy by simulating fuzzy boundaries between classes

Sequential pattern mining can prove to be very useful for predicating future activities, interpreting recurring phenomena, extracting similarities in a series of events, etc. For example, in the NASDAQ market, the problem of finding stocks whose closing prices are always a bout $β0 higher than or β1 times the stocks of a given company, reduces to linear pattern retrieval: given query X, find all sequences Y from the database S so that, Y = β0 + β1 X with confidence C.In this paper, we introduce a novel approach using the Simple Linear Regression (SLR) model to match and retrieve sequential patterns. We extend the one-dimensional R2 model to ER2 for multi-dimensional sequence matching. In addition, we present the SLR + FFT pruning technique to speed up data retrieval without incurring any false dismissal. Experimental results on both synthetic and real datasets show that the pruning ratio of SLR + FFT can be above 99%. Applying the retrieval technique to real stocks resulted in the discovery many interesting patterns, some of which are presented in the paper. Also, using ER2 as the similarity measure for on-line signature recognition yielded high accuracy.

Matching and retrieving sequential patterns using regression

This chapter presents a concept paper that describes methods to accelerate new materials discovery and optimization, by enabling faster recognition and use of important theoretical, computational, and experimental information aggregated from peer-reviewed and published materials-related scientific documents online. To obtain insights for the discovery of new materials and to study about existing materials, research and development scientists and engineers rely heavily on an ever-growing number of materials research publications, mostly available online, and that date back many decades. So, the major thrust of this concept paper is the use of technology to (i) extract “deep” meaning from a large corpus of relevant materials science documents; (ii) navigate, cluster, and present documents in a meaningful way; and (iii) evaluate and revise the materials-related query responses until the researchers are guided to their information destination. While the proposed methodology targets the interdisciplinary field of materials research, the tools to be developed can be generalized to enhance scientific discoveries and learning across a broad swathe of disciplines. The research will advance the machine-learning area of developing hierarchical, dynamic topic models to investigate trends in materials discovery over user-specified time periods. Also, the field of image-based document analysis will benefit tremendously from machine learning tools such as the use of deep belief networks for classification and text separation from document images. Developing an interactive visualization tool that can display modeling results from a large materials network perspective as well as a time-based perspective is an advancement in visualization studies.

Document Informatics for Scientific Learning and Accelerated Discovery

POSTER PAPER. This paper proposes an approach of indexing and retrieving degraded handwritten documents. We present a modified version of the popular Vector Model in information retrieval (IR). Our model incorporates top n candidates from a HR system into the scheme of calculating the term frequency (tf) and the inverted document frequency (idf). Standardized IR Tests show that the proposed approach outperforms the retrieval of ordinary HR text in terms of mean average precision (MAP) and R-Precision.

Indexing and retrieval of handwritten medical forms

Writer Identification can be seen as a multi-class learning problem where number of writers are different classes. One of the fundamental approaches to solve a multi-class problemis by breaking it into binary classification tasks. In this work weare proposing a generic approach for multi-class classification using an ensemble of binary classifiers. We assign a distributedoutput representation to each class in the form of codewords andan ensemble of binary classifiers is created where each classifierpredicts one bit of the codeword. Actual label is determined using Belief Propagation algorithm on a graph constructed from the code matrix. We have performed experiments on a new publiclyavailable IBM-UB-1 dataset for the task of writer identification to show the efficacy of our method.

Venu Govindaraju

Papers

Improving classifier accuracy by simulating fuzzy boundaries between classes

Matching and retrieving sequential patterns using regression

Document Informatics for Scientific Learning and Accelerated Discovery

Indexing and retrieval of handwritten medical forms

Multiclass Learning for Writer Identification Using Error-Correcting Codes