Scientists working with large volumes of high-dimensional data, such as global climate patterns, stellar spectra, or human gene distributions, regularly confront the problem of dimensionality reduction: finding meaningful low-dimensional structures hidden in their high-dimensional observations. The human brain confronts the same problem in everyday perception, extracting from its high-dimensional sensory inputs-30,000 auditory nerve fibers or 10(6) optic nerve fibers-a manageably small number of perceptually relevant features. Here we describe an approach to solving dimensionality reduction problems that uses easily measured local metric information to learn the underlying global geometry of a data set. Unlike classical techniques such as principal component analysis (PCA) and multidimensional scaling (MDS), our approach is capable of discovering the nonlinear degrees of freedom that underlie complex natural observations, such as human handwriting or images of a face under different viewing conditions. In contrast to previous algorithms for nonlinear dimensionality reduction, ours efficiently computes a globally optimal solution, and, for an important class of data manifolds, is guaranteed to converge asymptotically to the true structure.

/pdf/a-global-geometric-framework-for-nonlinear-dimensionality-5brl02iqtj.pdf

A global geometric framework for nonlinear dimensionality reduction.

We develop a face recognition algorithm which is insensitive to large variation in lighting direction and facial expression. Taking a pattern classification approach, we consider each pixel in an image as a coordinate in a high-dimensional space. We take advantage of the observation that the images of a particular face, under varying illumination but fixed pose, lie in a 3D linear subspace of the high dimensional image space-if the face is a Lambertian surface without shadowing. However, since faces are not truly Lambertian surfaces and do indeed produce self-shadowing, images will deviate from this linear subspace. Rather than explicitly modeling this deviation, we linearly project the image into a subspace in a manner which discounts those regions of the face with large deviation. Our projection method is based on Fisher's linear discriminant and produces well separated classes in a low-dimensional subspace, even under severe variation in lighting and facial expressions. The eigenface technique, another method based on linearly projecting the image space to a low dimensional subspace, has similar computational requirements. Yet, extensive experimental results demonstrate that the proposed "Fisherface" method has error rates that are lower than those of the eigenface technique for tests on the Harvard and Yale face databases.

/pdf/eigenfaces-vs-fisherfaces-recognition-using-class-specific-2qyevb22n0.pdf

Eigenfaces vs. Fisherfaces: recognition using class specific linear projection

Is perception of the whole based on perception of its parts? There is psychological and physiological evidence for parts-based representations in the brain, and certain computational theories of object recognition rely on such representations. But little is known about how brains or computers might learn the parts of objects. Here we demonstrate an algorithm for non-negative matrix factorization that is able to learn parts of faces and semantic features of text. This is in contrast to other methods, such as principal components analysis and vector quantization, that learn holistic, not parts-based, representations. Non-negative matrix factorization is distinguished from the other methods by its use of non-negativity constraints. These constraints lead to a parts-based representation because they allow only additive, not subtractive, combinations. When non-negative matrix factorization is implemented as a neural network, parts-based representations emerge by virtue of two properties: the firing rates of neurons are never negative and synaptic strengths do not change sign.

Learning the parts of objects by non-negative matrix factorization

We consider the problem of automatically recognizing human faces from frontal views with varying expression and illumination, as well as occlusion and disguise. We cast the recognition problem as one of classifying among multiple linear regression models and argue that new theory from sparse signal representation offers the key to addressing this problem. Based on a sparse representation computed by C1-minimization, we propose a general classification algorithm for (image-based) object recognition. This new framework provides new insights into two crucial issues in face recognition: feature extraction and robustness to occlusion. For feature extraction, we show that if sparsity in the recognition problem is properly harnessed, the choice of features is no longer critical. What is critical, however, is whether the number of features is sufficiently large and whether the sparse representation is correctly computed. Unconventional features such as downsampled images and random projections perform just as well as conventional features such as eigenfaces and Laplacianfaces, as long as the dimension of the feature space surpasses certain threshold, predicted by the theory of sparse representation. This framework can handle errors due to occlusion and corruption uniformly by exploiting the fact that these errors are often sparse with respect to the standard (pixel) basis. The theory of sparse representation helps predict how much occlusion the recognition algorithm can handle and how to choose the training images to maximize robustness to occlusion. We conduct extensive experiments on publicly available databases to verify the efficacy of the proposed algorithm and corroborate the above claims.

/pdf/robust-face-recognition-via-sparse-representation-4bfqv6y74o.pdf

Robust Face Recognition via Sparse Representation

Learning parts of objects by non-negative matrix factorization

An augmented reality system enhances a mobile user's situational awareness and provides new visualization functionality. The custom-built multimodal interface provides access to information encountered in urban environments. In this article, we detail our experiences with various input devices and modalities and discuss their advantages and drawbacks in the context of interaction tasks in mobile computing. We show how we integrated the input channels to use the modalities beneficially and how this enhances the interface's overall usability

Multimodal interaction with a wearable augmented reality system

Proposes an approach to 2D gesture recognition that models each gesture as a finite state machine (FSM) in the spatial-temporal space. The model construction works in a semi-automatic way. The structure of the model is first manually decided based on the observation of the spatial topology of the data. The model is refined iteratively between two stages: data segmentation and model training. We incorporate a modified Knuth-Morris-Pratt algorithm recognition procedure to speed up recognition. The computational efficiency of the FSM recognizers allows real-time online performance to be achieved.

/pdf/constructing-finite-state-machines-for-fast-gesture-3uyv4fsphk.pdf

Constructing finite state machines for fast gesture recognition

The human ability to process faces is remarkable. We can identify perhaps thousands offaces learned throughout our lifetime and read facial expression to understand such subtle qualities as emotion. These skills are quite robust, despite sometimes large changes in the visualstimulus due to expression, aging, and distractions such as glasses or changes in hairstyle or facialhair. Computers which model and recognize faces will be useful in a variety of applications,including criminal identification, human -computer interface, and animation. We discuss modelsfor representing faces and their applicability to the task of recognition, and present techniquesfor identifying faces and detecting eye blinks. 1 Introduction The human face, far beyond any other part of the body, communicates identity, emotion, race, andage, and also is quite useful in judging gender, size, and perhaps even character. The subject of visualprocessing of faces has received attention from philosophers and scientists for centuries. Aristotledevoted six chapters of the Historia Animalium to the study of facial appearance. Physiognomy,the practice or art of inferring intellectual or character qualities of a person from outward appear-ance, particularly the face, has had periods of fashion in various societies. Darwin considered facialexpression and its identification to be a significant advantage for the survival of species. Develop-mental studies have focused on strategies of recognition or identification and the differences betweeninfant and adult subjects. Neurological disorders of face perception have been isolated and studied,providing insight into normal as well as abnormal face processing. In recent years, computers havebeen introduced into various aspects of the subject, with systems attempting to model, display, andrecognize human faces.

Face Processing: Models For Recognition

We propose a novel formulation of minimal case solutions for determining the relative pose of perspective and generalized cameras given a partially known rotation, namely, a known axis of rotation. An axis of rotation may be easily obtained by detecting vertical vanishing points with computer vision techniques, or with the aid of sensor measurements from a smart phone. Given a known axis of rotation, our algorithms solve for the angle of rotation around the known axis along with the unknown translation. We formulate these relative pose problems as Quadratic Eigen value Problems which are very simple to construct. We run several experiments on synthetic and real data to compare our methods to the current state-of-the-art algorithms. Our methods provide several advantages over alternatives methods, including efficiency and accuracy, particularly in the presence of image and sensor noise as is often the case for mobile devices.

Solving for Relative Pose with a Partially Known Rotation is a Quadratic Eigenvalue Problem

Producing sports highlights is a labor-intensive work that requires some degree of specialization. We propose a model capable of automatically generating sports highlights with a focus on cricket. Cricket is a sport with a complex set of rules and is played for a longer time than most other sports. In this paper we propose a model that considers both event-based and excitement-based features to recognize and clip important events in a cricket match. Replays, audio intensity, player celebration, and playfield scenarios are examples of cues used to capture such events. To evaluate our framework, we conducted a set of experiments ranging from user studies to a comparison analysis between our highlights and the ones provided by the official broadcasters. The general approval by users and the significant overlap between both kinds of highlights indicate the usefulness of our model in real-life scenarios.

/pdf/automatic-cricket-highlight-generation-using-event-driven-kpjsb4b11a.pdf

Matthew Turk

Papers

Multimodal interaction with a wearable augmented reality system

Constructing finite state machines for fast gesture recognition

Face Processing: Models For Recognition

Solving for Relative Pose with a Partially Known Rotation is a Quadratic Eigenvalue Problem

Automatic Cricket Highlight Generation Using Event-Driven and Excitement-Based Features