Showing papers on "Mahalanobis distance published in 2009"

Distance Metric Learning for Large Margin Nearest Neighbor Classification

Abstract: The accuracy of k-nearest neighbor (kNN) classification depends significantly on the metric used to compute distances between different examples. In this paper, we show how to learn a Mahalanobis distance metric for kNN classification from labeled examples. The Mahalanobis metric can equivalently be viewed as a global linear transformation of the input space that precedes kNN classification using Euclidean distances. In our approach, the metric is trained with the goal that the k-nearest neighbors always belong to the same class while examples from different classes are separated by a large margin. As in support vector machines (SVMs), the margin criterion leads to a convex optimization based on the hinge loss. Unlike learning in SVMs, however, our approach requires no modification or extension for problems in multiway (as opposed to binary) classification. In our framework, the Mahalanobis distance metric is obtained as the solution to a semidefinite program. On several data sets of varying size and difficulty, we find that metrics trained in this way lead to significant improvements in kNN classification. Sometimes these results can be further improved by clustering the training examples and learning an individual metric within each cluster. We show how to learn and combine these local metrics in a globally integrated manner.

Fast Similarity Search for Learned Metrics

Abstract: We introduce a method that enables scalable similarity search for learned metrics. Given pairwise similarity and dissimilarity constraints between some examples, we learn a Mahalanobis distance function that captures the examples' underlying relationships well. To allow sublinear time similarity search under the learned metric, we show how to encode the learned metric parameterization into randomized locality-sensitive hash functions. We further formulate an indirect solution that enables metric learning and hashing for vector spaces whose high dimensionality makes it infeasible to learn an explicit transformation over the feature dimensions. We demonstrate the approach applied to a variety of image data sets, as well as a systems data set. The learned metrics improve accuracy relative to commonly used metric baselines, while our hashing construction enables efficient indexing with learned distances and very large databases.

Finding an unknown number of multivariate outliers

Abstract: We use the forward search to provide robust Mahalanobis distances to detect the presence of outliers in a sample of multivariate normal data. Theoretical results on order statistics and on estimation in truncated samples provide the distribution of our test statistic. We also introduce several new robust distances with associated distributional results. Comparisons of our procedure with tests using other robust Mahalanobis distances show the good size and high power of our procedure. We also provide a unification of results on correction factors for estimation from truncated samples.

Random Hypersurface Models for extended object tracking

Abstract: Target tracking algorithms usually assume that the received measurements stem from a point source. However, in many scenarios this assumption is not feasible so that measurements may stem from different locations, named measurement sources, on the target surface. Then, it is necessary to incorporate the target extent into the estimation procedure in order to obtain robust and precise estimation results. This paper introduces the novel concept of Random Hypersurface Models for extended targets. A Random Hypersurface Model assumes that each measurement source is an element of a randomly generated hypersurface. The applicability of this approach is demonstrated by means of an elliptic target shape. In this case, a Random Hypersurface Model specifies the random (relative) Mahalanobis distance of a measurement source to the center of the target object. As a consequence, good estimation results can be obtained even if the true target shape significantly differs from the modeled shape. Additionally, Random Hypersurface Models are computationally tractable with standard nonlinear stochastic state estimators.

Detection of outliers

Abstract: We present an overview of the major developments in the area of detection of outliers These include projection pursuit approaches as well as Mahalanobis distance-based procedures We also discuss principal component-based methods, since these are most applicable to the large datasets that have become more prevalent in recent years The major algorithms within each category are briefly discussed, together with current challenges and possible directions of future research Copyright © 2009 John Wiley & Sons, Inc For further resources related to this article, please visit the WIREs website

Clustering for Metric and Non-Metric Distance Measures ⁄ (full version)

Abstract: We study a generalization of the k-median problem with respect to an arbitrary dissimilarity measure D. Given a finite set P of size n, our goal is to find a set C of size k such that the sum of errors D(P,C) = P p2P minc2C ' D(p,c) “ is minimized. The main result in this paper can be stated as follows: There exists a (1+†)-approximation algorithm for the k-median problem with respect to D, if the 1-median problem can be approximated within a factor of (1+†) by taking a random sample of constant size and solving the 1-median problem on the sample exactly. This algorithms requires time n2 O(mk log(mk/†)) , where m is a constant that depends only on † and D. Using this characterization, we obtain the first linear time (1+†)-approximation algorithms for the k-median problem in an arbitrary metric space with bounded doubling dimension, for the Kullback-Leibler divergence (relative entropy), for the Itakura-Saito divergence, for Mahalanobis distances, and for some special cases of Bregman divergences. Moreover, we obtain previously known results for the Euclidean k-median problem and the Euclidean k-means problem in a simplified manner. Our results are based on a new analysis of an algorithm of Kumar, Sabharwal, and Sen from FOCS 2004.

Statistical Computing on Manifolds: From Riemannian Geometry to Computational Anatomy

Abstract: Computational anatomy is an emerging discipline that aims at analyzing and modeling the individual anatomy of organs and their biological variability across a population. The goal is not only to model the normal variations among a population, but also discover morphological differences between normal and pathological populations, and possibly to detect, model and classify the pathologies from structural abnormalities. Applications are very important both in neuroscience, to minimize the influence of the anatomical variability in functional group analysis, and in medical imaging, to better drive the adaptation of generic models of the anatomy (atlas) into patient-specific data (personalization). However, understanding and modeling the shape of organs is made difficult by the absence of physical models for comparing different subjects, the complexity of shapes, and the high number of degrees of freedom implied. Moreover, the geometric nature of the anatomical features usually extracted raises the need for statistics and computational methods on objects that do not belong to standard Euclidean spaces. We investigate in this chapter the Riemannian metric as a basis for developing generic algorithms to compute on manifolds. We show that few computational tools derived from this structure can be used in practice as the atoms to build more complex generic algorithms such as mean computation, Mahalanobis distance, interpolation, filtering and anisotropic diffusion on fields of geometric features. This computational framework is illustrated with the joint estimation and anisotropic smoothing of diffusion tensor images and with the modeling of the brain variability from sulcal lines.

A time-frequency classifier for human gait recognition

Abstract: Radar has established itself as an effective all-weather, day or night sensor. Radar signals can penetrate walls and provide information on moving targets. Recently, radar has been used as an effective biometric sensor for classification of gait. The return from a coherent radar system contains a frequency offset in the carrier frequency, known as the Doppler Effect. The movements of arms and legs give rise to micro Doppler which can be clearly detailed in the time-frequency domain using traditional or modern time-frequency signal representation. In this paper we propose a gait classifier based on subspace learning using principal components analysis(PCA). The training set consists of feature vectors defined as either time or frequency snapshots taken from the spectrogram of radar backscatter. We show that gait signature is captured effectively in feature vectors. Feature vectors are then used in training a minimum distance classifier based on Mahalanobis distance metric. Results show that gait classification with high accuracy and short observation window is achievable using the proposed classifier.

Positive Semidefinite Metric Learning with Boosting

Abstract: The learning of appropriate distance metrics is a critical problem in image classification and retrieval. In this work, we propose a boosting-based technique, termed \BoostMetric, for learning a Mahalanobis distance metric. One of the primary difficulties in learning such a metric is to ensure that the Mahalanobis matrix remains positive semidefinite. Semidefinite programming is sometimes used to enforce this constraint, but does not scale well. \BoostMetric is instead based on a key observation that any positive semidefinite matrix can be decomposed into a linear positive combination of trace-one rank-one matrices. \BoostMetric thus uses rank-one positive semidefinite matrices as weak learners within an efficient and scalable boosting-based learning process. The resulting method is easy to implement, does not require tuning, and can accommodate various types of constraints. Experiments on various datasets show that the proposed algorithm compares favorably to those state-of-the-art methods in terms of classification accuracy and running time.

Semi-supervised metric learning using pairwise constraints

Abstract: Distance metric has an important role in many machine learning algorithms. Recently, metric learning for semi-supervised algorithms has received much attention. For semi-supervised clustering, usually a set of pairwise similarity and dissimilarity constraints is provided as supervisory information. Until now, various metric learning methods utilizing pairwise constraints have been proposed. The existing methods that can consider both positive (must-link) and negative (cannot-link) constraints find linear transformations or equivalently global Mahalanobis metrics. Additionally, they find metrics only according to the data points appearing in constraints (without considering other data points). In this paper, we consider the topological structure of data along with both positive and negative constraints. We propose a kernel-based metric learning method that provides a non-linear transformation. Experimental results on synthetic and real-world data sets show the effectiveness of our metric learning method.

Positive Semidefinite Metric Learning with Boosting

Abstract: The learning of appropriate distance metrics is a critical problem in image classification and retrieval. In this work, we propose a boosting-based technique, termed BOOSTMETRIC, for learning a Mahalanobis distance metric. One of the primary difficulties in learning such a metric is to ensure that the Mahalanobis matrix remains positive semidefinite. Semidefinite programming is sometimes used to enforce this constraint, but does not scale well. BOOSTMETRIC is instead based on a key observation that any positive semidefinite matrix can be decomposed into a linear positive combination of trace-one rank-one matrices. BOOSTMETRIC thus uses rank-one positive semidefinite matrices as weak learners within an efficient and scalable boosting-based learning process. The resulting method is easy to implement, does not require tuning, and can accommodate various types of constraints. Experiments on various datasets show that the proposed algorithm compares favorably to those state-of-the-art methods in terms of classification accuracy and running time.

Human action recognition under log-euclidean riemannian metric

Abstract: This paper presents a new action recognition approach based on local spatio-temporal features. The main contributions of our approach are twofold. First, a new local spatio-temporal feature is proposed to represent the cuboids detected in video sequences. Specifically, the descriptor utilizes the covariance matrix to capture the self-correlation information of the low-level features within each cuboid. Since covariance matrices do not lie on Euclidean space, the Log-Euclidean Riemannian metric is used for distance measure between covariance matrices. Second, the Earth Mover’s Distance (EMD) is used for matching any pair of video sequences. In contrast to the widely used Euclidean distance, EMD achieves more robust performances in matching histograms/distributions with different sizes. Experimental results on two datasets demonstrate the effectiveness of the proposed approach.

Multiclass MTS for Simultaneous Feature Selection and Classification

Abstract: Multiclass Mahalanobis-Taguchi system (MMTS), the extension of MTS, is developed for simultaneous multiclass classification and feature selection. In MMTS, the multiclass measurement scale is constructed by establishing an individual Mahalanobis space for each class. To increase the validity of the measurement scale, the Gram-Schmidt process is performed to mutually orthogonalize the features and eliminate the multicollinearity. The important features are identified using the orthogonal arrays and the signal-to-noise ratio, and are then used to construct a reduced model measurement scale. The contribution of each important feature to classification is also derived according to the effect gain to develop a weighted Mahalanobis distance which is finally used as the distance metric for the classification of MMTS. Using the reduced model measurement scale, an unknown example will be classified into the class with minimum weighted Mahalanobis distance considering only the important features. For evaluating the effectiveness of MMTS, a numerical experiment is implemented, and the results show that MMTS outperforms other well-known algorithms not only on classification accuracy but also on feature selection efficiency. Finally, a real case about gestational diabetes mellitus is studied, and the results indicate the practicality of MMTS in real-world applications.

Automated processing of coral reef benthic images

Abstract: We describe an automated computer algorithm for the classification of coral reef benthic organisms and substrates sampled using a typical photographic quadrat survey. The technique compares subsections of a quadrat sample image (blocks) to a library of identified species blocks and computes a distance or probability of identification in a multidimensional hypervolume of discrimination metrics. The discrimination metrics include texture (calculated from a radial sampling of a two-dimensional discrete cosine transform) and three channels of a normalized color space. A standard multivariate classification technique based on the Mahalanobis distance was unsuccessful in discriminating substrata because of the large morphological variation inherent in reef organisms. An alternative classification scheme based on an exhaustive search through an organism reference library yielded classification maps comparable to those obtained by manual analysis.

What’s so special about Euclidean distance?

Abstract: In this paper, we provide an application-oriented characterization of a class of distance functions monotonically related to the Euclidean distance in terms of some general properties of distance functions between real-valued vectors. Our analysis hinges upon two fundamental properties of distance functions that we call “value-sensitivity” and “order-sensitivity”. We show how these two general properties, combined with natural monotonicity considerations, lead to characterization results that single out several versions of Euclidean distance from the wide class of separable distance functions. We then discuss and motivate our results in two different and apparently unrelated application areas—mobility measurement and spatial voting theory—and propose our characterization as a test for deciding whether Euclidean distance (or some suitable variant) should be used in your favourite application context.

Learning Bregman Distance Functions and Its Application for Semi-Supervised Clustering

Abstract: Learning distance functions with side information plays a key role in many machine learning and data mining applications. Conventional approaches often assume a Mahalanobis distance function. These approaches are limited in two aspects: (i) they are computationally expensive (even infeasible) for high dimensional data because the size of the metric is in the square of dimensionality; (ii) they assume a fixed metric for the entire input space and therefore are unable to handle heterogeneous data. In this paper, we propose a novel scheme that learns nonlinear Bregman distance functions from side information using a non-parametric approach that is similar to support vector machines. The proposed scheme avoids the assumption of fixed metric by implicitly deriving a local distance from the Hessian matrix of a convex function that is used to generate the Bregman distance function. We also present an efficient learning algorithm for the proposed scheme for distance function learning. The extensive experiments with semi-supervised clustering show the proposed technique (i) outperforms the state-of-the-art approaches for distance function learning, and (ii) is computationally efficient for high dimensional data.

Information Loss of the Mahalanobis Distance in High Dimensions: Application to Feature Selection

Abstract: When an infinite training set is used, the Mahalanobis distance between a pattern measurement vector of dimensionality D and the center of the class it belongs to is distributed as a chi2 with D degrees of freedom. However, the distribution of Mahalanobis distance becomes either Fisher or Beta depending on whether cross validation or resubstitution is used for parameter estimation in finite training sets. The total variation between chi2 and Fisher, as well as between chi2 and Beta, allows us to measure the information loss in high dimensions. The information loss is exploited then to set a lower limit for the correct classification rate achieved by the Bayes classifier that is used in subset feature selection.

Fuzzy C-means algorithm based on standard mahalanobis distances

Abstract: Some of the well-known fuzzy clustering algorithms are based on Euclidean distance function, which can only be used to detect spherical structural clusters. Gustafson-Kessel clustering algorithm and Gath-Geva clustering algorithm were developed to detect non-spherical structural clusters. However, the former needs added constraint of fuzzy covariance matrix, the later can only be used for the data with multivariate Gaussian distribution. Two improved Fuzzy C-Means algorithm based on different Mahalanobis distance, called FCM-M and FCM-CM were proposed by our previous works, In this paper, A improved Fuzzy C-Means algorithm based on a standard Mahalanobis distance (FCM-SM) is proposed The experimental results of three real data sets show that our proposed new algorithm has the better performance.

Landmark Localisation in 3D Face Data

Abstract: A comparison of several approaches that use graph matching and cascade filtering for landmark localization in 3D face data is presented. For the first method, we apply the structural graph matching algorithm “relaxation by elimination” using a simple “distance to local plane” node property and a “Euclidean distance” arc property. After the graph matching process has eliminated unlikely candidates, the most likely triplet is selected, by exhaustive search, as the minimum Mahalanobis distance over a six dimensional space, corresponding to three node variables and three arc variables. A second method uses state-of-the-art pose-invariant feature descriptors embedded into a cascade filter to localize the nose tip. After that, local graph matching is applied to localize the inner eye corners. We evaluate our systems by computing root mean square errors of estimated landmark locations against ground truth landmark localizations within the 3D Face Recognition Grand Challenge database. Our best system, which uses a novel pose-invariant shape descriptor, scores 99.77% successful localization of the nose and 96.82% successful localization of the eyes.

The minimum weighted covariance determinant estimator

Abstract: In this paper, we introduce weighted estimators of the location and dispersion of a multivariate data set with weights based on the ranks of the Mahalanobis distances. We discuss some properties of the estimators like the breakdown point, influence function and asymptotic variance. The outlier detection capacities of different weight functions are compared. A simulation study is given to investigate the finite-sample behavior of the estimators.

Mahalanobis Distance Based Non-negative Sparse Representation for Face Recognition

Abstract: Sparse representation for machine learning has been exploited in past years. Several sparse representation based classification algorithms have been developed for some applications, for example, face recognition. In this paper, we propose an improved sparse representation based classification algorithm. Firstly, for a discriminative representation, a non-negative constraint of sparse coefficient is added to sparse representation problem. Secondly, Mahalanobis distance is employed instead of Euclidean distance to measure the similarity between original data and reconstructed data. The proposed classification algorithm for face recognition has been evaluated under varying illumination and pose using standard face databases. The experimental results demonstrate that the performance of our algorithm is better than that of the up-to-date face recognition algorithm based on sparse representation.

A Comparison of the Mahalanobis-Taguchi System to A Standard Statistical Method for Defect Detection

Abstract: The Mahalanobis-Taguchi System is a diagnosis and forecasting method for multivariate data. Mahalanobis distance is a measure based on correlations between the variables and different patterns that can be identified and analyzed with respect to a base or reference group. This paper presents a comparison of the Mahalanobis-Taguchi System and a standard statistical technique for defect detection by identifying abnormalities. The objective of this research is to provide a method for defect detection with acceptable alpha (probability of type I) and beta (probability of type II) errors.