Showing papers on "Information geometry published in 2006"

A Riemannian Framework for Tensor Computing

Abstract: Tensors are nowadays a common source of geometric information. In this paper, we propose to endow the tensor space with an affine-invariant Riemannian metric. We demonstrate that it leads to strong theoretical properties: the cone of positive definite symmetric matrices is replaced by a regular and complete manifold without boundaries (null eigenvalues are at the infinity), the geodesic between two tensors and the mean of a set of tensors are uniquely defined, etc. We have previously shown that the Riemannian metric provides a powerful framework for generalizing statistics to manifolds. In this paper, we show that it is also possible to generalize to tensor fields many important geometric data processing algorithms such as interpolation, filtering, diffusion and restoration of missing data. For instance, most interpolation and Gaussian filtering schemes can be tackled efficiently through a weighted mean computation. Linear and anisotropic diffusion schemes can be adapted to our Riemannian framework, through partial differential evolution equations, provided that the metric of the tensor space is taken into account. For that purpose, we provide intrinsic numerical schemes to compute the gradient and Laplace-Beltrami operators. Finally, to enforce the fidelity to the data (either sparsely distributed tensors or complete tensors fields) we propose least-squares criteria based on our invariant Riemannian distance which are particularly simple and efficient to solve.

Statistics on the Manifold of Multivariate Normal Distributions: Theory and Application to Diffusion Tensor MRI Processing

Abstract: This paper is dedicated to the statistical analysis of the space of multivariate normal distributions with an application to the processing of Diffusion Tensor Images (DTI). It relies on the differential geometrical properties of the underlying parameters space, endowed with a Riemannian metric, as well as on recent works that led to the generalization of the normal law on Riemannian manifolds. We review the geometrical properties of the space of multivariate normal distributions with zero mean vector and focus on an original characterization of the mean, covariance matrix and generalized normal law on that manifold. We extensively address the derivation of accurate and efficient numerical schemes to estimate these statistical parameters. A major application of the present work is related to the analysis and processing of DTI datasets and we show promising results on synthetic and real examples.

Metric learning for text documents

Abstract: Many algorithms in machine learning rely on being given a good distance metric over the input space. Rather than using a default metric such as the Euclidean metric, it is desirable to obtain a metric based on the provided data. We consider the problem of learning a Riemannian metric associated with a given differentiable manifold and a set of points. Our approach to the problem involves choosing a metric from a parametric family that is based on maximizing the inverse volume of a given data set of points. From a statistical perspective, it is related to maximum likelihood under a model that assigns probabilities inversely proportional to the Riemannian volume element. We discuss in detail learning a metric on the multinomial simplex where the metric candidates are pull-back metrics of the Fisher information under a Lie group of transformations. When applied to text document classification the resulting geodesic distance resemble, but outperform, the tfidf cosine similarity measure.

DTI segmentation by statistical surface evolution

Abstract: We address the problem of the segmentation of cerebral white matter structures from diffusion tensor images (DTI). A DTI produces, from a set of diffusion-weighted MR images, tensor-valued images where each voxel is assigned with a 3 times 3 symmetric, positive-definite matrix. This second order tensor is simply the covariance matrix of a local Gaussian process, with zero-mean, modeling the average motion of water molecules. As we will show in this paper, the definition of a dissimilarity measure and statistics between such quantities is a nontrivial task which must be tackled carefully. We claim and demonstrate that, by using the theoretically well-founded differential geometrical properties of the manifold of multivariate normal distributions, it is possible to improve the quality of the segmentation results obtained with other dissimilarity measures such as the Euclidean distance or the Kullback-Leibler divergence. The main goal of this paper is to prove that the choice of the probability metric, i.e., the dissimilarity measure, has a deep impact on the tensor statistics and, hence, on the achieved results. We introduce a variational formulation, in the level-set framework, to estimate the optimal segmentation of a DTI according to the following hypothesis: Diffusion tensors exhibit a Gaussian distribution in the different partitions. We must also respect the geometric constraints imposed by the interfaces existing among the cerebral structures and detected by the gradient of the DTI. We show how to express all the statistical quantities for the different probability metrics. We validate and compare the results obtained on various synthetic data-sets, a biological rat spinal cord phantom and human brain DTIs

Extended SMART algorithms for non-negative matrix factorization

Abstract: In this paper we derive a family of new extended SMART (Simultaneous Multiplicative Algebraic Reconstruction Technique) algorithms for Non-negative Matrix Factorization (NMF). The proposed algorithms are characterized by improved efficiency and convergence rate and can be applied for various distributions of data and additive noise. Information theory and information geometry play key roles in the derivation of new algorithms. We discuss several loss functions used in information theory which allow us to obtain generalized forms of multiplicative NMF learning adaptive algorithms. We also provide flexible and relaxed forms of the NMF algorithms to increase convergence speed and impose an additional constraint of sparsity. The scope of these results is vast since discussed generalized divergence functions include a large number of useful loss functions such as the Amari α– divergence, Relative entropy, Bose-Einstein divergence, Jensen-Shannon divergence, J-divergence, Arithmetic-Geometric (AG) Taneja divergence, etc. We applied the developed algorithms successfully to Blind (or semi blind) Source Separation (BSS) where sources may be generally statistically dependent, however are subject to additional constraints such as nonnegativity and sparsity. Moreover, we applied a novel multilayer NMF strategy which improves performance of the most proposed algorithms.

Riemannian manifold learning for nonlinear dimensionality reduction

Abstract: In recent years, nonlinear dimensionality reduction (NLDR) techniques have attracted much attention in visual perception and many other areas of science. We propose an efficient algorithm called Riemannian manifold learning (RML). A Riemannian manifold can be constructed in the form of a simplicial complex, and thus its intrinsic dimension can be reliably estimated. Then the NLDR problem is solved by constructing Riemannian normal coordinates (RNC). Experimental results demonstrate that our algorithm can learn the data's intrinsic geometric structure, yielding uniformly distributed and well organized low-dimensional embedding data.

Riemannian Manifold Learning for Nonlinear Dimensionality Reduction

Abstract: In recent years, nonlinear dimensionality reduction (NLDR) techniques have attracted much attention in visual perception and many other areas of science. We propose an efficient algorithm called Riemannian manifold learning (RML). A Riemannian manifold can be constructed in the form of a simplicial complex, and thus its intrinsic dimension can be reliably estimated. Then the NLDR problem is solved by constructing Riemannian normal coordinates (RNC). Experimental results demonstrate that our algorithm can learn the data's intrinsic geometric structure, yielding uniformly distributed and well organized low-dimensional embedding data.

Shape analysis using the Fisher-Rao Riemannian metric: unifying shape representation and deformation

Abstract: We show that the Fisher-Rao Riemannian metric is a natural, intrinsic tool for computing shape geodesics. When a parameterized probability density function is used to represent a landmark-based shape, the modes of deformation are automatically established through the Fisher information of the density. Consequently, given two shapes parameterized by the same density model, the geodesic distance between them under the action of the Fisher-Rao metric is a convenient shape distance measure. It has the advantage of being an intrinsic distance measure and invariant to reparameterization. We first model shape landmarks using a Gaussian mixture model and then compute geodesic distances between two shapes using the Fisher-Rao metric corresponding to the mixture model. We illustrate our approach by computing Fisher geodesics between 2D corpus callosum shapes. Shape representation via the mixture model and shape deformation via the Fisher geodesic are hereby unified in this approach.

Black hole as emergent holographic geometry of weakly interacting hot Yang-Mills gas

Abstract: We demonstrate five-dimensional anti-de Sitter black hole emerges as dual geometry holographic to weakly interacting = 4 superconformal Yang-Mills theory. We first note that an ideal probe of the dual geometry is the Yang-Mills instanton, probing point by point in spacetime. We then study instanton moduli space at finite temperature by adopting Hitchin's proposal that geometry of the moduli space is definable by Fisher-Rao "information geometry". In Yang-Mills theory, the information metric is measured by a novel class of gauge-invariant, nonlocal operators in the instanton sector. We show that the moduli space metric exhibits (1) asymptotically anti-de Sitter, (2) horizon at radial distance set by the Yang-Mills temperature, and (3) after Wick rotation of the moduli space to the Lorentzian signature, a singularity at the origin. We argue that the dual geometry emerges even for rank of gauge groups of order unity and for weak `t Hooft coupling.

Information Intrinsic Geometric Flows

Abstract: Geometric Flow Theory is cross fertilized by diverse elements coming from Pure Mathematic and Mathematical Physic, but its foundation is mainly based on Riemannian Geometry, as explained by M. Berger in a recent panoramic view of this discipline, its extension to complex manifolds, the Erich Kahler’s Geometry, vaunted for its unabated vitality by J.P. Bourguignon, and Minimal Surface Theory. This paper would like to initiate seminal studies for applying intrinsic geometric flows in the framework of information geometry theory. More specifically, after having introduced Information metric deduced for Complex Auto‐Regressive (CAR) models from Fisher Matrix (Siegel Metric and Hyper‐Abelian Metric from Entropic Kahler Potential), we study asymptotic behavior of reflection coefficients of CAR models driven by intrinsic Information geometric Kahler‐Ricci and Calabi flows. These Information geometric flows can be used in different contexts to define distance between CAR models interpreted as geodesics of Entropy Manifold. We conclude with potential application of Intrinsic Geometric Flow on Gauss Map to transform Manifold of any dimension by mean of Generalized Weierstrass Formula introduced by Kenmotsu that can represent arbitrary surfaces with non‐vanishing mean curvature in terms of the mean curvature function and the Gauss map. One of the advantages of the generalized formulae is that they allow to construct a new class of deformations of surfaces by use of Intrinsic Geometric Flow on Gauss Map. We conclude with the Heat equation interpretation in the framework of Information Geometry.

5d Black Hole as Emergent Geometry of Weakly Interacting 4d Hot Yang-Mills Gas

Abstract: We demonstrate five-dimensional anti-de Sitter black hole emerges as dual geometry holographic to weakly interacting N=4 superconformal Yang-Mills theory. We first note that an ideal probe of the dual geometry is the Yang-Mills instanton, probing point by point in spacetime. We then study instanton moduli space at finite temperature by adopting Hitchin's proposal that geometry of the moduli space is definable by Fisher-Rao "information geometry". In Yang-Mills theory, the information metric is measured by a novel class of gauge-invariant, nonlocal operators in the instanton sector. We show that the moduli space metric exhibits (1) asymptotically anti-de Sitter, (2) horizon at radial distance set by the Yang-Mills temperature, and (3) after Wick rotation of the moduli space to the Lorentzian signature, a singularity at the origin. We argue that the dual geometry emerges even for rank of gauge groups of order unity and for weak `t Hooft coupling.

A new closed-form information metric for shape analysis

Abstract: Shape matching plays a prominent role in the analysis of medical and biological structures. Recently, a unifying framework was introduced for shape matching that uses mixture-models to couple both the shape representation and deformation. Essentially, shape distances were defined as geodesics induced by the Fisher-Rao metric on the manifold of mixture-model represented shapes. A fundamental drawback of the Fisher-Rao metric is that it is NOT available in closed-form for the mixture model. Consequently, shape comparisons are computationally very expensive. Here, we propose a new Riemannian metric based on generalized φ- entropy measures. In sharp contrast to the Fisher-Rao metric, our new metric is available in closed-form. Geodesic computations using the new metric are considerably more efficient. Discriminative capabilities of this new metric are studied by pairwise matching of corpus callosum shapes. Comparisons are conducted with the Fisher-Rao metric and the thin-plate spline bending energy.

Sur la forme de la boule unité de la norme stable unidimensionnelle

Abstract: We study the stable norm on the first homology of a Riemannian polyhedron. In the one-dimensional case (metric graphs), the geometry of the unit ball of this norm is completely described by the combinatorial structure of the graph. For a smooth manifold of dimension ≥3 and using polyhedral techniques, we show that a large class of polytopes appears as unit ball of the stable norm associated to some metric conformal to a given one.

Extended SMART Algorithms for Non-negative Matrix Factorization Invited Paper

Abstract: In this paper we derive a family of new extended SMART (Simultaneous Multiplicative Algebraic Reconstruction Technique) algorithms for Non-negative Matrix Factorization (NMF). The proposed algorithms are characterized by improved efficiency and convergence rate and can be applied for various distributions of data and additive noise. Information theory and information geometry play key roles in the derivation of new algorithms. We discuss several loss functions used in information theory which allow us to obtain generalized forms of multiplicative NMF learning adaptive algorithms. We also provide flexible and relaxed forms of the NMF algorithms to increase convergence speed and impose an additional constraint of sparsity. The scope of these results is vast since discussed generalized divergence functions include a large number of useful loss functions such as the Amari α- divergence. Relative entropy, Bose-Einstein divergence, Jensen-Shannon divergence, J-divergence, Arithmetic-Geometric (AG) Taneja divergence, etc. We applied the developed algorithms successfully to Blind (or semi blind) Source Separation (BSS) where sources tray be generally statistically dependent, however are subject to additional constraints such as non-negativity and sparsity. Moreover, we applied a novel multilayer NMF strategy which improves performance of the most proposed algorithms.

The minimal entropy conjecture for nonuniform rank one lattices

Abstract: The Besson–Courtois–Gallot theorem is proven for noncompact finite volume Riemannian manifolds. In particular, no bounded geometry assumptions are made. This proves the minimal entropy conjecture for nonuniform rank one lattices.

Generalized information-entropy measures and Fisher information

Abstract: We show how Fisher's information already known particular character as the fundamental information geometric object which plays the role of a metric tensor for a statistical differential manifold, can be derived in a relatively easy manner through the direct application of a generalized logarithm and exponential formalism to generalized information-entropy measures. We shall first shortly describe how the generalization of information-entropy measures naturally comes into being if this formalism is employed and recall how the relation between all the information measures is best understood when described in terms of a particular logarithmic Kolmogorov-Nagumo average. Subsequently, extending Kullback-Leibler's relative entropy to all these measures defined on a manifold of parametrized probability density functions, we obtain the metric which turns out to be the Fisher information matrix elements times a real multiplicative deformation parameter. The metrics independence from the non-extensive character of the system, and its proportionality to the rate of change of the multiplicity under a variation of the statistical probability parameter space, emerges naturally in the frame of this representation.

Color Data Coding for Three-Dimensional Mesh Models Considering Connectivity and Geometry Information

Abstract: In this paper, we propose a new predictive coding scheme for color data of three-dimensional (3-D) mesh models. We exploit connectivity and geometry information to improve coding efficiency. After ordering all vertices in a 3-D mesh model with a connectivity coding technique, we propose a geometry predictor to compress the color data efficiently. The predicted color can be obtained by a weighted sum of reconstructed colors for adjacent vertices using both angles and distances between the current vertex and adjacent vertices. Simulation results show that the proposed scheme provides enhanced coding efficiency over previous works for various 3-D mesh models.

Exact Pairwise Error Probability of Differential Space-Time Codes in Spatially Correlated Channels

Abstract: In this paper, we derive an analytical expression for the exact pairwise error probability (PEP) of a differential space-time coded system operating over a spatially correlated slow fading channel. An analytic model for spatial correlation is used which fully accounts for antenna spacing, antenna geometry and non-isotropic scattering distributions. Inclusion of spatial information in error performance analysis provides valuable insights into the physical factors determining the performance of a differential space-time code (DSTC). Using this new PEP expression, we investigate the effects of antenna spacing, antenna geometries and azimuth power distribution parameters (angle of arrival/departure and angular spread) on the performance of a differential space-time block code (DSTBC) proposed in the literature for two transmit antennas.

A GPU-based Algorithm for Estimating 3D Geometry and Motion in Near Real-time

Abstract: Real-time 3D geometry and motion estimation has many important applications in areas such as robot navigation and dynamic image-based rendering. A novel algorithm is proposed in this paper for estimating 3D geometry and motion of dynamic scenes based on captured stereo sequences. All computations are conducted in the 2D image space of the center view and the results are represented in forms of disparity maps and disparity flow maps. A dynamic programming based technique is used for searching global optimal disparity maps and disparity flow maps under an energy minimization framework. To achieve high processing speed, most operations are implemented on the Graphics Processing Units (GPU) of programmable graphics hardware. As a result, the derived algorithm is capable of producing both 3D geometry and motion information for dynamic scenes in near real-time. Experiments on two trinocular stereo sequences demonstrate that the proposed algorithm can handle scenes that contain non-rigid motion as well as those captured by moving cameras.

Unitary Orbits in a Full Matrix Algebra

Abstract: The Hilbert manifold ∑ ∞ consisting of positive invertible (unitized) Hilbert-Schmidt operators has a rich structure and geometry. The geometry of unitary orbits Ω⊂∑ ∞ is studied from the topological and metric viewpoints: we seek for conditions that ensure the existence of a smooth local structure for the set Ω, and we study the convexity of this set for the geodesic structures that arise when we give ∑ ∞ different Riemannian metrics.

Modelling the interaction of stochastic systems with electromagnetic fields

Abstract: The analysis presented in this paper, shows how the basic statistics in the interaction of stochastic systems with electromagnetic fields, i.e., the average and the variance of linear observables, can be computed from the probability distributions defining the stochastic geometry. The most difficult part in this computation is the construction of the covariance operator which requires the solution of a series of integral equations over the average geometry. The RHSs of these integral equations are determined by an expansion of the covariance operator of a regular field trace on the stochastic geometry. This determines the computational complexity of the probabilistic approach. In the presentation of this paper, some numerical experiments on concrete examples are shown, and the computational complexity of the probabilistic approach are investigated whether it also determines the minimal computational effort of the statistical approach.

Partial regularity of mass-minimizing rectifiable sections

Abstract: Let B be a fiber bundle with compact fiber F over a compact Riemannian n-manifold Mn. There is a natural Riemannian metric on the total space B consistent with the metric on M. With respect to that metric, the volume of a rectifiable section σ: M → B is the mass of the image σ(M) as a rectifiable n-current in B.

Application of the Fisher-Rao Metric to Structure Detection

Abstract: Certain structure detection problems can be solved by sampling a parameter space for the different structures at a finite number of points and checking each point to see if the corresponding structure has a sufficient number of inlying measurements. The measurement space is a Riemannian manifold and the measurements relevant to a given structure are near to or on a submanifold which constitutes the structure. The probability density function for the errors in the measurements is described using a generalisation of the Gaussian density to Riemannian manifolds. The conditional probability density function for the measurements yields the Fisher information which defines a metric, known as the Fisher-Rao metric, on the parameter space. The main result is a derivation of an asymptotic approximation to the Fisher-Rao metric, under the assumption that the measurement noise is small. Using this approximation to the Fisher-Rao metric, the parameter space is sampled, such that each point of the parameter space is near to at least one sample point, to within the level of accuracy allowed by the measurement errors. The probability of a false detection of a structure is estimated. The feasibility of this approach to structure detection is tested experimentally using the example of line detection in digital images.

Variational Approximation for Fokker–Planck Equation on Riemannian Manifold

Abstract: Under the bounded geometry assumption on Riemannian manifold M, a variational approximation for Fokker–Planck equation on M is constructed by the scheme of Jordan et al. in SIAM J Math Anal 29(1):1–17, 1998. Moreover, the uniqueness and global L p -estimate of the solution for 1 < p < dim(M)/(dim(M) − 1) are obtained for a broad class of potential.

Analyticity of Riemannian exponential maps on ${\rm Diff}(\T)$

Abstract: We study the exponential maps induced by Sobolev type right-invariant (weak) Riemannian metrics of order $k\ge1$ on the Lie group of smooth, orientation preserving diffeomorphisms of the circle. We prove that each of them defines an {\em analytic} Frechet chart of the identity.

Information geometry and phase transitions

Abstract: We present, from an information theoretic viewpoint, an analysis of phase transitions and critical phenomena in quantum systems. Our study is based on geometrical considerations within the Riemannian space of thermodynamic parameters that characterize the system. A metric for the space can be derived from an appropriate definition of distance between quantum states. For this purpose, we consider generalized α-divergences that include the standard Kullback–Leibler relative entropy. The use of other measures of information distance is taken into account, and the thermodynamic stability of the system is discussed from this geometric perspective.

Geometric properties of almost contact metric 3-submersions

Abstract: In this paper, we discuss some geometric properties of three types of Riemannian submersions whose total space is an almost contact metric manifold with 3-structure. The study is focused on the transference of structures.

Exploratory Datamining in Music

Abstract: This thesis deals with methods and techniques for music exploration, mainly focussing on the task of music retrieval. This task has become an important part of the modern music society in which music is distributed effectively via for example the Internet. This calls for automatic music retrieval and general machine learning in order to provide organization and navigation abilities. This Master's Thesis investigates and compares traditional similarity measures for audio retrieval based on density models, namely the Kullback-Leibler divergence, Earth Mover Distance, Cross-Likelihood Ratio and some variations of these are examined. The methods are evaluated on a custom data set, represented by Mel-Frequency Cepstral Coefficients and a pitch estimation. In terms of optimal model complexity and structure, a maximum retrieval rate of »74-75% is obtained by the Cross-Likelihood Ratio in song retrieval, and »66% in clip retrieval. An alternative method for music exploration and similarity is introduced based on a local perspective, adaptive metrics and the objective to retain the topology of the original feature space for explorative tasks. The method is defined on the basis of Information Geometry and Riemannian metrics. Three metrics (or distance functions) are investigated, namely an unsupervised locally weighted covariance based metric, an unsupervised log-likelihood based metric and finally a supervised metric formulated in terms of the Fisher Information Matrix. The Fisher Information Matrix is reformulated to capture the change in conditional probability of pre-defined auxiliary information given a distance vector in feature space. The metrics are mainly evaluated in simple clustering applications and finally applied to the music similarity task, providing initial results using such adaptive metrics. The results obtained (max »69%) for the supervised metric are in general superior to or comparable with the traditional similarity measures on the clip level depending on the model complexity.