Optimal transport (OT) theory can be informally described using the words of the French mathematician Gaspard Monge (1746-1818): A worker with a shovel in hand has to move a large pile of sand lying on a construction site. The goal of the worker is to erect with all that sand a target pile with a prescribed shape (for example, that of a giant sand castle). Naturally, the worker wishes to minimize her total effort, quantified for instance as the total distance or time spent carrying shovelfuls of sand. Mathematicians interested in OT cast that problem as that of comparing two probability distributions, two different piles of sand of the same volume. They consider all of the many possible ways to morph, transport or reshape the first pile into the second, and associate a "global" cost to every such transport, using the "local" consideration of how much it costs to move a grain of sand from one place to another. Recent years have witnessed the spread of OT in several fields, thanks to the emergence of approximate solvers that can scale to sizes and dimensions that are relevant to data sciences. Thanks to this newfound scalability, OT is being increasingly used to unlock various problems in imaging sciences (such as color or texture processing), computer vision and graphics (for shape manipulation) or machine learning (for regression, classification and density fitting). This short book reviews OT with a bias toward numerical methods and their applications in data sciences, and sheds lights on the theoretical properties of OT that make it particularly useful for some of these applications.

Computational Optimal Transport

Time-Frequency Analysis.

We take a new look at the relation between the optimal transport problem and the Schrodinger bridge problem from a stochastic control perspective. Our aim is to highlight new connections between the two that are richer and deeper than those previously described in the literature. We begin with an elementary derivation of the Benamou---Brenier fluid dynamic version of the optimal transport problem and provide, in parallel, a new fluid dynamic version of the Schrodinger bridge problem. We observe that the latter establishes an important connection with optimal transport without zero-noise limits and solves a question posed by Eric Carlen in 2006. Indeed, the two variational problems differ by a Fisher information functional. We motivate and consider a generalization of optimal mass transport in the form of a (fluid dynamic) problem of optimal transport with prior. This can be seen as the zero-noise limit of Schrodinger bridges when the prior is any Markovian evolution. We finally specialize to the Gaussian case and derive an explicit computational theory based on matrix Riccati differential equations. A numerical example involving Brownian particles is also provided.

/pdf/on-the-relation-between-optimal-transport-and-schrodinger-49elnteqni.pdf

On the Relation Between Optimal Transport and Schrödinger Bridges: A Stochastic Control Viewpoint

We address the problem of steering the state of a linear stochastic system to a prescribed distribution over a finite horizon with minimum energy, and the problem to maintain the state at a stationary distribution over an infinite horizon with minimum power. For both problems the control and Gaussian noise channels are allowed to be distinct, thereby, placing the results of this paper outside of the scope of previous work both in probability and in control. The special case where the disturbance and control enter through the same channels has been addressed in the first part of this work that was presented as Part I. Herein, we present sufficient conditions for optimality in terms of a system of dynamically coupled Riccati equations in the finite horizon case and in terms of algebraic conditions for the stationary case. We then address the question of feasibility for both problems. For the finite-horizon case, provided the system is controllable, we prove that without any restriction on the directionality of the stochastic disturbance it is always possible to steer the state to any arbitrary Gaussian distribution over any specified finite time-interval. For the stationary infinite horizon case, it is not always possible to maintain the state at an arbitrary Gaussian distribution through constant state-feedback. It is shown that covariances of admissible stationary Gaussian distributions are characterized by a certain Lyapunov-like equation and, in fact, they coincide with the class of stationary state covariances that can be attained by a suitable stationary colored noise as input. We finally address the question of how to compute suitable controls numerically. We present an alternative to solving the system of coupled Riccati equations, by expressing the optimal controls in the form of solutions to (convex) semi-definite programs for both cases. We conclude with an example to steer the state covariance of the distribution of inertial particles to an admissible stationary Gaussian distribution over a finite interval, to be maintained at that stationary distribution thereafter by constant-gain state-feedback control.

https://escholarship.org/content/qt0464p57f/qt0464p57f.pdf?t=pwur7k

Optimal Steering of a Linear Stochastic System to a Final Probability Distribution—Part III

Review: Divergence measures for statistical data processing-An annotated bibliography

The paper is concerned with the practical problem of how to compare power spectral densities of multivariable time-series. For scalar time-series several notions of distance (divergences) have been proposed and studied starting from the early 1970s while multivariable ones have only recently began to receive any attention. In the paper, two classes of divergence measures inspired by classical prediction theory are introduced. These divergences naturally induce Riemannian metrics on the cone of multivariable densities. The metrics amount to the quadratic term in the divergence between “infinitesimally close to each other” power spectra. For one of the two we provide explicit formulae for the corresponding geodesics and geodesic distance. A close connection between the geometry of power spectra and the geometry of the Fisher-Rao metric is noted.

Distances and Riemannian Metrics for Multivariate Spectral Densities

This paper explores a geometric framework for modeling nonstationary but slowly varying time series, based on the assumption that short-windowed power spectra capture their spectral character, and that energy transference in the frequency domain has a physical significance. The framework relies on certain notions of transportation distance and their respective geodesics to model possible nonparametric changes in the power spectral density with respect to time. We discuss the relevance of this framework to applications in spectral tracking, spectral averaging, and speech morphing.

/pdf/geometric-methods-for-spectral-analysis-41wgr7qvq8.pdf

Geometric Methods for Spectral Analysis

We introduce and compare certain distance measures between covariance matrices. These originate in information theory, quantum mechanics and optimal transport. More specifically, we show that the Bures/Hellinger distance between covariance matrices coincides with the Wasserstein-2 distance between the corresponding Gaussian distributions. We also note that this Bures/Hellinger/Wasserstein distance can be expressed as the solution to a linear matrix inequality (LMI). A consequence of this fact is that the computational cost in covariance approximation problems scales nicely with the size of the matrices involved. We discuss the relevance of this metric in spectral-line detection and spectral morphing.

On the Geometry of Covariance Matrices

Author(s): Ning, Lipeng; Jiang, Xianhua; Georgiou, Tryphon | Abstract: We consider problems of estimation of structured covariance matrices, and in particular of matrices with a Toeplitz structure. We follow a geometric viewpoint that is based on some suitable notion of distance. To this end, we overview and compare several alternatives metrics and divergence measures. We advocate a specific one which represents the Wasserstein distance between the corresponding Gaussians distributions and show that it coincides with the so-called Bures/Hellinger distance between covariance matrices as well. Most importantly, besides the physically appealing interpretation, computation of the metric requires solving a linear matrix inequality (LMI). As a consequence, computations scale nicely for problems involving large covariance matrices, and linear prior constraints on the covariance structure are easy to handle. We compare this transportation/Bures/Hellinger metric with the maximum likelihood and the Burg methods as to their performance with regard to estimation of power spectra with spectral lines on a representative case study from the literature.

/pdf/geometric-methods-for-estimation-of-structured-covariances-2itst5mp1d.pdf

Geometric methods for estimation of structured covariances

Spectral analysis of time-series has been an important tool of science for a very long time. Indeed, periodicities of celestial events and of weather phenomena sparked the curiosity and imagination of early thinkers in the history of science. More recently, the refined mathematical techniques for spectral analysis of the past fifty years form the basis for a wide range of technological developments from medical imaging to communications. However, in spite of the centrality of a spectral representation of time-series, no universal agreement exists on what is a suitable metric between such representations. In this paper we discuss three alternative metrics along with their application in morphing speech signals. Morphing can be naturally effected via a deformation of power spectra along geodesics of the corresponding geometry. The acoustic effect of morphing between two speakers is documented at a website.

Xianhua Jiang

Papers

Distances and Riemannian Metrics for Multivariate Spectral Densities

Geometric Methods for Spectral Analysis

On the Geometry of Covariance Matrices

Geometric methods for estimation of structured covariances

Metrics and Morphing of Power Spectra