Showing papers by "Jean-Michel Loubes published in 2022"

An improved central limit theorem and fast convergence rates for entropic transportation costs

Abstract: We prove a central limit theorem for the entropic transportation cost between subgaussian probability measures, centered at the population cost. This is the first result which allows for asymptotically valid inference for entropic optimal transport between measures which are not necessarily discrete. In the compactly supported case, we complement these results with new, faster, convergence rates for the expected entropic transportation cost between empirical measures. Our proof is based on strengthening convergence results for dual solutions to the entropic optimal transport problem.

Weak limits of entropy regularized Optimal Transport; potentials, plans and divergences

Abstract: : This work deals with the asymptotic distribution of both potentials and couplings of entropic regularized optimal transport for compactly supported probabilities in R d . We first provide the central limit theorem of the Sinkhorn potentials—the solutions of the dual problem—as a Gaussian process in C s (Ω). Then we obtain the weak limits of the couplings—the solutions of the primal problem—evaluated on inte- grable functions, proving a conjecture of Harchaoui, Liu and Pal (2020). In both cases, their limit is a real Gaussian random variable. Finally we consider the weak limit of the entropic Sinkhorn divergence under both assumptions H 0 : P = Q or H 1 : P 6 = Q. Under H 0 the limit is a quadratic form applied to a Gaussian process in a Sobolev space, while under H 1 , the limit is Gaussian. We provide also a different characterisa-tion of the limit under H 0 in terms of an infinite sum of an i.i.d. sequence of standard Gaussian random variables. Such results enable statistical inference based on entropic regularized optimal transport.

A survey of Identification and mitigation of Machine Learning algorithmic biases in Image Analysis

Abstract: The problem of algorithmic bias in machine learning has gained a lot of attention in recent years due to its concrete and potentially hazardous implications in society. In much the same manner, biases can also alter modern industrial and safety-critical applications where machine learning are based on high dimensional inputs such as images. This issue has however been mostly left out of the spotlight in the machine learning literature. Contrarily to societal applications where a set of proxy variables can be provided by the common sense or by regulations to draw the attention on potential risks, industrial and safety-critical applications are most of the times sailing blind. The variables related to undesired biases can indeed be indirectly represented in the input data, or can be unknown, thus making them harder to tackle. This raises serious and well-founded concerns towards the commercial deployment of AI-based solutions, especially in a context where new regulations clearly address the issues opened by undesired biases in AI. Consequently, we propose here to make an overview of recent advances in this area, firstly by presenting how such biases can demonstrate themselves, then by exploring different ways to bring them to light, and by probing different possibilities to mitigate them. We finally present a practical remote sensing use-case of industrial Fairness.

GAN Estimation of Lipschitz Optimal Transport Maps

Abstract: This paper introduces the first statistically consistent estimator of the optimal transport map between two probability distributions, based on neural networks. Building on theoretical and practical advances in the field of Lipschitz neural networks, we define a Lipschitz-constrained generative adversarial network penalized by the quadratic transportation cost. Then, we demonstrate that, under regularity assumptions, the obtained generator converges uniformly to the optimal transport map as the sample size increases to infinity. Furthermore, we show through a number of numerical experiments that the learnt mapping has promising performances. In contrast to previous work tackling either statistical guarantees or practicality, we provide an expressive and feasible estimator which paves way for optimal transport applications where the asymptotic behaviour must be certified.

Diffeomorphic Registration Using Sinkhorn Divergences

Abstract: The diffeomorphic registration framework enables to define an optimal matching function between two probability measures with respect to a data-fidelity loss function. The non convexity of the optimization problem renders the choice of this loss function crucial to avoid poor local minima. Recent work showed experimentally the efficiency of entropy-regularized optimal transportation costs, as they are computationally fast and differentiable while having few minima. Following this approach, we provide in this paper a new framework based on Sinkhorn divergences, unbiased entropic optimal transportation costs, and prove the statistical consistency with rate of the empirical optimal deformations.

Quantile-constrained Wasserstein projections for robust interpretability of numerical and machine learning models

Abstract: Robustness studies of black-box models is recognized as a necessary task for numerical models based on structural equations and predictive models learned from data. These studies must assess the model’s robustness to possible misspecification of regarding its inputs (e.g., covariate shift). The study of black-box models, through the prism of uncertainty quantification (UQ), is often based on sensitivity analysis involving a probabilistic structure imposed on the inputs, while ML models are solely constructed from observed data. Our work aim at unifying the UQ and ML interpretability approaches, by providing relevant and easy-to-use tools for both paradigms. To provide a generic and understandable framework for robustness studies, we define perturbations of input information relying on quantile constraints and projections with respect to the Wasserstein distance between probability measures, while preserving their dependence structure. We show that this perturbation problem can be analytically solved. Ensuring regularity constraints by means of isotonic polynomial approximations leads to smoother perturbations, which can be more suitable in practice. Numerical experiments on real case studies, from the UQ and ML fields, highlight the computational feasibility of such studies and provide local and global insights on the robustness of black-box models to input perturbations.

Dimension Reduction for time series with Variational AutoEncoders

Abstract: In this work, we explore dimensionality reduction techniques for univariate and multivariate time series data. We especially conduct a comparison between wavelet decomposition and convolutional variational autoencoders for dimension reduction. We show that variational autoencoders are a good option for reducing the dimension of high dimensional data like ECG. We make these comparisons on a real world, publicly available, ECG dataset that has lots of variability and use the reconstruction error as the metric. We then explore the robustness of these models with noisy data whether for training or inference. These tests are intended to reflect the problems that exist in real-world time series data and the VAE was robust to both tests.

Gaussian Processes on Distributions based on Regularized Optimal Transport

Abstract: We present a novel kernel over the space of probability measures based on the dual formulation of optimal regularized transport. We propose an Hilbertian embedding of the space of probabilities using their Sinkhorn potentials, which are solutions of the dual entropic relaxed optimal transport between the probabilities and a reference measure $\mathcal{U}$. We prove that this construction enables to obtain a valid kernel, by using the Hilbert norms. We prove that the kernel enjoys theoretical properties such as universality and some invariances, while still being computationally feasible. Moreover we provide theoretical guarantees on the behaviour of a Gaussian process based on this kernel. The empirical performances are compared with other traditional choices of kernels for processes indexed on distributions.

Attraction-repulsion clustering: a way of promoting diversity linked to demographic parity in fair clustering

Abstract: Abstract We consider the problem of diversity enhancing clustering , i.e, developing clustering methods which produce clusters that favour diversity with respect to a set of protected attributes such as race, sex, age, etc. In the context of fair clustering , diversity plays a major role when fairness is understood as demographic parity. To promote diversity, we introduce perturbations to the distance in the unprotected attributes that account for protected attributes in a way that resembles attraction-repulsion of charged particles in Physics. These perturbations are defined through dissimilarities with a tractable interpretation. Cluster analysis based on attraction-repulsion dissimilarities penalizes homogeneity of the clusters with respect to the protected attributes and leads to an improvement in diversity. An advantage of our approach, which falls into a pre-processing set-up, is its compatibility with a wide variety of clustering methods and whit non-Euclidean data. We illustrate the use of our procedures with both synthetic and real data and provide discussion about the relation between diversity, fairness, and cluster structure.

Nonparametric Bayesian Regression and Classification on Manifolds, With Applications to 3D Cochlear Shapes

Abstract: Advanced shape analysis studies such as regression and classification need to be performed on curved manifolds, where often, there is a lack of standard statistical formulations. To overcome these limitations, we introduce a novel machine-learning method on the shape space of curves that avoids direct inference on infinite-dimensional spaces and instead performs Bayesian inference with spherical Gaussian processes decomposition. As an application, we study the shape of the cochlear spiral-shaped cavity within the petrous part of the temporal bone. This problem is particularly challenging due to the relationship between shape and gender, especially in children. Experimental results for both synthetic and real data show improved performance compared to state-of-the-art methods.

Multi-Variate Risk Measures under Wasserstein Barycenter

Abstract: When the uni-variate risk measure analysis is generalized into the multi-variate setting, many complex theoretical and applied problems arise, and therefore the mathematical models used for risk quantification usually present model risk. As a result, regulators have started to require that the internal models used by financial institutions are more precise. For this task, we propose a novel multi-variate risk measure, based on the notion of the Wasserstein barycenter. The proposed approach robustly characterizes the company’s exposure, filtering the partial information available from individual sources into an aggregate risk measure, providing an easily computable estimation of the total risk incurred. The new approach allows effective computation of Wasserstein barycenter risk measures in any location–scatter family, including the Gaussian case. In such cases, the Wasserstein barycenter Value-at-Risk belongs to the same family, thus it is characterized just by its mean and deviation. It is important to highlight that the proposed risk measure is expressed in closed analytic forms which facilitate its use in day-to-day risk management. The performance of the new multi-variate risk measures is illustrated in United States market indices of high volatility during the global financial crisis (2008) and during the COVID-19 pandemic situation, showing that the proposed approach provides the best forecasts of risk measures not only for “normal periods”, but also for periods of high volatility.

Fairness constraint in Structural Econometrics and Application to fair estimation using Instrumental Variables

Abstract: A supervised machine learning algorithm determines a model from a learning sample that will be used to predict new observations. To this end, it aggregates individual characteristics of the observations of the learning sample. But this information aggregation does not consider any potential selection on unobservables and any status-quo biases which may be contained in the training sample. The latter bias has raised concerns around the so-called fairness of machine learning algorithms, especially towards disadvantaged groups. In this chapter, we review the issue of fairness in machine learning through the lenses of structural econometrics models in which the unknown index is the solution of a functional equation and issues of endogeneity are explicitly accounted for. We model fairness as a linear operator whose null space contains the set of strictly fair indexes. A fair solution is obtained by projecting the unconstrained index into the null space of this operator or by directly finding the closest solution of the functional equation into this null space. We also acknowledge that policymakers may incur a cost when moving away from the status quo. Achieving approximate fairness is obtained by introducing a fairness penalty in the learning procedure and balancing more or less heavily the influence between the statusquo and a full fair solution.