Scikit-learn is a Python module integrating a wide range of state-of-the-art machine learning algorithms for medium-scale supervised and unsupervised problems. This package focuses on bringing machine learning to non-specialists using a general-purpose high-level language. Emphasis is put on ease of use, performance, documentation, and API consistency. It has minimal dependencies and is distributed under the simplified BSD license, encouraging its use in both academic and commercial settings. Source code, binaries, and documentation can be downloaded from http://scikit-learn.sourceforge.net.

/pdf/scikit-learn-machine-learning-in-python-23clfh9q83.pdf

Scikit-learn: Machine Learning in Python

Summary. We propose the elastic net, a new regularization and variable selection method. Real world data and a simulation study show that the elastic net often outperforms the lasso, while enjoying a similar sparsity of representation. In addition, the elastic net encourages a grouping effect, where strongly correlated predictors tend to be in or out of the model together.The elastic net is particularly useful when the number of predictors (p) is much bigger than the number of observations (n). By contrast, the lasso is not a very satisfactory variable selection method in the

/pdf/regularization-and-variable-selection-via-the-elastic-net-4z8q6j5o0i.pdf

Regularization and variable selection via the elastic net

We develop fast algorithms for estimation of generalized linear models with convex penalties. The models include linear regression, two-class logistic regression, and multinomial regression problems while the penalties include l(1) (the lasso), l(2) (ridge regression) and mixtures of the two (the elastic net). The algorithms use cyclical coordinate descent, computed along a regularization path. The methods can handle large problems and can also deal efficiently with sparse features. In comparative timings we find that the new algorithms are considerably faster than competing methods.

/pdf/regularization-paths-for-generalized-linear-models-via-39r1t606ac.pdf

Regularization Paths for Generalized Linear Models via Coordinate Descent

Despite widespread adoption, machine learning models remain mostly black boxes. Understanding the reasons behind predictions is, however, quite important in assessing trust, which is fundamental if one plans to take action based on a prediction, or when choosing whether to deploy a new model. Such understanding also provides insights into the model, which can be used to transform an untrustworthy model or prediction into a trustworthy one. In this work, we propose LIME, a novel explanation technique that explains the predictions of any classifier in an interpretable and faithful manner, by learning an interpretable model locally varound the prediction. We also propose a method to explain models by presenting representative individual predictions and their explanations in a non-redundant way, framing the task as a submodular optimization problem. We demonstrate the flexibility of these methods by explaining different models for text (e.g. random forests) and image classification (e.g. neural networks). We show the utility of explanations via novel experiments, both simulated and with human subjects, on various scenarios that require trust: deciding if one should trust a prediction, choosing between models, improving an untrustworthy classifier, and identifying why a classifier should not be trusted.

/pdf/why-should-i-trust-you-explaining-the-predictions-of-any-ejtlrtykxr.pdf

"Why Should I Trust You?": Explaining the Predictions of Any Classifier

This paper demonstrates theoretically and empirically that a greedy algorithm called orthogonal matching pursuit (OMP) can reliably recover a signal with m nonzero entries in dimension d given O(m ln d) random linear measurements of that signal. This is a massive improvement over previous results, which require O(m2) measurements. The new results for OMP are comparable with recent results for another approach called basis pursuit (BP). In some settings, the OMP algorithm is faster and easier to implement, so it is an attractive alternative to BP for signal recovery problems.

/pdf/signal-recovery-from-random-measurements-via-orthogonal-5adhd8dwmq.pdf

Signal Recovery From Random Measurements Via Orthogonal Matching Pursuit

This article introduces a non parametric warping model for functional data. When the outcome of an experiment is a sample of curves, data can be seen as realizations of a stochastic process, which takes into account the small variations between the different observed curves. The aim of this work is to define a mean pattern which represents the main behaviour of the set of all the realizations. So we define the structural expectation of the underlying stochastic function. Then we provide empirical estimators of this structural expectation and of each individual warping function. Consistency and asymptotic normality for such estimators are proved.

Non parametric estimation of the structural expectation of a stochastic increasing function

In this paper, we build an estimator of the Hurst exponent of a frac- tional L evy motion. The stochastic process is observed with random noise errors in the following framework: continuous time and discrete observation times. In both cases, we prove consistency of our wavelet type estimator. Moreover we perform some simulations in order to study numerically the asymptotic behaviour of this estimate.

/pdf/hurst-exponent-estimation-of-fractional-levy-motion-4wrhjbuaff.pdf

Hurst exponent estimation of Fractional Lévy Motion

The aim of this article is to review the different rates of convergence encountered in inverse
problems, with both deterministic and stochastic noise. Indeed, in the litterature, several
regularity conditions are often assumed leading to apparently different rates. We point out
the different points of view and provide global assumptions that handle most of the cases
encountered. Moreover we discuss optimality of some different usual estimators in the
minimax but also the maxiset framework.

Review of rates of convergence and regularity conditions for inverse problems

In this paper, we consider the Group Lasso estimator of the covariance matrix of a stochastic process corrupted by an additive noise. We propose to estimate the covariance matrix in a high-dimensional setting under the assumption that the process has a sparse representation in a large dictionary of basis functions. Using a matrix regression model, we propose a new methodology for high-dimensional covariance matrix estimation based on empirical contrast regularization by a group Lasso penalty. Using such a penalty, the method selects a sparse set of basis functions in the dictionary used to approximate the process, leading to an approximation of the covariance matrix into a low dimensional space. Consistency of the estimator is studied in Frobenius and operator norms and an application to sparse PCA is proposed.

/pdf/group-lasso-estimation-of-high-dimensional-covariance-118mpfxjb2.pdf

Group Lasso Estimation of High-dimensional Covariance Matrices

: This work deals with the asymptotic distribution of both potentials and couplings of entropic regularized optimal transport for compactly supported probabilities in R d . We ﬁrst provide the central limit theorem of the Sinkhorn potentials—the solutions of the dual problem—as a Gaussian process in C s (Ω). Then we obtain the weak limits of the couplings—the solutions of the primal problem—evaluated on inte- grable functions, proving a conjecture of Harchaoui, Liu and Pal (2020). In both cases, their limit is a real Gaussian random variable. Finally we consider the weak limit of the entropic Sinkhorn divergence under both assumptions H 0 : P = Q or H 1 : P 6 = Q. Under H 0 the limit is a quadratic form applied to a Gaussian process in a Sobolev space, while under H 1 , the limit is Gaussian. We provide also a diﬀerent characterisa-tion of the limit under H 0 in terms of an inﬁnite sum of an i.i.d. sequence of standard Gaussian random variables. Such results enable statistical inference based on entropic regularized optimal transport.

Jean-Michel Loubes

Papers

Non parametric estimation of the structural expectation of a stochastic increasing function

Hurst exponent estimation of Fractional Lévy Motion

Review of rates of convergence and regularity conditions for inverse problems

Group Lasso Estimation of High-dimensional Covariance Matrices

Weak limits of entropy regularized Optimal Transport; potentials, plans and divergences