Scikit-learn is a Python module integrating a wide range of state-of-the-art machine learning algorithms for medium-scale supervised and unsupervised problems. This package focuses on bringing machine learning to non-specialists using a general-purpose high-level language. Emphasis is put on ease of use, performance, documentation, and API consistency. It has minimal dependencies and is distributed under the simplified BSD license, encouraging its use in both academic and commercial settings. Source code, binaries, and documentation can be downloaded from http://scikit-learn.sourceforge.net.

/pdf/scikit-learn-machine-learning-in-python-23clfh9q83.pdf

Scikit-learn: Machine Learning in Python

Summary. We propose the elastic net, a new regularization and variable selection method. Real world data and a simulation study show that the elastic net often outperforms the lasso, while enjoying a similar sparsity of representation. In addition, the elastic net encourages a grouping effect, where strongly correlated predictors tend to be in or out of the model together.The elastic net is particularly useful when the number of predictors (p) is much bigger than the number of observations (n). By contrast, the lasso is not a very satisfactory variable selection method in the

/pdf/regularization-and-variable-selection-via-the-elastic-net-4z8q6j5o0i.pdf

Regularization and variable selection via the elastic net

We develop fast algorithms for estimation of generalized linear models with convex penalties. The models include linear regression, two-class logistic regression, and multinomial regression problems while the penalties include l(1) (the lasso), l(2) (ridge regression) and mixtures of the two (the elastic net). The algorithms use cyclical coordinate descent, computed along a regularization path. The methods can handle large problems and can also deal efficiently with sparse features. In comparative timings we find that the new algorithms are considerably faster than competing methods.

/pdf/regularization-paths-for-generalized-linear-models-via-39r1t606ac.pdf

Regularization Paths for Generalized Linear Models via Coordinate Descent

Despite widespread adoption, machine learning models remain mostly black boxes. Understanding the reasons behind predictions is, however, quite important in assessing trust, which is fundamental if one plans to take action based on a prediction, or when choosing whether to deploy a new model. Such understanding also provides insights into the model, which can be used to transform an untrustworthy model or prediction into a trustworthy one. In this work, we propose LIME, a novel explanation technique that explains the predictions of any classifier in an interpretable and faithful manner, by learning an interpretable model locally varound the prediction. We also propose a method to explain models by presenting representative individual predictions and their explanations in a non-redundant way, framing the task as a submodular optimization problem. We demonstrate the flexibility of these methods by explaining different models for text (e.g. random forests) and image classification (e.g. neural networks). We show the utility of explanations via novel experiments, both simulated and with human subjects, on various scenarios that require trust: deciding if one should trust a prediction, choosing between models, improving an untrustworthy classifier, and identifying why a classifier should not be trusted.

/pdf/why-should-i-trust-you-explaining-the-predictions-of-any-ejtlrtykxr.pdf

"Why Should I Trust You?": Explaining the Predictions of Any Classifier

This paper demonstrates theoretically and empirically that a greedy algorithm called orthogonal matching pursuit (OMP) can reliably recover a signal with m nonzero entries in dimension d given O(m ln d) random linear measurements of that signal. This is a massive improvement over previous results, which require O(m2) measurements. The new results for OMP are comparable with recent results for another approach called basis pursuit (BP). In some settings, the OMP algorithm is faster and easier to implement, so it is an attractive alternative to BP for signal recovery problems.

/pdf/signal-recovery-from-random-measurements-via-orthogonal-5adhd8dwmq.pdf

Signal Recovery From Random Measurements Via Orthogonal Matching Pursuit

In this paper, we propose a new method to build fair Neural-Network classifiers by using a constraint based on the Wasserstein distance. More specifically, we detail how to efficiently compute the gradients of Wasserstein-2 regularizers for Neural-Networks. The proposed strategy is then used to train Neural-Networks decision rules which favor fair predictions. Our method fully takes into account two specificities of Neural-Networks training: (1) The network parameters are indirectly learned based on automatic differentiation and on the loss gradients, and (2) batch training is the gold standard to approximate the parameter gradients, as it requires a reasonable amount of computations and it can efficiently explore the parameters space. Results are shown on synthetic data, as well as on the UCI Adult Income Dataset. Our method is shown to perform well compared with 'ZafarICWWW17' and linear-regression with Wasserstein-1 regularization, as in 'JiangUAI19', in particular when non-linear decision rules are required for accurate predictions.

Using Wasserstein-2 regularization to ensure fair decisions with Neural-Network classifiers.

La gestion et l'analyse de donnees massives sont systematiquement associees a une architecture de donnees distribuees dans des environnements Hadoop et maintenant Spark. Cet article propose aux statisticiens une introduction a ces technologies en comparant les performances obtenues par l'utilisation elementaire de trois environnements de reference : R, Python Scikit-learn, Spark MLlib sur trois cas d'usage publics : reconnaissance de caracteres, recommandation de films, categorisation de produits. Comme principal resultat, il en ressort que si Spark est tres performant pour la preparation des donnees et la recommandation par filtrage collaboratif (factorisation non negative), les implementations actuelles des methodes classiques d'apprentissage (regression logistique, forets aleatoires) dans MLlib ou SparkML ne concurrencent pas ou mal une utilisation habituelle de ces methodes (R, Python Scikit-learn) dans une architecture integree au sens de non distribuee.

https://hal.archives-ouvertes.fr/hal-01350099/document

Apprentissage sur Données Massives; trois cas d'usage avec R, Python et Spark.

Understanding the behavior of a black-box model with probabilistic inputs can be based on the decomposition of a parameter of interest (e.g., its variance) into contributions attributed to each coalition of inputs (i.e., subsets of inputs). In this paper, we produce conditions for obtaining unambiguous and interpretable decompositions of very general parameters of interest. This allows to recover known decompositions, holding under weaker assumptions than stated in the literature.

On the coalitional decomposition of parameters of interest

The invention relates to a method for estimating, by a computer, a journey time of a vehicle on a road network, the road network being defined in the form of a mesh comprising a plurality of segments x, each segment x being associated with a mean speed unaffected by weather conditions (Va) for crossing the segment, each segment x being associated with a worsened state (M) of weather conditions, the method including: a step of calculating a mean speed affected by weather conditions (Vc) for crossing the segment from the mean speed unaffected by weather conditions (Va) and climate-modelling parameters (P, P*) defined locally for each segment x in accordance with the worsened state (M) of weather conditions; a step of calculating the time required to cross a segment x at a crossing time t from the mean speed affected by weather conditions (Vc) for the given segment x; and a step of calculating the journey time from the calculation of the time required to cross the segments x.

Method for estimating a journey time of a vehicle on a road network

Monge-Kantorovich distances, otherwise known as Wasserstein distances, have received a growing attention in statistics and machine learning as a powerful discrepancy measure for probability distributions. In this paper, we focus on forecasting a Gaussian process indexed by probability distributions. For this, we provide a family of positive definite kernels built using transportation based distances. We provide a probabilistic understanding of these kernels and characterize the corresponding stochastic processes. We prove that the Gaussian processes indexed by distributions corresponding to these kernels can be efficiently forecast, opening new perspectives in Gaussian process modeling.

Jean-Michel Loubes

Papers

Using Wasserstein-2 regularization to ensure fair decisions with Neural-Network classifiers.

Apprentissage sur Données Massives; trois cas d'usage avec R, Python et Spark.

On the coalitional decomposition of parameters of interest

Method for estimating a journey time of a vehicle on a road network

A Gaussian Process Regression Model for Distribution Inputs