Scikit-learn is a Python module integrating a wide range of state-of-the-art machine learning algorithms for medium-scale supervised and unsupervised problems. This package focuses on bringing machine learning to non-specialists using a general-purpose high-level language. Emphasis is put on ease of use, performance, documentation, and API consistency. It has minimal dependencies and is distributed under the simplified BSD license, encouraging its use in both academic and commercial settings. Source code, binaries, and documentation can be downloaded from http://scikit-learn.sourceforge.net.

/pdf/scikit-learn-machine-learning-in-python-23clfh9q83.pdf

Scikit-learn: Machine Learning in Python

Summary. We propose the elastic net, a new regularization and variable selection method. Real world data and a simulation study show that the elastic net often outperforms the lasso, while enjoying a similar sparsity of representation. In addition, the elastic net encourages a grouping effect, where strongly correlated predictors tend to be in or out of the model together.The elastic net is particularly useful when the number of predictors (p) is much bigger than the number of observations (n). By contrast, the lasso is not a very satisfactory variable selection method in the

/pdf/regularization-and-variable-selection-via-the-elastic-net-4z8q6j5o0i.pdf

Regularization and variable selection via the elastic net

We develop fast algorithms for estimation of generalized linear models with convex penalties. The models include linear regression, two-class logistic regression, and multinomial regression problems while the penalties include l(1) (the lasso), l(2) (ridge regression) and mixtures of the two (the elastic net). The algorithms use cyclical coordinate descent, computed along a regularization path. The methods can handle large problems and can also deal efficiently with sparse features. In comparative timings we find that the new algorithms are considerably faster than competing methods.

/pdf/regularization-paths-for-generalized-linear-models-via-39r1t606ac.pdf

Regularization Paths for Generalized Linear Models via Coordinate Descent

Despite widespread adoption, machine learning models remain mostly black boxes. Understanding the reasons behind predictions is, however, quite important in assessing trust, which is fundamental if one plans to take action based on a prediction, or when choosing whether to deploy a new model. Such understanding also provides insights into the model, which can be used to transform an untrustworthy model or prediction into a trustworthy one. In this work, we propose LIME, a novel explanation technique that explains the predictions of any classifier in an interpretable and faithful manner, by learning an interpretable model locally varound the prediction. We also propose a method to explain models by presenting representative individual predictions and their explanations in a non-redundant way, framing the task as a submodular optimization problem. We demonstrate the flexibility of these methods by explaining different models for text (e.g. random forests) and image classification (e.g. neural networks). We show the utility of explanations via novel experiments, both simulated and with human subjects, on various scenarios that require trust: deciding if one should trust a prediction, choosing between models, improving an untrustworthy classifier, and identifying why a classifier should not be trusted.

/pdf/why-should-i-trust-you-explaining-the-predictions-of-any-ejtlrtykxr.pdf

"Why Should I Trust You?": Explaining the Predictions of Any Classifier

This paper demonstrates theoretically and empirically that a greedy algorithm called orthogonal matching pursuit (OMP) can reliably recover a signal with m nonzero entries in dimension d given O(m ln d) random linear measurements of that signal. This is a massive improvement over previous results, which require O(m2) measurements. The new results for OMP are comparable with recent results for another approach called basis pursuit (BP). In some settings, the OMP algorithm is faster and easier to implement, so it is an attractive alternative to BP for signal recovery problems.

/pdf/signal-recovery-from-random-measurements-via-orthogonal-5adhd8dwmq.pdf

Signal Recovery From Random Measurements Via Orthogonal Matching Pursuit

We provide the asymptotic distribution of the major indexes used in the statistical literature to quantify disparate treatment in machine learning. We aim at promoting the use of confidence intervals when testing the so-called group disparate impact. We illustrate on some examples the importance of using confidence intervals and not a single value.

Confidence Intervals for Testing Disparate Impact in Fair Learning

In this paper, we propose a dimension reduction model for spatially dependent variables Namely, we investigate an extension of the \emph{inverse regression} method under strong mixing condition This method is based on estimation of the matrix of covariance of the expectation of the explanatory given the dependent variable, called the \emph{inverse regression} Then, we study, under strong mixing condition, the weak and strong consistency of this estimate, using a kernel estimate of the \emph{inverse regression} We provide the asymptotic behaviour of this estimate A spatial predictor based on this dimension reduction approach is also proposed This latter appears as an alternative to the spatial non-parametric predictor

https://hal.archives-ouvertes.fr/hal-00347813/document

Kernel Inverse Regression for spatial random fields

Today the integration of Artificial Intelligence (AI) solutions is part of the strategy in the industrial environment. We focus on anomaly detection in the framework of manufacturing electronic cards manufacturing under mass production conditions (24/7). Early anomaly detection is critical to avoid defects. Researches and applications of anomaly detection techniques in the industry have been published but when they face production constraints success is not guaranteed. Today’s manufacturing systems are complex and involve different behaviors. We propose and evaluate a new realistic methodology for detecting conditional anomalies that could be successfully implemented in production. The proposed solution is based on Variational Autoencoders (VAEs) which provide interesting scores under the near real-time constraints of the production environment. The results have been thoroughly evaluated and validated with the support of expert process engineers.

Conditional Anomaly Detection for Quality and Productivity Improvement of Electronics Manufacturing Systems

Quantifying the impact of public health protection measures on the spread of SARS-CoV-2.

We tackle the problem of recovering an unknown signal observed in an ill-posed inverse problem framework. More precisely, we study a procedure commonly used in numerical analysis or image deblurring: minimizing an empirical loss function balanced by an l 1 penalty, acting as a sparsity constraint. We prove that, by choosing a proper loss function, this estimation technique enables to build an adaptive estimator, in the sense that it converges at the optimal rate of convergence without prior knowledge of the regularity of the true solution.

Jean-Michel Loubes

Papers

Confidence Intervals for Testing Disparate Impact in Fair Learning

Kernel Inverse Regression for spatial random fields

Conditional Anomaly Detection for Quality and Productivity Improvement of Electronics Manufacturing Systems

Quantifying the impact of public health protection measures on the spread of SARS-CoV-2.

ℓ1 Penalty for Ill-Posed Inverse Problems