Tree boosting is a highly effective and widely used machine learning method. In this paper, we describe a scalable end-to-end tree boosting system called XGBoost, which is used widely by data scientists to achieve state-of-the-art results on many machine learning challenges. We propose a novel sparsity-aware algorithm for sparse data and weighted quantile sketch for approximate tree learning. More importantly, we provide insights on cache access patterns, data compression and sharding to build a scalable tree boosting system. By combining these insights, XGBoost scales beyond billions of examples using far fewer resources than existing systems.

/pdf/xgboost-a-scalable-tree-boosting-system-48ocu6x4c7.pdf

XGBoost: A Scalable Tree Boosting System

THE DESIGN AND ANALYSIS OF EXPERIMENTS. By Oscar Kempthorne. New York, John Wiley and Sons, Inc., 1952. 631 pp. $8.50. This book by a teacher of statistics (as well as a consultant for \"experimenters\") is a comprehensive study of the philosophical background for the statistical design of experiment. It is necessary to have some facility with algebraic notation and manipulation to be able to use the volume intelligently. The problems are presented from the theoretical point of view, without such practical examples as would be helpful for those not acquainted with mathematics. The mathematical justification for the techniques is given. As a somewhat advanced treatment of the design and analysis of experiments, this volume will be interesting and helpful for many who approach statistics theoretically as well as practically. With emphasis on the \"why,\" and with description given broadly, the author relates the subject matter to the general theory of statistics and to the general problem of experimental inference. MARGARET J. ROBERTSON

The Design and Analysis of Experiments

SciPy is an open source scientific computing library for the Python programming language. SciPy 1.0 was released in late 2017, about 16 years after the original version 0.1 release. SciPy has become a de facto standard for leveraging scientific algorithms in the Python programming language, with more than 600 unique code contributors, thousands of dependent packages, over 100,000 dependent repositories, and millions of downloads per year. This includes usage of SciPy in almost half of all machine learning projects on GitHub, and usage by high profile projects including LIGO gravitational wave analysis and creation of the first-ever image of a black hole (M87). The library includes functionality spanning clustering, Fourier transforms, integration, interpolation, file I/O, linear algebra, image processing, orthogonal distance regression, minimization algorithms, signal processing, sparse matrix handling, computational geometry, and statistics. In this work, we provide an overview of the capabilities and development practices of the SciPy library and highlight some recent technical developments.

/pdf/scipy-1-0-fundamental-algorithms-for-scientific-computing-in-111lr790ol.pdf

SciPy 1.0--Fundamental Algorithms for Scientific Computing in Python

Despite widespread adoption, machine learning models remain mostly black boxes. Understanding the reasons behind predictions is, however, quite important in assessing trust, which is fundamental if one plans to take action based on a prediction, or when choosing whether to deploy a new model. Such understanding also provides insights into the model, which can be used to transform an untrustworthy model or prediction into a trustworthy one. In this work, we propose LIME, a novel explanation technique that explains the predictions of any classifier in an interpretable and faithful manner, by learning an interpretable model locally varound the prediction. We also propose a method to explain models by presenting representative individual predictions and their explanations in a non-redundant way, framing the task as a submodular optimization problem. We demonstrate the flexibility of these methods by explaining different models for text (e.g. random forests) and image classification (e.g. neural networks). We show the utility of explanations via novel experiments, both simulated and with human subjects, on various scenarios that require trust: deciding if one should trust a prediction, choosing between models, improving an untrustworthy classifier, and identifying why a classifier should not be trusted.

/pdf/why-should-i-trust-you-explaining-the-predictions-of-any-ejtlrtykxr.pdf

"Why Should I Trust You?": Explaining the Predictions of Any Classifier

The optimal and robust design of structures has gained much attention in the past ten years due to the ever increasing need for manufacturers to build robust systems at the lowest cost. Reliability-based design optimization (RBDO) allows the analyst to minimize some cost function while ensuring some minimal performances cast as admissible probabilities of failure for a set of performance functions. In order to address real-world problems in which the performance is assessed through computational models (e.g. large scale finite element models) meta-modelling techniques have been developed in the past decade. This paper introduces adaptive kriging surrogate models to solve the RBDO problem. The latter is cast in an augmented space that "sums up" the range of the design space and the aleatory uncertainty in the design parameters and the environmental conditions. Thus the surrogate model is used (i) for evaluating robust estimates of the probabilities of failure (and for enhancing the computational experimental design by adaptive sampling) in order to achieve the requested accuracy and (ii) for applying the gradient-based optimization algorithm. The approach is applied to the optimal design of imperfect stiffened cylinder shells used in submarine engineering. For this application the performance of the structure is related to buckling which is addressed here by means of the asymptotic numerical method.

Reliability-based design optimization of imperfect shells using adaptive kriging meta-models

https://hal.archives-ouvertes.fr/hal-00592933/document

Modélisation probabiliste de champs d'imperfections géométriques de coques résistantes de sous-marins

Le calcul de la charge critique de flambage
 est realise par la Methode Asymptotique Numerique (MAN) couplee a un modele elements
 finis de coques au sein du code EVE. Le modele stochastique prend en compte la
 variabilite spatiale et aleatoire des proprietes materiaux et de l’epaisseur des coques
 grâce a la theorie des champs aleatoires dont la representation est basee sur leur
 developpement de Karhunen-Loeve. Le defaut de forme initial de la structure est modelise
 comme une combinaison aleatoire de ses modes de flambement les plus critiques.
 L’estimation de la probabilite de defaillance, representative du niveau de surete,
 utilise une methode de simulation a variance reduite connue sous le nom de Subset
 Simulation et implementee dans le code fiabiliste FERUM.

Analyse fiabiliste du flambage des coques avec prise en compte du caractère aléatoire et de la variabilité spatiale des défauts de forme et d'épaisseur, et des propriétés matériaux

Reliability-based design optimization (RBDO) has gained much attention in the past fifteen years as a way of introducing robustness in the process of designing structures and systems in an optimal manner. Indeed classical optimization (e.g. minimize some cost under mechanical constraints) usually leads to solutions that lie at the boundary of the admissible domain, and that are consequently rather sensitive to uncertainty in the design parameters. In contrast, RBDO aims at designing the system in a robust way by minimizing some cost function under reliability constraints. Thus RBDO methods have to mix optimization algorithms together with reliability calculations. The classical approach known as "double-loop" consists in nesting the computation of the failure probability with respect to the current design within the optimization loop. It is not applicable to industrial models (e.g. finite element models) due to the associated computational burden. In contrast, methods based on the approximation of the reliability (e.g. FORM) may not be applicable to real-world problems. In this context, an original method has been developed that tries to circumvent the abovementioned drawbacks of the existing approaches. It is based on the adaptive construction of a meta-model for the expensive-to-evaluate mechanical model, and on the subset simulation technique for the efficient and accurate computation of the failure probability and its sensitivities with respect to the design variables. The proposed methodology is briefly described in this paper before it is applied to the reliability-based design of an imperfect submarine pressure hull.

Vincent Dubourg

Papers

Reliability-based design optimization of imperfect shells using adaptive kriging meta-models

Modélisation probabiliste de champs d'imperfections géométriques de coques résistantes de sous-marins

Analyse fiabiliste du flambage des coques avec prise en compte du caractère aléatoire et de la variabilité spatiale des défauts de forme et d'épaisseur, et des propriétés matériaux

Reliability-based design optimization of an imperfect submarine pressure hull

Meta-model-based importance for reliability sensitivity analysis