Evaluating the Quality of Machine Learning Explanations: A Survey on Methods and Metrics
TLDR
A comprehensive overview of methods proposed in the current literature for the evaluation of ML explanations is presented, finding that the quantitative metrics for both model-based and example-based explanations are primarily used to evaluate the parsimony/simplicity of interpretability, and subjective measures have been embraced as the focal point for the human-centered evaluation of explainable systems.Abstract:
The most successful Machine Learning (ML) systems remain complex black boxes to end-users, and even experts are often unable to understand the rationale behind their decisions. The lack of transparency of such systems can have severe consequences or poor uses of limited valuable resources in medical diagnosis, financial decision-making, and in other high-stake domains. Therefore, the issue of ML explanation has experienced a surge in interest from the research community to application domains. While numerous explanation methods have been explored, there is a need for evaluations to quantify the quality of explanation methods to determine whether and to what extent the offered explainability achieves the defined objective, and compare available explanation methods and suggest the best explanation from the comparison for a specific task. This survey paper presents a comprehensive overview of methods proposed in the current literature for the evaluation of ML explanations. We identify properties of explainability from the review of definitions of explainability. The identified properties of explainability are used as objectives that evaluation metrics should achieve. The survey found that the quantitative metrics for both model-based and example-based explanations are primarily used to evaluate the parsimony/simplicity of interpretability, while the quantitative metrics for attribution-based explanations are primarily used to evaluate the soundness of fidelity of explainability. The survey also demonstrated that subjective measures, such as trust and confidence, have been embraced as the focal point for the human-centered evaluation of explainable systems. The paper concludes that the evaluation of ML explanations is a multidisciplinary research topic. It is also not possible to define an implementation of evaluation metrics, which can be applied to all explanation methods.read more
Citations
More filters
Journal ArticleDOI
Current Challenges and Future Opportunities for XAI in Machine Learning-Based Clinical Decision Support Systems: A Systematic Review
Anna Markella Antoniadi,Yuhan Du,Yasmine Guendouz,Lan Wei,Claudia Mazo,Brett A. Becker,Catherine Mooney +6 more
TL;DR: An overall distinct lack of application of XAI is found in the context of CDSS and, in particular, a lack of user studies exploring the needs of clinicians is found.
Journal Article
The Disagreement Problem in Explainable Machine Learning: A Practitioner's Perspective
Satya Sai Krishna,Tessa Han,Alex Gu,Javin Pombra,Shahin Jabbari,Steven C. Wu,Himabindu Lakkaraju +6 more
TL;DR: This work introduces and study the disagreement problem in explainable machine learning, formalizes the notion of disagreement between explanations, and analyzes how often such disagreements occur in practice, and how do practitioners resolve these disagreements.
Journal ArticleDOI
Research and Application of Machine Learning for Additive Manufacturing
Jian Qin,Fu Hu,Yang Liu,Paul Witherell,Charlie C. L. Wang,David W. Rosen,T.E. Simpson,Yuan Lu,Qian Tang +8 more
TL;DR: In this article , the authors employ a systematic literature review method to identify, assess, and analyse published literature on additive manufacturing, including design for additive manufacturing (DfAM), material analytics, in situ monitoring and defect detection, property prediction and sustainability.
Posted Content
Transparency of Deep Neural Networks for Medical Image Analysis: A Review of Interpretability Methods.
TL;DR: In this article, a narrative review of interpretability methods for deep learning models for medical image analysis applications is presented, which is based on the type of generated explanations and technical similarities.
Journal ArticleDOI
Transparency of deep neural networks for medical image analysis: A review of interpretability methods
TL;DR: In this paper , the authors identify nine different interpretability methods that have been used for understanding deep learning models for medical image analysis applications based on the type of generated explanations and technical similarities.
References
More filters
Posted Content
Distilling the Knowledge in a Neural Network
TL;DR: This work shows that it can significantly improve the acoustic model of a heavily used commercial system by distilling the knowledge in an ensemble of models into a single model and introduces a new type of ensemble composed of one or more full models and many specialist models which learn to distinguish fine-grained classes that the full models confuse.
Proceedings ArticleDOI
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
TL;DR: In this article, the authors propose LIME, a method to explain models by presenting representative individual predictions and their explanations in a non-redundant way, framing the task as a submodular optimization problem.
Journal ArticleDOI
Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead
TL;DR: This Perspective clarifies the chasm between explaining black boxes and using inherently interpretable models, outlines several key reasons why explainable black boxes should be avoided in high-stakes decisions, identifies challenges to interpretable machine learning, and provides several example applications whereinterpretable models could potentially replace black box models in criminal justice, healthcare and computer vision.
Journal ArticleDOI
A Survey of Methods for Explaining Black Box Models
Riccardo Guidotti,Anna Monreale,Salvatore Ruggieri,Franco Turini,Fosca Giannotti,Dino Pedreschi +5 more
TL;DR: In this paper, the authors provide a classification of the main problems addressed in the literature with respect to the notion of explanation and the type of black box decision support systems, given a problem definition, a black box type, and a desired explanation, this survey should help the researcher to find the proposals more useful for his own work.
Posted Content
Towards A Rigorous Science of Interpretable Machine Learning
Finale Doshi-Velez,Been Kim +1 more
TL;DR: This position paper defines interpretability and describes when interpretability is needed (and when it is not), and suggests a taxonomy for rigorous evaluation and exposes open questions towards a more rigorous science of interpretable machine learning.
Related Papers (5)
Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)
Amina Adadi,Mohammed Berrada +1 more