Open AccessPosted Content
Evaluating the Correctness of Explainable AI Algorithms for Classification.
Reads0
Chats0
TLDR
This article developed a method to quantitatively evaluate the correctness of XAI algorithms by creating datasets with known explanation ground truth for binary classification problems and found that classification accuracy is positively correlated with explanation accuracy.Abstract:
Explainable AI has attracted much research attention in recent years with feature attribution algorithms, which compute "feature importance" in predictions, becoming increasingly popular. However, there is little analysis of the validity of these algorithms as there is no "ground truth" in the existing datasets to validate their correctness. In this work, we develop a method to quantitatively evaluate the correctness of XAI algorithms by creating datasets with known explanation ground truth. To this end, we focus on the binary classification problems. String datasets are constructed using formal language derived from a grammar. A string is positive if and only if a certain property is fulfilled. Symbols serving as explanation ground truth in a positive string are part of an explanation if and only if they contributes to fulfilling the property. Two popular feature attribution explainers, Local Interpretable Model-agnostic Explanations (LIME) and SHapley Additive exPlanations (SHAP), are used in our experiments.We show that: (1) classification accuracy is positively correlated with explanation accuracy; (2) SHAP provides more accurate explanations than LIME; (3) explanation accuracy is negatively correlated with dataset complexity.read more
Citations
More filters
Posted Content
Order in the Court: Explainable AI Methods Prone to Disagreement.
TL;DR: The authors compare LIME, Integrated Gradients, DeepLIFT, Grad-SHAP, Deep-SHAPE, and attention-based explanations, applied to two neural architectures trained on single and pair-sequence language tasks.
Book ChapterDOI
An Initial Study of Machine Learning Underspecification Using Feature Attribution Explainable AI Algorithms: A COVID-19 Virus Transmission Case Study
James Hinns,Xiuyi Fan,Siyuan Liu,Veera Raghava Reddy Kovvuri,Mehmet Orcun Yalcin,Markus Roggenbach +5 more
TL;DR: In this article, the authors propose to identify underspecification using feature attribution algorithms developed in Explainable AI. But their work is limited to a single dataset and does not consider unseen data.
Journal ArticleDOI
A Perspective on Explanations of Molecular Prediction Models
TL;DR: In this article , explainable artificial intelligence (XAI) is applied to predict solubility, blood-brain barrier permeability, and the scent of molecules in deep learning models.
Posted Content
Developing a Fidelity Evaluation Approach for Interpretable Machine Learning.
TL;DR: In this paper, a three phase approach is proposed to evaluate the fidelity of the explanation to the underlying black box for tabular data, and two popular explainable methods using this evaluation method.
References
More filters
Proceedings ArticleDOI
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
TL;DR: In this article, the authors propose LIME, a method to explain models by presenting representative individual predictions and their explanations in a non-redundant way, framing the task as a submodular optimization problem.
Proceedings Article
A unified approach to interpreting model predictions
Scott M. Lundberg,Su-In Lee +1 more
TL;DR: In this article, a unified framework for interpreting predictions, SHAP (SHapley Additive exPlanations), is presented, which assigns each feature an importance value for a particular prediction.
Book
An Introduction to Kolmogorov Complexity and Its Applications
Ming Li,Paul M. B. Vitányi +1 more
TL;DR: The Journal of Symbolic Logic as discussed by the authors presents a thorough treatment of the subject with a wide range of illustrative applications such as the randomness of finite objects or infinite sequences, Martin-Loef tests for randomness, information theory, computational learning theory, the complexity of algorithms, and the thermodynamics of computing.
Journal ArticleDOI
Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead
TL;DR: This Perspective clarifies the chasm between explaining black boxes and using inherently interpretable models, outlines several key reasons why explainable black boxes should be avoided in high-stakes decisions, identifies challenges to interpretable machine learning, and provides several example applications whereinterpretable models could potentially replace black box models in criminal justice, healthcare and computer vision.