Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI

At the dawn of the fourth industrial revolution, we are witnessing a fast and widespread adoption of artificial intelligence (AI) in our daily life, which contributes to accelerating the shift towards a more algorithmic society. However, even with such unprecedented advancements, a key impediment to the use of AI-based systems is that they often lack transparency. Indeed, the black-box nature of these systems allows powerful predictions, but it cannot be directly explained. This issue has triggered a new debate on explainable AI (XAI). A research field holds substantial promise for improving trust and transparency of AI-based systems. It is recognized as the sine qua non for AI to continue making steady progress without disruption. This survey provides an entry point for interested researchers and practitioners to learn key aspects of the young and rapidly growing body of research related to XAI. Through the lens of the literature, we review the existing approaches regarding the topic, discuss trends surrounding its sphere, and present major research trajectories.

Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)

Recent research has revealed that the output of deep neural networks (DNNs) can be easily altered by adding relatively small perturbations to the input vector. In this paper, we analyze an attack in an extremely limited scenario where only one pixel can be modified. For that we propose a novel method for generating one-pixel adversarial perturbations based on differential evolution (DE). It requires less adversarial information (a black-box attack) and can fool more types of networks due to the inherent features of DE. The results show that 67.97% of the natural images in Kaggle CIFAR-10 test dataset and 16.04% of the ImageNet (ILSVRC 2012) test images can be perturbed to at least one target class by modifying just one pixel with 74.03% and 22.91% confidence on average. We also show the same vulnerability on the original CIFAR-10 dataset. Thus, the proposed attack explores a different take on adversarial machine learning in an extreme limited scenario, showing that current DNNs are also vulnerable to such low dimension attacks. Besides, we also illustrate an important application of DE (or broadly speaking, evolutionary computation) in the domain of adversarial machine learning: creating tools that can effectively generate low-cost adversarial attacks against neural networks for evaluating robustness.

One Pixel Attack for Fooling Deep Neural Networks

Machine learning is evolved from a collection of powerful techniques in AI areas and has been extensively used in data mining, which allows the system to learn the useful structural patterns and models from training data Machine learning algorithms can be basically classified into four categories: supervised, unsupervised, semi-supervised and reinforcement learning In this chapter, widely-used machine learning algorithms are introduced Each algorithm is briefly explained with some examples

Introduction to Machine Learning

In the last years, Artificial Intelligence (AI) has achieved a notable momentum that may deliver the best of expectations over many application sectors across the field. For this to occur, the entire community stands in front of the barrier of explainability, an inherent problem of AI techniques brought by sub-symbolism (e.g. ensembles or Deep Neural Networks) that were not present in the last hype of AI. Paradigms underlying this problem fall within the so-called eXplainable AI (XAI) field, which is acknowledged as a crucial feature for the practical deployment of AI models. This overview examines the existing literature in the field of XAI, including a prospect toward what is yet to be reached. We summarize previous efforts to define explainability in Machine Learning, establishing a novel definition that covers prior conceptual propositions with a major focus on the audience for which explainability is sought. We then propose and discuss about a taxonomy of recent contributions related to the explainability of different Machine Learning models, including those aimed at Deep Learning methods for which a second taxonomy is built. This literature analysis serves as the background for a series of challenges faced by XAI, such as the crossroads between data fusion and explainability. Our prospects lead toward the concept of Responsible Artificial Intelligence, namely, a methodology for the large-scale implementation of AI methods in real organizations with fairness, model explainability and accountability at its core. Our ultimate goal is to provide newcomers to XAI with a reference material in order to stimulate future research advances, but also to encourage experts and professionals from other disciplines to embrace the benefits of AI in their activity sectors, without any prior bias for its lack of interpretability.

Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI.

With rapid progress and significant successes in a wide spectrum of applications, deep learning is being applied in many safety-critical environments. However, deep neural networks (DNNs) have been recently found vulnerable to well-designed input samples called adversarial examples . Adversarial perturbations are imperceptible to human but can easily fool DNNs in the testing/deploying stage. The vulnerability to adversarial examples becomes one of the major risks for applying DNNs in safety-critical environments. Therefore, attacks and defenses on adversarial examples draw great attention. In this paper, we review recent findings on adversarial examples for DNNs, summarize the methods for generating adversarial examples, and propose a taxonomy of these methods. Under the taxonomy, applications for adversarial examples are investigated. We further elaborate on countermeasures for adversarial examples. In addition, three major challenges in adversarial examples and the potential solutions are discussed.

Adversarial Examples: Attacks and Defenses for Deep Learning

With rapid progress and significant successes in a wide spectrum of applications, deep learning is being applied in many safety-critical environments. However, deep neural networks have been recently found vulnerable to well-designed input samples, called adversarial examples. Adversarial examples are imperceptible to human but can easily fool deep neural networks in the testing/deploying stage. The vulnerability to adversarial examples becomes one of the major risks for applying deep neural networks in safety-critical environments. Therefore, attacks and defenses on adversarial examples draw great attention. In this paper, we review recent findings on adversarial examples for deep neural networks, summarize the methods for generating adversarial examples, and propose a taxonomy of these methods. Under the taxonomy, applications for adversarial examples are investigated. We further elaborate on countermeasures for adversarial examples and explore the challenges and the potential solutions.

Distributed Denial of Service (DDoS) attacks grow rapidly and become one of the fatal threats to the Internet. Automatically detecting DDoS attack packets is one of the main defense mechanisms. Conventional solutions monitor network traffic and identify attack activities from legitimate network traffic based on statistical divergence. Machine learning is another method to improve identifying performance based on statistical features. However, conventional machine learning techniques are limited by the shallow representation models. In this paper, we propose a deep learning based DDoS attack detection approach (DeepDefense). Deep learning approach can automatically extract high-level features from low-level ones and gain powerful representation and inference. We design a recurrent deep neural network to learn patterns from sequences of network traffic and trace network attack activities. The experimental results demonstrate a better performance of our model compared with conventional machine learning models. We reduce the error rate from 7.517% to 2.103% compared with conventional machine learning method in the larger data set.

DeepDefense: Identifying DDoS Attack via Deep Learning

Detection and defense of DDoS attack–based on deep learning in OpenFlow-based SDN

Protecting computer systems is a critical and ongoing problem, given that real-time malware detection is hard. The state-of-the-art for defense cannot keep pace with the increasing level of sophistication of malware. The industry, for instance, relies heavily on anti-virus technology for threat, which is effective for malware with known signatures, but not sustainable given the massive amount of malware samples released daily, as well as and its inefficacy in dealing with zero-day and polymorphic/metamorphic malware (practical detection rates range from 25% to 50%). Behavior-based approaches attempt to identify malware behaviors using instruction sequences, computation trace logic, and system (or API) call sequences. These solutions have been mostly based on conventional machine learning (ML) models with hand-craft features, such as K-nearest neighbor, SVM, and decision tree algorithms. However, current solutions based on ML suffer from high false-positive rates, mainly because of (i) the complexity and diversity of current software and malware, which are hard to capture during the learning phase of thealgorithms, (ii) sub-optimal feature extraction, and (iii) limited/outdated dataset. Since malware has been continuously evolving, existing protection mechanisms do not cope well with the increasedsophistication and complexity of these attacks, especially those performed by advanced persistent threats (APT), which are multi-module, stealthy, and target- focused. Furthermore, malware campaigns are not homogeneous--malware sophistication varies depending on the target, the type of service exploited as part of the attack (e.g., Internet Banking, relationship sites), the attack spreading source (e.g., phishing, drive-by downloads), and the location of the target. The accuracy of malware classification depends on gaining sufficient context information and extracting meaningful abstraction of behaviors. In problems about detecting malicious behavior based on sequence of system calls, longer sequences likely contain more information. However, classical ML- based detectors (i.e., Random Forest, Naive Bayes) often use short windows of system calls during the decision process and may not be able to extract enough features for accurate detection in a long term window. Thus, the main drawback of such approaches is to accomplish accurate detection, since it is difficult to analyze complex and longer sequences of malicious behaviors with limited window sizes, especially when malicious and benign behaviors are interposed. In contrast, Deep Learning models are capable of analyzing longer sequences of system calls and making better decisions through higher level information extraction and semantic knowledge learning. However, Deep Learning requires more computation time to estimate the probability of detection when the model needs to be retrained incrementally, a common requirement for malware detection when new variants and samples are frequently added to the training set. The trade-off is challenging: fast and not-so-accurate (classical ML methods) versus time-consuming and accurate detection (emerging Deep Learning methods). Our proposal is to leverage the best of the two worlds with Spectrum, a practical multi-stage malware- detection system operating in collaboration with the operating system (OS).

Xiaoyong Yuan

Papers

Adversarial Examples: Attacks and Defenses for Deep Learning

Adversarial Examples: Attacks and Defenses for Deep Learning

DeepDefense: Identifying DDoS Attack via Deep Learning

Detection and defense of DDoS attack–based on deep learning in OpenFlow-based SDN

PhD Forum: Deep Learning-Based Real-Time Malware Detection with Multi-Stage Analysis