In the healthcare domain, a transformative shift is envisioned towards Healthcare 5.0. It expands the operational boundaries of Healthcare 4.0 and leverages patient-centric digital wellness. Healthcare 5.0 focuses on real-time patient monitoring, ambient control and wellness, and privacy compliance through assisted technologies like artificial intelligence (AI), Internet-of-Things (IoT), big data, and assisted networking channels. However, healthcare operational procedures, verifiability of prediction models, resilience, and lack of ethical and regulatory frameworks are potential hindrances to the realization of Healthcare 5.0. Recently, explainable AI (EXAI) has been a disruptive trend in AI that focuses on the explainability of traditional AI models by leveraging the decision-making of the models and prediction outputs. The explainability factor opens new opportunities to the black-box models and brings confidence in healthcare stakeholders to interpret the machine learning (ML) and deep learning (DL) models. EXAI is focused on improving clinical health practices and brings transparency to the predictive analysis, which is crucial in the healthcare domain. Recent surveys on EXAI in healthcare have not significantly focused on the data analysis and interpretation of models, which lowers its practical deployment opportunities. Owing to the gap, the proposed survey explicitly details the requirements of EXAI in Healthcare 5.0, the operational and data collection process. Based on the review method and presented research questions, systematically, the article unfolds a proposed architecture that presents an EXAI ensemble on the computerized tomography (CT) image classification and segmentation process. A solution taxonomy of EXAI in Healthcare 5.0 is proposed, and operational challenges are presented. A supported case study on electrocardiogram (ECG) monitoring is presented that preserves the privacy of local models via federated learning (FL) and EXAI for metric validation. The case-study is supported through experimental validation. The analysis proves the efficacy of EXAI in health setups that envisions real-life model deployments in a wide range of clinical applications.

/pdf/explainable-ai-for-healthcare-5-0-opportunities-and-1n7he1lb.pdf

Explainable AI for Healthcare 5.0: Opportunities and Challenges

Abstract Due to computational advances in the past decades, so-called intelligent systems can learn from increasingly complex data, analyze situations, and support users in their decision-making to address them. However, in practice, the complexity of these intelligent systems renders the user hardly able to comprehend the inherent decision logic of the underlying machine learning model. As a result, the adoption of this technology, especially for high-stake scenarios, is hampered. In this context, explainable artificial intelligence offers numerous starting points for making the inherent logic explainable to people. While research manifests the necessity for incorporating explainable artificial intelligence into intelligent systems, there is still a lack of knowledge about how to socio-technically design these systems to address acceptance barriers among different user groups. In response, we have derived and evaluated a nascent design theory for explainable intelligent systems based on a structured literature review, two qualitative expert studies, a real-world use case application, and quantitative research. Our design theory includes design requirements, design principles, and design features covering the topics of global explainability, local explainability, personalized interface design, as well as psychological/emotional factors. 

/pdf/a-nascent-design-theory-for-explainable-intelligent-systems-24ww4608.pdf

A nascent design theory for explainable intelligent systems

Deep learning models have been increasingly applied to medical images for tasks such as lesion detection, segmentation, and diagnosis. However, the field suffers from the lack of concrete definitions for usable explanations in different settings. To identify specific aspects of explainability that may catalyse building trust in deep learning models, we will use some techniques to demonstrate many aspects of explaining convolutional neural networks in a medical imaging context. One important factor influencing clinician’s trust is how well a model can justify its predictions or outcomes. Clinicians need understandable explanations about why a machine-learned prediction was made so they can assess whether it is accurate and clinically useful. The provision of appropriate explanations has been generally understood to be critical for establishing trust in deep learning models. However, there lacks a clear understanding on what constitutes an explanation that is both understandable and useful across different domains such as medical image analysis, which hampers efforts towards developing explanatory tool sets specifically tailored towards these tasks. In this paper, we investigated two major directions for explaining convolutional neural networks: feature-based post hoc explanatory methods that try to explain already trained and fixed target models and preliminary analysis and choice of the model architecture with an accuracy of 98% ± 0.156% from 36 CNN architectures with different configurations. 

/pdf/cxai-explaining-convolutional-neural-networks-for-medical-erm92vur.pdf

CXAI: Explaining Convolutional Neural Networks for Medical Imaging Diagnostic

Accurate traffic forecasting is vital to an intelligent transportation system. Although many deep learning models have achieved state-of-art performance for short-term traffic forecasting of up to 1 hour, long-term traffic forecasting that spans multiple hours remains a major challenge. Moreover, most of the existing deep learning traffic forecasting models are black box, presenting additional challenges related to explainability and interpretability. We develop Graph Pyramid Autoformer (X-GPA), an explainable attention-based spatial-temporal graph neural network that uses a novel pyramid autocorrelation attention mechanism. It enables learning from long temporal sequences on graphs and improves long-term traffic forecasting accuracy. Our model can achieve up to 35 % better long-term traffic forecast accuracy than that of several state-of-the-art methods. The attention-based scores from the X-GPA model provide spatial and temporal explanations based on the traffic dynamics, which change for normal vs. peak-hour traffic and weekday vs. weekend traffic.

Explainable Graph Pyramid Autoformer for Long-Term Traffic Forecasting

Research has demonstrated that algorithmic and AI (AAI) technologies produce significant social harms. This short intervention turns the focus from social harm production to social harm reduction and seeks to map how and what AAI technology features affect harm reduction efforts by AAI system designers.
The intervention argues that six cumulative features are relevant. The technological agency embedded in AAI systems makes conscious harm reduction by design possible but also potentially strips previous human harm safeguards. Complexity destabilises designers’ capability to predict and control system performance, justify outcomes, and analyse and ensure the legitimacy system ontologies. Uninterpretability further complicates harm-reduction efforts by making analytical tracing of system logics difficult and introducing alien ontologies. Non-linear performance further destabilises outcome prediction capabilities, while indeterminacy and dynamicity provide the ultimate challenges for harm reduction. If the AAI systems performance patterns are indeterminate in nature or prone to change during use due to lifelong learning, designers lose direct control over the systems. While the technology features challenge harm reduction, concerted efforts in managing and containing their effects allow designers to engage in harm reduction.

Technology matters: how algorithm and artificial intelligent technology features affect harms reduction efforts

Organizations are increasingly employing complex black-box machine learning models in high-stakes decision-making. A popular approach to addressing the problem of opacity of black-box machine learning models is the use of post-hoc explainability methods. These methods approximate the logic of underlying machine learning models with the aim of explaining their internal workings, so that human examiners can understand them. In turn, it has been alluded that the insights from post-hoc explainability methods can be used to help regulate black-box machine learning. This article examines the validity of these claims. By examining whether the insights derived from post-hoc explainability methods in post-model deployment can prima facie meet legal definitions in European (read European Union) non-discrimination law, we argue that machine learning post-hoc explanation methods cannot guarantee the insights they generate. Ultimately, we argue that the use of post-hoc explanatory methods is useful in many cases, but that these methods have limitations that prohibit reliance as the sole mechanism to guarantee fairness of model outcomes in high-stakes decision-making. By way of an ancillary function, the inadequacy of European Non-Discrimination Law for algorithmic decision-making is demonstrated too. 

https://link.springer.com/content/pdf/10.1007/s43681-022-00142-y.pdf

Muhammed Ali

Papers

Explainable artificial intelligence (XAI) post-hoc explainability methods: risks and limitations in non-discrimination law