In the last years, Artificial Intelligence (AI) has achieved a notable momentum that may deliver the best of expectations over many application sectors across the field. For this to occur, the entire community stands in front of the barrier of explainability, an inherent problem of AI techniques brought by sub-symbolism (e.g. ensembles or Deep Neural Networks) that were not present in the last hype of AI. Paradigms underlying this problem fall within the so-called eXplainable AI (XAI) field, which is acknowledged as a crucial feature for the practical deployment of AI models. This overview examines the existing literature in the field of XAI, including a prospect toward what is yet to be reached. We summarize previous efforts to define explainability in Machine Learning, establishing a novel definition that covers prior conceptual propositions with a major focus on the audience for which explainability is sought. We then propose and discuss about a taxonomy of recent contributions related to the explainability of different Machine Learning models, including those aimed at Deep Learning methods for which a second taxonomy is built. This literature analysis serves as the background for a series of challenges faced by XAI, such as the crossroads between data fusion and explainability. Our prospects lead toward the concept of Responsible Artificial Intelligence, namely, a methodology for the large-scale implementation of AI methods in real organizations with fairness, model explainability and accountability at its core. Our ultimate goal is to provide newcomers to XAI with a reference material in order to stimulate future research advances, but also to encourage experts and professionals from other disciplines to embrace the benefits of AI in their activity sectors, without any prior bias for its lack of interpretability.

Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI.

Artificial intelligence (AI) research in healthcare is accelerating rapidly, with potential applications being demonstrated across various domains of medicine. However, there are currently limited examples of such techniques being successfully deployed into clinical practice. This article explores the main challenges and limitations of AI in healthcare, and considers the steps required to translate these potentially transformative technologies from research to clinical practice. Key challenges for the translation of AI systems in healthcare include those intrinsic to the science of machine learning, logistical difficulties in implementation, and consideration of the barriers to adoption as well as of the necessary sociocultural or pathway changes. Robust peer-reviewed clinical evaluation as part of randomised controlled trials should be viewed as the gold standard for evidence generation, but conducting these in practice may not always be appropriate or feasible. Performance metrics should aim to capture real clinical applicability and be understandable to intended users. Regulation that balances the pace of innovation with the potential for harm, alongside thoughtful post-market surveillance, is required to ensure that patients are not exposed to dangerous interventions nor deprived of access to beneficial innovations. Mechanisms to enable direct comparisons of AI systems must be developed, including the use of independent, local and representative test sets. Developers of AI algorithms must be vigilant to potential dangers, including dataset shift, accidental fitting of confounders, unintended discriminatory bias, the challenges of generalisation to new populations, and the unintended negative consequences of new algorithms on health outcomes. The safe and timely translation of AI research into clinically validated and appropriately regulated systems that can benefit everyone is challenging. Robust clinical evaluation, using metrics that are intuitive to clinicians and ideally go beyond measures of technical accuracy to include quality of care and patient outcomes, is essential. Further work is required (1) to identify themes of algorithmic bias and unfairness while developing mitigations to address these, (2) to reduce brittleness and improve generalisability, and (3) to develop methods for improved interpretability of machine learning predictions. If these goals can be achieved, the benefits for patients are likely to be transformational.

/pdf/key-challenges-for-delivering-clinical-impact-with-4ha0epal3a.pdf

Key challenges for delivering clinical impact with artificial intelligence.

The nascent field of fair machine learning aims to ensure that decisions guided by algorithms are equitable. Over the last several years, three formal definitions of fairness have gained prominence: (1) anti-classification, meaning that protected attributes---like race, gender, and their proxies---are not explicitly used to make decisions; (2) classification parity, meaning that common measures of predictive performance (e.g., false positive and false negative rates) are equal across groups defined by the protected attributes; and (3) calibration, meaning that conditional on risk estimates, outcomes are independent of protected attributes. Here we show that all three of these fairness definitions suffer from significant statistical limitations. Requiring anti-classification or classification parity can, perversely, harm the very groups they were designed to protect; and calibration, though generally desirable, provides little guarantee that decisions are equitable. In contrast to these formal fairness criteria, we argue that it is often preferable to treat similarly risky people similarly, based on the most statistically accurate estimates of risk that one can produce. Such a strategy, while not universally applicable, often aligns well with policy objectives; notably, this strategy will typically violate both anti-classification and classification parity. In practice, it requires significant effort to construct suitable risk estimates. One must carefully define and measure the targets of prediction to avoid retrenching biases in the data. But, importantly, one cannot generally address these difficulties by requiring that algorithms satisfy popular mathematical formalizations of fairness. By highlighting these challenges in the foundation of fair machine learning, we hope to help researchers and practitioners productively advance the area.

The Measure and Mismeasure of Fairness: A Critical Review of Fair Machine Learning.

The Future of the Internet and How to Stop It Jonathan Zittrain. New Haven: Yale University Press, 2008. 342 pages. $30.00.Jonathan Zittrain's book, The Future of the Internet and How to Stop It, discusses the tension between generativity, the "capacity to produce unanticipated change through unfiltered contributions from a broad and varied audience" and the security problems inherent in today's personal computers and Internet system (70). This generativity, Zittrain argues, is the greatest strength of personal computers and the Internet and had led to great innovations, including creative software, peer-to-peer networking, blogs and content sites such as Wikipedia, and even the Internet itself. Yet this same generativity has also spawned security, legal, and social problems that may lead to the demise of personal computers and the Internet as they exist today (101). If we do not find solutions for these problems, Zittrain warns, we will be faced with government or corporate control of personal computers and the Internet, similar to current controls of cellular telephones and cable converter boxes, which will effectively shut down the innovations that personal computers and the Internet currently enable (245). Personal computers, according to Zittrain, will become nothing but appliances programmed by the manufacturer, capable only of limited tasks, and stripped of the tools and capabilities necessary for innovation.The first part of the book traces the history of personal computers, mainframes, proprietary servers, and the Internet. This section highlights how differences in technology and administration fostered generativity in personal computers and on the Internet and prohibited innovation on the mainframes and proprietary networks. Yet the innovations made possible by the generative nature of the Internet and personal computers also created its vulnerabilities: viruses, worms, malware, and spyware. Despite his obvious support of generativity, Zittrain admits that the current situation is "not sustainable" (43), but warns that the obvious solutions-such as limited functionality, governmental controls, and censorship-completely undermine the creativity made possible by personal computers and the Internet. The sec- ond part of the book defines generativity and explores its benefits and detriments. Here, Zittrain proposes that information technology functions best when generative, while admitting that we cannot ignore the problems caused by generativity (64), and warns equally of the dangers of controlled appliances: censorships, surveillance, forced legal compliance, and unannounced changes in machine functions. Chapter 6 in this section considers Wikikpedia as a self-regulating generative system that functions imperfectly but well. Part Three proposes technological, legal, and social solutions to the problems discussed earlier in the book.The Future of the Internet intertwines its analysis of current personal computer/Internet problems with current and historical examples from business, law, culture (Internet and otherwise), and technology. These examples, often worthwhile and interesting in and of themselves, underpin Zittrain's analysis rationally and support his conclusions. The examples show that the kind of sledgehammer solutions that Zittrain predicts are not merely possibilities or potentialities, but already have been and are being employed by various companies and governments. Perhaps best of all, Zittrain's use of these historical and current examples avoids the sensationalism often seen in other works that predict governmental and corporate control of options, behavior, and creativity. …

The Future of the Internet and How to Stop It

A key goal of the fair-ML community is to develop machine-learning based systems that, once introduced into a social context, can achieve social and legal outcomes such as fairness, justice, and due process. Bedrock concepts in computer science---such as abstraction and modular design---are used to define notions of fairness and discrimination, to produce fairness-aware learning algorithms, and to intervene at different stages of a decision-making pipeline to produce "fair" outcomes. In this paper, however, we contend that these concepts render technical interventions ineffective, inaccurate, and sometimes dangerously misguided when they enter the societal context that surrounds decision-making systems. We outline this mismatch with five "traps" that fair-ML work can fall into even as it attempts to be more context-aware in comparison to traditional data science. We draw on studies of sociotechnical systems in Science and Technology Studies to explain why such traps occur and how to avoid them. Finally, we suggest ways in which technical designers can mitigate the traps through a refocusing of design in terms of process rather than solutions, and by drawing abstraction boundaries to include social actors rather than purely technical ones.

https://dl.acm.org/doi/pdf/10.1145/3287560.3287598

Fairness and Abstraction in Sociotechnical Systems

Emerging vulnerabilities demand new conversations With public and academic attention increasingly focused on the new role of machine learning in the health information economy, an unusual and no-longer-esoteric category of vulnerabilities in machine-learning systems could prove important. These vulnerabilities allow a small, carefully designed change in how inputs are presented to a system to completely alter its output, causing it to confidently arrive at manifestly wrong conclusions. These advanced techniques to subvert otherwise-reliable machine-learning systems—so-called adversarial attacks—have, to date, been of interest primarily to computer science researchers (1). However, the landscape of often-competing interests within health care, and billions of dollars at stake in systems' outputs, implies considerable problems. We outline motivations that various players in the health care system may have to use adversarial attacks and begin a discussion of what to do about them. Far from discouraging continued innovation with medical machine learning, we call for active engagement of medical, technical, legal, and ethical experts in pursuit of efficient, broadly available, and effective health care that machine learning will enable.

https://science.sciencemag.org/content/sci/363/6433/1287.full.pdf

Adversarial attacks on medical machine learning

Actuarial risk assessments might be unduly perceived as a neutral way to counteract implicit bias and increase the fairness of decisions made at almost every juncture of the criminal justice system, from pretrial release to sentencing, parole and probation. In recent times these assessments have come under increased scrutiny, as critics claim that the statistical techniques underlying them might reproduce existing patterns of discrimination and historical biases that are reflected in the data. Much of this debate is centered around competing notions of fairness and predictive accuracy, resting on the contested use of variables that act as "proxies" for characteristics legally protected against discrimination, such as race and gender. We argue that a core ethical debate surrounding the use of regression in risk assessments is not simply one of bias or accuracy. Rather, it's one of purpose. If machine learning is operationalized merely in the service of predicting individual future crime, then it becomes difficult to break cycles of criminalization that are driven by the iatrogenic effects of the criminal justice system itself. We posit that machine learning should not be used for prediction, but rather to surface covariates that are fed into a causal model for understanding the social, structural and psychological drivers of crime. We propose an alternative application of machine learning and causal inference away from predicting risk scores to risk mitigation.

/pdf/interventions-over-predictions-reframing-the-ethical-debate-466hlu7nl9.pdf

Joichi Ito

Papers

Adversarial attacks on medical machine learning

Interventions over Predictions: Reframing the Ethical Debate for Actuarial Risk Assessment

Interventions over Predictions: Reframing the Ethical Debate for Actuarial Risk Assessment

Why Westerners Fear Robots and the Japanese Do Not

Supposedly 'Fair' Algorithms Can Perpetuate Discrimination