Artificial intelligence in business: state of the art and future research agenda

The development of theory, frameworks and tools for Explainable AI (XAI) is a very active area of research these days, and articulating any kind of coherence on a vision and challenges is itself a challenge. At least two sometimes complementary and colliding threads have emerged. The first focuses on the development of pragmatic tools for increasing the transparency of automatically learned prediction models, as for instance by deep or reinforcement learning. The second is aimed at anticipating the negative impact of opaque models with the desire to regulate or control impactful consequences of incorrect predictions, especially in sensitive areas like medicine and law. The formulation of methods to augment the construction of predictive models with domain knowledge can provide support for producing human understandable explanations for predictions. This runs in parallel with AI regulatory concerns, like the European Union General Data Protection Regulation, which sets standards for the production of explanations from automated or semi-automated decision making. Despite the fact that all this research activity is the growing acknowledgement that the topic of explainability is essential, it is important to recall that it is also among the oldest fields of computer science. In fact, early AI was re-traceable, interpretable, thus understandable by and explainable to humans. The goal of this research is to articulate the big picture ideas and their role in advancing the development of XAI systems, to acknowledge their historical roots, and to emphasise the biggest challenges to moving forward.

Explainable Artificial Intelligence: Concepts, Applications, Research Challenges and Visions

AI is undergoing a paradigm shift with the rise of models (e.g., BERT, DALL-E, GPT-3) that are trained on broad data at scale and are adaptable to a wide range of downstream tasks. We call these models foundation models to underscore their critically central yet incomplete character. This report provides a thorough account of the opportunities and risks of foundation models, ranging from their capabilities (e.g., language, vision, robotics, reasoning, human interaction) and technical principles(e.g., model architectures, training procedures, data, systems, security, evaluation, theory) to their applications (e.g., law, healthcare, education) and societal impact (e.g., inequity, misuse, economic and environmental impact, legal and ethical considerations). Though foundation models are based on standard deep learning and transfer learning, their scale results in new emergent capabilities,and their effectiveness across so many tasks incentivizes homogenization. Homogenization provides powerful leverage but demands caution, as the defects of the foundation model are inherited by all the adapted models downstream. Despite the impending widespread deployment of foundation models, we currently lack a clear understanding of how they work, when they fail, and what they are even capable of due to their emergent properties. To tackle these questions, we believe much of the critical research on foundation models will require deep interdisciplinary collaboration commensurate with their fundamentally sociotechnical nature.

On the Opportunities and Risks of Foundation Models.

Digital pathology is not only one of the most promising fields of diagnostic medicine, but at the same time a hot topic for fundamental research. Digital pathology is not just the transfer of histopathological slides into digital representations. The combination of different data sources (images, patient records, and *omics data) together with current advances in artificial intelligence/machine learning enable to make novel information accessible and quantifiable to a human expert, which is not yet available and not exploited in current medical settings. The grand goal is to reach a level of usable intelligence to understand the data in the context of an application task, thereby making machine decisions transparent, interpretable and explainable. The foundation of such an "augmented pathologist" needs an integrated approach: While machine learning algorithms require many thousands of training examples, a human expert is often confronted with only a few data points. Interestingly, humans can learn from such few examples and are able to instantly interpret complex patterns. Consequently, the grand goal is to combine the possibilities of artificial intelligence with human intelligence and to find a well-suited balance between them to enable what neither of them could do on their own. This can raise the quality of education, diagnosis, prognosis and prediction of cancer and other diseases. In this paper we describe some (incomplete) research issues which we believe should be addressed in an integrated and concerted effort for paving the way towards the augmented pathologist.

Towards the Augmented Pathologist: Challenges of Explainable-AI in Digital Pathology

The right to be forgotten states that a data owner has the right to erase her data from an entity storing it. In the context of machine learning (ML), the right to be forgotten requires an ML model owner to remove the data owner's data from the training set used to build the ML model, a process known as machine unlearning. While originally designed to protect the privacy of the data owner, we argue that machine unlearning may leave some imprint of the data in the ML model and thus create unintended privacy risks. 
In this paper, we perform the first study on investigating the unintended information leakage caused by machine unlearning. We propose a novel membership inference attack which leverages the different outputs of an ML model's two versions to infer whether the deleted sample is part of the training set. Our experiments over five different datasets demonstrate that the proposed membership inference attack achieves strong performance. More importantly, we show that our attack in multiple cases outperforms the classical membership inference attack on the original ML model, which indicates that machine unlearning can have counterproductive effects on privacy. We notice that the privacy degradation is especially significant for well-generalized ML models where classical membership inference does not perform well. We further investigate two mechanisms to mitigate the newly discovered privacy risks and show that the only effective mechanism is to release the predicted label only. We believe that our results can help improve privacy in practical implementation of machine unlearning.

When Machine Unlearning Jeopardizes Privacy

https://scholarship.law.bu.edu/cgi/viewcontent.cgi?article=1816&context=faculty_scholarship

Humans forget, machines remember: Artificial intelligence and the Right to Be Forgotten

To understand the Right to be Forgotten in context of artificial intelligence, it is necessary to first delve into an overview of the concepts of human and AI memory and forgetting. Our current law appears to treat human and machine memory alike – supporting a fictitious understanding of memory and forgetting that does not comport with reality. (Some authors have already highlighted the concerns on the perfect remembering.) This Article will examine the problem of AI memory and the Right to be Forgotten, using this example as a model for understanding the failures of current privacy law to reflect the realities of AI technology. 
First, this Article analyzes the legal background behind the Right to be Forgotten, in order to understand its potential applicability to AI, including a discussion on the antagonism between the values of privacy and transparency under current E.U. privacy law. Next, the Authors explore whether the Right to be Forgotten is practicable or beneficial in an AI/machine learning context, in order to understand whether and how the law should address the Right to Be Forgotten in a post-AI world. The Authors discuss the technical problems faced when adhering to strict interpretation of data deletion requirements under the Right to be Forgotten, ultimately concluding that it may be impossible to fulfill the legal aims of the Right to be Forgotten in artificial intelligence environments. Finally, this Article addresses the core issue at the heart of the AI and Right to be Forgotten problem: the unfortunate dearth of interdisciplinary scholarship supporting privacy law and regulation.

Humans Forget, Machines Remember: Artificial Intelligence and the Right to Be Forgotten

The COVID-19 pandemic has caused millions of deaths and disastrous consequences around the world, with lasting repercussions for every field of law, including privacy and technology. The unique characteristics of this pandemic have precipitated an increase in use of new technologies, including remote communications platforms, healthcare robots, and medical AI. Public and private actors are using new technologies, like heat sensing, and technologically-influenced programs, like contact tracing, alike in response, leading to a rise in government and corporate surveillance in sectors like healthcare, employment, education, and commerce. Advocates have raised the alarm for privacy and civil liberties violations, but the emergency nature of the pandemic has drowned out many concerns. 

This Article is the first comprehensive account of privacy impacts related to technology and public health responses to the COVID-19 crisis. Many have written on the general need for better health privacy protections, education privacy protections, consumer privacy protections, and protections against government and corporate surveillance. However, this Article is the first comprehensive article to examine these problems of privacy and technology specifically in light of the pandemic, arguing that the lens of the pandemic exposes the need for both widescale and small-scale reform of privacy law. This Article approaches these problems with a focus on technical realities and social salience, and with a critical awareness of digital and political inequities, crafting normative recommendations with these concepts in mind.

Understanding privacy in this time of pandemic is critical for law and policymaking in the near future and for the long-term goals of creating a future society that protects both civil liberties and public health. It is also important to create a contemporary scholarly understanding of privacy in pandemic at this moment in time, as a matter of historical record. By examining privacy in pandemic, in the midst of pandemic, this Article seeks to create a holistic scholarly foundation for future work on privacy, technology, public health, and legal responses to global crises.

/pdf/privacy-in-pandemic-law-technology-and-public-health-in-the-47q2d0jo1h.pdf

Privacy in Pandemic: Law, Technology, and Public Health in the COVID-19 Crisis

Privacy is often a subjective value, taking on meaning from specific social, historical, and cultural contexts. Western privacy scholars have so far generally limited academic study to focus on Western ideals of privacy. However, privacy – or some notion of it – can be found in almost every culture and every nation, including the growing economic powerhouse that is the People’s Republic of China. Focusing on China as a case study of non-Western privacy norms is important today, given the rapid rise of the Chinese economy and its corresponding impact on worldwide cultural norms and law. Simply put, it is naive to believe that privacy law will develop in the near future without the influence of China. First, though China does not have a developed body of privacy law, any privacy laws China chooses to implement now and in the future will certainly affect business and organizations seeking to invest in and engage with Chinese consumers. With the size and economic impact of the Chinese consumer base, international technology organizations would be remiss in ignoring the development of Chinese privacy laws. Second, understanding Chinese conceptions of privacy is crucial to the study of international privacy law. China has a great deal of influence in the Asian region, which is particularly important given the somewhat unclear status of the APEC Privacy Principles. Given the long-arm reach of many nations’ privacy laws (particularly the EU’s current and proposed regulations), several signs indicate that China may follow the EU’s example in enforcing its privacy laws worldwide. For these reasons, it is imperative that privacy scholars and practitioners pay close attention to the field of Chinese privacy law. As Chinese privacy laws today are still fairly underdeveloped, now is the time to carefully study how cultural norms have impacted the development of Chinese privacy law thus far and how future regulations can and should develop, given Chinese cultural foundations of privacy. This paper will attempt to illuminate the state of privacy law in China, by evaluating current legal developments, exploring cultural and historical perceptions of privacy, and analyzing how China’s unique perceptions of privacy may influence the future development of new norms in China’s growing privacy regime. In particular, this paper looks to the significant contrast between traditional Western ideals of privacy as an individual right and traditional Chinese ideals of privacy related to the concepts of “saving face” and community values, ultimately leading to recommendations for future data regulation policies based on the unique cultural and historical values of Chinese perceptions of privacy.

Saving Face: Unfolding the Screen of Chinese Privacy Law

As countries begin to vaccinate their populations against COVID-19, creating systems to verify vaccine records will be vital to reopening businesses, educational institutions, and travel. We consider the challenges of building vaccine record verification (VRV) systems that involve data sharing by health care providers, methods for verifying vaccine records, and regulation of how entities (e.g., workplaces, schools, businesses, and airlines) may request proof of vaccination. In particular, we focus on the opportunities and risks associated with digital vaccine passport apps. We propose three ethical principles to guide the building of VRV systems: 1) aligning systems with vaccine prioritization, 2) upholding fairness and equity, and 3) building trustworthy technology that protects the public's health data.

Tiffany C. Li

Papers

Humans forget, machines remember: Artificial intelligence and the Right to Be Forgotten

Humans Forget, Machines Remember: Artificial Intelligence and the Right to Be Forgotten

Privacy in Pandemic: Law, Technology, and Public Health in the COVID-19 Crisis

Saving Face: Unfolding the Screen of Chinese Privacy Law

Building robust and ethical vaccination verification systems