Proceedings ArticleDOI
Formalizing Trust in Artificial Intelligence: Prerequisites, Causes and Goals of Human Trust in AI
Alon Jacovi,Ana Marasović,Tim Miller,Yoav Goldberg +3 more
- pp 624-635
TLDR
In this paper, the authors discuss a model of trust inspired by sociologists' notion of interpersonal trust (i.e., trust between people) and discuss how to design trustworthy AI, how to evaluate whether trust has manifested, and whether it is warranted.Abstract:
Trust is a central component of the interaction between people and AI, in that 'incorrect' levels of trust may cause misuse, abuse or disuse of the technology. But what, precisely, is the nature of trust in AI? What are the prerequisites and goals of the cognitive mechanism of trust, and how can we promote them, or assess whether they are being satisfied in a given interaction? This work aims to answer these questions. We discuss a model of trust inspired by, but not identical to, interpersonal trust (i.e., trust between people) as defined by sociologists. This model rests on two key properties: the vulnerability of the user; and the ability to anticipate the impact of the AI model's decisions. We incorporate a formalization of 'contractual trust', such that trust between a user and an AI model is trust that some implicit or explicit contract will hold, and a formalization of 'trustworthiness' (that detaches from the notion of trustworthiness in sociology), and with it concepts of 'warranted' and 'unwarranted' trust. We present the possible causes of warranted trust as intrinsic reasoning and extrinsic behavior, and discuss how to design trustworthy AI, how to evaluate whether trust has manifested, and whether it is warranted. Finally, we elucidate the connection between trust and XAI using our formalization.read more
Citations
More filters
Posted Content
Underspecification Presents Challenges for Credibility in Modern Machine Learning
Alexander D'Amour,Katherine Heller,Dan Moldovan,Ben Adlam,Babak Alipanahi,Alex Beutel,Christina Chen,Jonathan Deaton,Jacob Eisenstein,Matthew D. Hoffman,Farhad Hormozdiari,Neil Houlsby,Shaobo Hou,Ghassen Jerfel,Alan Karthikesalingam,Mario Lucic,Yi-An Ma,Cory Y. McLean,Diana Mincu,Akinori Mitani,Andrea Montanari,Zachary Nado,Vivek T. Natarajan,Christopher Nielson,Thomas F. Osborne,Rajiv Raman,Kim Ramasamy,Rory Sayres,Jessica Schrouff,Martin G. Seneviratne,Shannon Sequeira,Harini Suresh,Victor Veitch,Max Vladymyrov,Xuezhi Wang,Kellie Webster,Steve Yadlowsky,Taedong Yun,Xiaohua Zhai,D. Sculley +39 more
TL;DR: This work shows the need to explicitly account for underspecification in modeling pipelines that are intended for real-world deployment in any domain, and shows that this problem appears in a wide variety of practical ML pipelines.
A Framework for Understanding Sources of Harm throughout the Machine Learning Life Cycle
Harini Suresh,John V. Guttag +1 more
TL;DR: In this paper, the authors identify seven potential sources of downstream harm in machine learning, spanning data collection, development, and deployment, and propose a framework to facilitate more productive and precise communication around these issues.
Posted Content
On the Opportunities and Risks of Foundation Models.
Rishi Bommasani,Drew A. Hudson,Ehsan Adeli,Russ B. Altman,Simran Arora,Sydney von Arx,Michael S. Bernstein,Jeannette Bohg,Antoine Bosselut,Emma Brunskill,Erik Brynjolfsson,Shyamal Buch,Dallas Card,Rodrigo Castellon,Niladri S. Chatterji,Annie Chen,Kathleen Creel,Jared Davis,Dora Demszky,Chris Donahue,Moussa Doumbouya,Esin Durmus,Stefano Ermon,John Etchemendy,Kawin Ethayarajh,Li Fei-Fei,Chelsea Finn,Trevor Gale,Lauren Gillespie,Karan Goel,Noah D. Goodman,Shelby Grossman,Neel Guha,Tatsunori Hashimoto,Peter Henderson,John Hewitt,Daniel E. Ho,Jenny Hong,Kyle Hsu,Jing Huang,Thomas Icard,Saahil Jain,Dan Jurafsky,Pratyusha Kalluri,Siddharth Karamcheti,Geoff Keeling,Fereshte Khani,Omar Khattab,Pang Wei Koh,Mark Krass,Ranjay Krishna,Rohith Kuditipudi,Ananya Kumar,Faisal Ladhak,Mina Lee,Tony Lee,Jure Leskovec,Isabelle Levent,Xiang Lisa Li,Xuechen Li,Tengyu Ma,Ali Ahmad Malik,Christopher D. Manning,Suvir Mirchandani,Eric Mitchell,Zanele Munyikwa,Suraj Nair,Avanika Narayan,Deepak Narayanan,Ben Newman,Allen Nie,Juan Carlos Niebles,Hamed Nilforoshan,Julian Nyarko,Giray Ogut,Laurel Orr,Isabel Papadimitriou,Joon Sung Park,Chris Piech,Eva Portelance,Christopher Potts,Aditi Raghunathan,Rob Reich,Hongyu Ren,Frieda Rong,Yusuf H. Roohani,Camilo Ruiz,Jack Ryan,Christopher Ré,Dorsa Sadigh,Shiori Sagawa,Keshav Santhanam,Andy Shih,Krishnan Srinivasan,Alex Tamkin,Rohan Taori,Armin W. Thomas,Florian Tramèr,Rose E. Wang,William Yang Wang,Bohan Wu,Jiajun Wu,Yuhuai Wu,Sang Michael Xie,Michihiro Yasunaga,Jiaxuan You,Matei Zaharia,Michael Zhang,Tianyi Zhang,Xikun Zhang,Yuhui Zhang,Lucia Zheng,Kaitlyn Zhou,Percy Liang +113 more
TL;DR: The authors provides a thorough account of the opportunities and risks of foundation models, ranging from their capabilities (e.g., language, vision, robotics, reasoning, human interaction) and technical principles(e. g.g. model architectures, training procedures, data, systems, security, evaluation, theory) to their applications.
Posted Content
A Framework for Understanding Sources of Harm throughout the Machine Learning Life Cycle
Harini Suresh,John V. Guttag +1 more
TL;DR: In this paper, the authors identify seven potential sources of downstream harm in machine learning, spanning data collection, development, and deployment, and propose a framework to facilitate more productive and precise communication around these issues.
Journal ArticleDOI
Good Proctor or "Big Brother"? Ethics of Online Exam Supervision Technologies.
TL;DR: In this paper, the authors provide a sustained moral philosophical analysis of online exam supervision technologies, focusing on ethical notions of academic integrity, fairness, non-maleficence, transparency, privacy, autonomy, liberty, and trust.
References
More filters
Journal ArticleDOI
An Integrative Model Of Organizational Trust
TL;DR: In this paper, a definition of trust and a model of its antecedents and outcomes are presented, which integrate research from multiple disciplines and differentiate trust from similar constructs, and several research propositions based on the model are presented.
Proceedings Article
Explaining and Harnessing Adversarial Examples
TL;DR: It is argued that the primary cause of neural networks' vulnerability to adversarial perturbation is their linear nature, supported by new quantitative results while giving the first explanation of the most intriguing fact about them: their generalization across architectures and training sets.
Proceedings Article
Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps
TL;DR: In this paper, the gradient of the class score with respect to the input image is computed to compute a class saliency map, which can be used for weakly supervised object segmentation using classification ConvNets.
Journal ArticleDOI
Trust as a Social Reality
J. David Lewis,Andrew J. Weigert +1 more
TL;DR: Trust is seen to include both emotional and cognitive dimensions and to function as a deep assumption underwriting social order as mentioned in this paper, and trust is an underdeveloped concept in sociology, promising theoretical formulations are available in the recent work of Luhmann and Barber.
Journal ArticleDOI
Trust in Automation: Designing for Appropriate Reliance
John D. Lee,Katrina A. See +1 more
TL;DR: This review considers trust from the organizational, sociological, interpersonal, psychological, and neurological perspectives, and considers how the context, automation characteristics, and cognitive processes affect the appropriateness of trust.