Surveys [8] show that people generally have a positive attitude towards autonomous cars. However, these studies neglect that cars have different levels of autonomy and that User Acceptance (UA) and User Experience (UX) with autonomous systems differ with regard to the degree of system autonomy. The National Highway Traffic Safety Administration (NHTSA) defines five degrees of car autonomy which vary in the penetration of cars with Advanced Driver Assistance Systems (ADAS) and the extent to which a car is taken over by autonomous systems. Based on these levels, we conducted an online-questionnaire study (N = 336), in which we investigated how UA and UX factors, such as Perceived Ease of Use, Attitude Towards using the system, Perceived Behavioral Control, Behavioral Intention to use a system, Trust and Fun, differ with regard to the degree of autonomy in cars. We show that UA and UX are highest in levels of autonomy that already have been deployed in modern cars. More specifically, perceived control and fun decrease continuously with higher autonomy. Furthermore, our results indicate that pre-experience with ADAS and demographics, such as age and gender, have an influence on UA and UX.

Towards Autonomous Cars: The Effect of Autonomy Levels on Acceptance and User Experience

We interact daily with computers that appear and behave like humans. Some researchers propose that people apply the same social norms to computers as they do to humans, suggesting that social psychological knowledge can be applied to our interactions with computers. In contrast, theories of human–automation interaction postulate that humans respond to machines in unique and specific ways. We believe that anthropomorphism—the degree to which an agent exhibits human characteristics—is the critical variable that may resolve this apparent contradiction across the formation, violation, and repair stages of trust. Three experiments were designed to examine these opposing viewpoints by varying the appearance and behavior of automated agents. Participants received advice that deteriorated gradually in reliability from a computer, avatar, or human agent. Our results showed (a) that anthropomorphic agents were associated with greater trust resilience, a higher resistance to breakdowns in trust; (b) that these effects were magnified by greater uncertainty; and c) that incorporating human-like trust repair behavior largely erased differences between the agents. Automation anthropomorphism is therefore a critical variable that should be carefully incorporated into any general theory of human–agent trust as well as novel automation design.

Almost human: Anthropomorphism increases trust resilience in cognitive agents.

Today, AI is being increasingly used to help human experts make decisions in high-stakes scenarios. In these scenarios, full automation is often undesirable, not only due to the significance of the outcome, but also because human experts can draw on their domain knowledge complementary to the model's to ensure task success. We refer to these scenarios as AI-assisted decision making, where the individual strengths of the human and the AI come together to optimize the joint decision outcome. A key to their success is to appropriately calibrate human trust in the AI on a case-by-case basis; knowing when to trust or distrust the AI allows the human expert to appropriately apply their knowledge, improving decision outcomes in cases where the model is likely to perform poorly. This research conducts a case study of AI-assisted decision making in which humans and AI have comparable performance alone, and explores whether features that reveal case-specific model information can calibrate trust and improve the joint performance of the human and AI. Specifically, we study the effect of showing confidence score and local explanation for a particular prediction. Through two human experiments, we show that confidence score can help calibrate people's trust in an AI model, but trust calibration alone is not sufficient to improve AI-assisted decision making, which may also depend on whether the human can bring in enough unique knowledge to complement the AI's errors. We also highlight the problems in using local explanation for AI-assisted decision making scenarios and invite the research community to explore new approaches to explainability for calibrating human trust in AI.

Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making

Introduction matters: Manipulating trust in automation and reliance in automated driving.

Autonomous vehicles are being viewed with scepticism in their ability to improve safety and the driving experience. A critical issue with automated driving at this stage of its development is that it is not yet reliable and safe. When automated driving fails, or is limited, the autonomous mode disengages and the drivers are expected to resume manual driving. For this transition to occur safely, it is imperative that drivers react in an appropriate and timely manner. Recent data released from the California trials provide compelling insights into the current factors influencing disengagements of autonomous mode. Here we show that the number of accidents observed has a significantly high correlation with the autonomous miles travelled. The reaction times to take control of the vehicle in the event of a disengagement was found to have a stable distribution across different companies at 0.83 seconds on average. However, there were differences observed in reaction times based on the type of disengagements, type of roadway and autonomous miles travelled. Lack of trust caused by the exposure to automated disengagements was found to increase the likelihood to take control of the vehicle manually. Further, with increased vehicle miles travelled the reaction times were found to increase, which suggests an increased level of trust with more vehicle miles travelled. We believe that this research would provide insurers, planners, traffic management officials and engineers fundamental insights into trust and reaction times that would help them design and engineer their systems.

/pdf/autonomous-vehicles-disengagements-accidents-and-reaction-47rge5oso7.pdf

Autonomous Vehicles: Disengagements, Accidents and Reaction Times.

To investigate the impact of visualizing car uncertainty on drivers' trust during an automated driving scenario, a simulator study was conducted. A between-group design experiment with 59 Swedish drivers was carried out where a continuous representation of the uncertainty of the car's ability to autonomously drive during snow conditions was displayed to one of the groups, whereas omitted for the control group. The results show that, on average, the group of drivers who were provided with the uncertainty representation took control of the car faster when needed, while they were, at the same time, the ones who spent more time looking at other things than on the road ahead. Thus, drivers provided with the uncertainty information could, to a higher degree, perform tasks other than driving without compromising with driving safety. The analysis of trust shows that the participants who were provided with the uncertainty information trusted the automated system less than those who did not receive such information, which indicates a more proper trust calibration than in the control group.

Presenting system uncertainty in automotive UIs for supporting trust calibration in autonomous driving

More and more complex semi-automated systems are being developed, aiding human operators to collect and analyze data and information and even to recommend decisions and act upon these. The goal of ...

/pdf/transparency-for-future-semi-automated-systems-effects-of-4ibqf46sf2.pdf

Transparency for Future Semi-Automated Systems : Effects of transparency on operator performance, workload and trust

This paper presents an empirical study that addresses the effects the visualization of uncertainty has on decision-making. We focus our investigations on an area where uncertainty plays an important role and the decision time is limited. For that, we selected an air defense scenario, where expert operators have a few minutes to make a well-informed decision based on uncertain sensor data regarding the identity of an object and where the consequences of a late or wrong decision are severe. An approach for uncertainty visualization is proposed and tested using a prototype that supports the interactive analysis of multivariate spatio-temporal sensor data. The uncertainty visualization embeds the accuracy of the sensor data values using the thickness of the lines in the graphical representation of the sensor values. Semi-transparent filled circles represent the uncertain position, while a track quality value between 0 and 1 accounts for the quality of the estimated track for each target. Twenty-two experienced air traffic operators were divided into two groups (with and without uncertainty visualization) and carried out identification and prioritization tasks using the prototype. The results show that the group aided by visualizations of uncertainty needed significantly fewer attempts to make a final identification, and a significant difference between the groups when considering the identities and priorities assigned was observed (participants with uncertainty visualization selected higher priority values and more hostile and suspect identities). These results may show that experts put themselves in the “worst-case scenario” in the presence of uncertainty when safety is an issue. Additionally, the presentation of uncertainty neither increased the participants' expressed workload, nor the time needed to make a classification. However, the inclusion of the uncertainty information did not have a significant effect on the performance (true positives, false negatives and false positives) or the participants' expressed confidence in their decisions.

Special Section on Uncertainty and Parameter Space Analysis in Visualization Effects of visualizing uncertainty on decision-making in a target identification scenario

Effects of visualizing uncertainty on decision-making in a target identification scenario

To find correlations and cause and effect relationships in multivariate data sets is central in many data analysis problems. A common way of representing causal relations among variables is to use ...

Tove Helldin

Papers

Presenting system uncertainty in automotive UIs for supporting trust calibration in autonomous driving

Transparency for Future Semi-Automated Systems : Effects of transparency on operator performance, workload and trust

Special Section on Uncertainty and Parameter Space Analysis in Visualization Effects of visualizing uncertainty on decision-making in a target identification scenario

Effects of visualizing uncertainty on decision-making in a target identification scenario

Understanding Indirect Causal Relationships in Node-Link Graphs