In this paper, we perform the first analysis of behavior-based authentication within and across multiple virtual reality (VR) systems. The proliferation of VR in everyday applications such as healthcare, manufacturing, education, and remote teleoperation necessitates an understanding of operation over multiple VR systems as it is likely that a user will utilize several systems and requiring the user to re-train to a new system is time consuming. We collect the first multi-system dataset for VR biometrics consisting of 46 users performing a ball-throwing interaction using the Oculus Quest, HTC Vive, and HTC Vive Cosmos. Each user provides 10 training samples on the first day and 10 test samples on the second day for each of the three devices. We collect position and orientation data for devices such as the headset, left hand controller, and right hand controller, as well as trigger positions for the dominant hand controller. Our approach uses pairwise matches between trajectory features to represent high intra-user consistency and inter-user discriminative capacity. We analyze within-system and cross-system authentication accuracy for the 41 right-handed users in our 46-user dataset over varying combinations of matches for features such as point positions, device orientations, linear and angular velocities, and trigger grab or release for the right hand controller, left hand controller, and headset. Our approach provides maximum within-system authentication accuracy of 97%, 91%, and 91% when test trajectories are compared to training trajectories for the Vive, Quest, and Cosmos respectively. We provide maximum accuracies of 58% for testing with Cosmos and training with Quest, 70% for testing with Cosmos and training with Vive, and 85% for testing with Quest and training with Vive.

Within-System and Cross-System Behavior-Based Biometric Authentication in Virtual Reality

In this paper, we provide an approach on using behavioral biometrics to perform cross-system high-assurance authentication of users in virtual reality (VR) environments. VR is currently being explored as a critical tool to ensure seamless delivery of essential services, such as education, healthcare, and personal finance, while enabling users to work from home environments. Due to the sensitive nature of personal data generated, VR applications for essential services need to provide secure access. Traditional PIN or password-based credentials can be breached by malicious impostors, or be handed over by an intended user of a VR system to a confederate to assist the intended user in completing a task, e.g., an exam or a physical therapy routine. Existing approaches that use the behavior of the user in VR as a biometric signature fail when users provide enrollment and use-time data on different VR systems. We use Siamese neural networks to learn a distance function that characterizes the systematic differences between data provided across pairs of dissimilar VR systems. Our approach provides average equal error rates (EERs) ranging from 1.38% to 3.86% for authentication using a benchmark dataset that consists of 41 users performing a ball-throwing task with 3 VR systems-an Oculus Quest, an HTC Vive, and an HTC Vive Cosmos. To compare to prior approaches in VR biometrics, we also obtain average accuracies for the task of identification, where given an input user's trajectory in a use-time VR system, we use Siamese networks to return the user with the top matching trajectory in an enrollment VR system as the label. We report identification results ranging from 87.82% to 98.53% with average improvements of 29.78%±8.58% and 30.78%±3.68% over existing approaches that use generic distance matching and fully convolutional networks on the enrollment dataset respectively.

Using Siamese Neural Networks to Perform Cross-System Behavioral Authentication in Virtual Reality

With the increasing knowledge advancements and availability of mixed reality (MR), the number of its purposes and applications in vehicles rises. Mixed reality may help to increase road safety, support more immersive (non-) driving related activities, and finally enhance driving and passenger experience. MR may also be the enabling technology to increase trust and acceptance in automated vehicles and therefore help on the transition towards automated driving. Further, automated driving extends use cases of virtual reality and other immersive technologies. However, there are still a number of challenges with the use of mixed reality when applied in vehicles, and also several human factors issues need to be solved. This paper aims at presenting a research agenda for using mixed reality technology for automotive user interfaces (UIs) by identifying opportunities and challenges.

A Research Agenda for Mixed Reality in Automated Vehicles

/pdf/everyday-automation-experience-a-research-agenda-2hflgil1gj.pdf

Everyday automation experience: A research agenda

Paving the way to future mobility, teleoperation of vehicles promises a reachable solution to effectively use the benefits of automated driving as long as fully automated vehicles (SAE 5) are not entirely feasible. Safety and reliability are assured by a human operator who remotely observes the vehicle and takes over control in cases of disturbances that exceed the vehicle automation’s skills. In order to integrate the vehicle’s automation and human remote-operation, we developed a novel user-centered human-machine interface (HMI) for teleoperation. It is tailored to the remote-operation of a highly automated shuttle (SAE 4) by a public transport control center and based on a systematic analysis of scenarios, of which detailed requirements were derived. Subsequently, a paper-pencil prototype was generated and refined until a click-dummy emerged. This click-dummy was evaluated by twelve control center professionals. The experts were presented the prototype in regular mode and were then asked to solve three scenarios with disturbances in the system. Using structured interview and questionnaire methodology, the prototype was evaluated regarding its usability, situation awareness, acceptance, and perceived workload. Results support our HMI design for teleoperation of a highly automated shuttle, especially regarding usability, acceptance, and workload. Participant ratings and comments indicated particularly high satisfaction with the interaction design to resolve disturbances and the presentation of camera images. Participants’ feedbacks provide valuable information for a refined HMI design as well as for further research.

Teleoperation of Highly Automated Vehicles in Public Transport: User-Centered Design of a Human-Machine Interface for Remote-Operation and Its Expert Usability Evaluation

In the domain of automated driving, numerous (technological) problems were solved in recent years, but still many limitations are around that could eventually prevent the deployment of automated driving systems (ADS) beyond SAE level 3. A remote operating fallback authority might be a promising solution. In order for teleoperation to function reliably and universal, it will make use of existing infrastructure, such as cellular networks. Unfortunately, cellular networks might suffer from variable performance. In this work, we investigate the effects of latency on task performance and perceived workload for different driving scenarios. Results from a simulator study (N=28) suggest that latency has negative influence on driving performance and subjective factors and led to a decreased confidence in Teleoperated Driving during the study. A latency of about 300 ms already led to a deteriorated driving performance, whereas variable latency did not consequently deteriorate driving performance.

Armin Becher

Papers

Teleoperation: The Holy Grail to Solve Problems of Automated Driving? Sure, but Latency Matters