Latency between a user's movement and visual feedback is inevitable in every Virtual Reality application, as signal transmission and processing take time. Unfortunately, a high end-to-end latency impairs perception and motor performance. While it is possible to reduce feedback delay to tens of milliseconds, these delays will never completely vanish. Currently, there is a gap in literature regarding the impact of feedback delays on perception and motor performance as well as on their interplay in virtual environments employing full-body avatars. With the present study at hand, we address this gap by performing a systematic investigation of different levels of delay across a variety of perceptual and motor tasks during full-body action inside a Cave Automatic Virtual Environment. We presented participants with their virtual mirror image, which responded to their actions with feedback delays ranging from 45 to 350 ms. We measured the impact of these delays on motor performance, sense of agency, sense of body ownership and simultaneity perception by means of psychophysical procedures. Furthermore, we looked at interaction effects between these aspects to identify possible dependencies. The results show that motor performance and simultaneity perception are affected by latencies above 75 ms. Although sense of agency and body ownership only decline at a latency higher than 125 ms, and deteriorate for a latency greater than 300 ms, they do not break down completely even at the highest tested delay. Interestingly, participants perceptually infer the presence of delays more from their motor error in the task than from the actual level of delay. Whether or not participants notice a delay in a virtual environment might therefore depend on the motor task and their performance rather than on the actual delay.

/pdf/the-impact-of-latency-on-perceptual-judgments-and-motor-2n6b19542g.pdf

The impact of latency on perceptual judgments and motor performance in closed-loop interaction in virtual reality

Latency or system response time (ie, the delay between user input and system response) is a fundamental factor affecting human-computer interaction (HCI) If latency exceeds a critical threshold, user performance and experience get impaired Therefore, several design guidelines giving recommendations on maximum latencies for an optimal user experience have been developed within the last five centuries Concentrating on the lower boundary latencies, these guidelines are critically reviewed and contrasted with recent empirical findings Results of the review reveal that latencies below 100 ms were seldom considered in guidelines so far even though smaller latencies have been shown to be perceivable to the user and impact user performance negatively Thus, empirical evidence suggests a need for updated guidelines for designing latency in HCI

System Latency Guidelines Then and Now – Is Zero Latency Really Considered Necessary?

Measuring Visual Latency in VR and AR devices has become increasingly complicated as many of the components will influence others in multiple loops and ultimately affect the human cognitive and sensory perception. In this paper we present a new method based on the idea that the performance of humans on a rapid motor task will remain constant, and that any added delay will correspond to the system latency. We ask users to perform a task inside different video see-through devices and also in front of a computer. We also calculate the latency of the systems using a hardware instrumentation-based measurement technique for bench-marking. Results show that this new form of latency measurement through human cognitive performance can be reliable and comparable to hardware instrumentation-based measurement. Our method is adaptable to many forms of user interaction. It is particularly suitable for systems, such as AR and VR, where externalizing signals is difficult, or where it is important to measure latency while the system is in use by a user.

/pdf/measuring-system-visual-latency-through-cognitive-latency-on-4okyfkqmmp.pdf

Measuring System Visual Latency through Cognitive Latency on Video See-Through AR devices

End-to-end latency corresponds to the temporal difference between a user input and the corresponding output from a system. It has been shown to degrade user performance in both direct and indirect interaction. If it can be reduced to some extend, latency can also be compensated through software compensation by trying to predict the future position of the cursor based on previous positions, velocities and accelerations. In this paper, we propose a hybrid hardware and software prediction technique specifically designed for partially compensating end-to-end latency in indirect pointing. We combine a computer mouse with a high frequency accelerometer to predict the future location of the pointer using Euler based equations. Our prediction method results in more accurate prediction than previously introduced prediction algorithms for direct touch. A controlled experiment also revealed that it can improve target acquisition time in pointing tasks.

/pdf/using-high-frequency-accelerometer-and-mouse-to-compensate-3gwbhf8zm0.pdf

Using High Frequency Accelerometer and Mouse to Compensate for End-to-end Latency in Indirect Interaction

Haptic communication, the exchange of force and tactile information during dancing or moving a table together, has been shown to benefit the performance of human partners. Similarly, it could also be used to improve the performance of robots working in contact with a human operator. As we move to more robot integrated workspaces, how common network features such as delay or jitter impact haptic communication need to be better understood. Here using a human-like interactive robotic controller, that has been found to be indistinguishable by humans to human interaction, we evaluate how subjects’ performance and perception is altered by varying levels of transmission delay. We find that subjects are able to recognise haptic delay at very small levels within haptic interaction. However, while they are consciously aware of the delay they can only compensate for it up until a certain point, after which they perceive it as the addition of noise/impedance into the system.

Short Time Delay Does Not Hinder Haptic Communication Benefits

Latency is detrimental to interactive systems, especially pseudo-physical systems that emulate real-world behaviour. It prevents users from making quick corrections to their movement, and causes their experience to deviate from their expectations. Latency is a result of the processing and transport delays inherent in current computer systems. As such, while a number of studies have hypothesized that any latency will have a degrading effect, few have been able to test this for latencies less than $\scriptstyle \sim$ 50 ms. In this study we investigate the effects of latency on pointing and steering tasks. We design an apparatus with a latency lower than typical interactive systems, using it to perform interaction tasks based on Fitts's law and the Steering law. We find evidence that latency begins to affect performance at $\scriptstyle \sim$ 16 ms, and that the effect is non-linear. Further, we find latency does not affect the various components of an aiming motion equally. We propose a three stage characterisation of pointing movements with each stage affected independently by latency. We suggest that understanding how users execute movement is essential for studying latency at low levels, as high level metrics such as total movement time may be misleading.

/pdf/the-effects-of-low-latency-on-pointing-and-steering-tasks-2raqltkqhv.pdf

Per Karlstrm

Papers

The Effects of Low Latency on Pointing and Steering Tasks