Multimodal selection techniques for dense and occluded 3D virtual environments

TLDR: This paper presents new forms of the ray casting and bubble cursor selection techniques, which are augmented with visual, audio, and haptic feedback, to support selection within dense and occluded 3D target environments.

Abstract: Object selection is a primary interaction technique which must be supported by any interactive three-dimensional virtual reality application. Although numerous techniques exist, few have been designed to support the selection of objects in dense target environments, or the selection of objects which are occluded from the user's viewpoint. There is, thus, a limited understanding on how these important factors will affect selection performance. In this paper, we present a set of design guidelines and strategies to aid the development of selection techniques which can compensate for environment density and target visibility. Based on these guidelines, we present new forms of the ray casting and bubble cursor selection techniques, which are augmented with visual, audio, and haptic feedback, to support selection within dense and occluded 3D target environments. We perform a series of experiments to evaluate these new techniques, varying both the environment density and target visibility. The results provide an initial understanding of how these factors affect selection performance. Furthermore, the results showed that our new techniques adequately allowed users to select targets which were not visible from their initial viewpoint. The audio and haptic feedback did not provide significant improvements, and our analysis indicated that our introduced visual feedback played the most critical role in aiding the selection task.