Manipulating a virtual object with appropriate passive haptic cues provides a satisfying sense of presence in virtual reality. However, scaling such experiences to support multiple virtual objects is a challenge as each one needs to be accompanied with a precisely-located haptic proxy object. We propose a solution that overcomes this limitation by hacking human perception. We have created a framework for repurposing passive haptics, called haptic retargeting, that leverages the dominance of vision when our senses conflict. With haptic retargeting, a single physical prop can provide passive haptics for multiple virtual objects. We introduce three approaches for dynamically aligning physical and virtual objects: world manipulation, body manipulation and a hybrid technique which combines both world and body manipulation. Our study results indicate that all our haptic retargeting techniques improve the sense of presence when compared to typical wand-based 3D control of virtual objects. Furthermore, our hybrid haptic retargeting achieved the highest satisfaction and presence scores while limiting the visible side-effects during interaction.

/pdf/haptic-retargeting-dynamic-repurposing-of-passive-haptics-139x6wfz6v.pdf

Haptic Retargeting: Dynamic Repurposing of Passive Haptics for Enhanced Virtual Reality Experiences

We introduce semantic pointing, a novel interaction technique that improves target acquisition in graphical user interfaces (GUIs). Semantic pointing uses two independent sizes for each potential target presented to the user: one size in motor space adapted to its importance for the manipulation, and one size in visual space adapted to the amount of information it conveys. This decoupling between visual and motor size is achieved by changing the control-to-display ratio according to cursor distance to nearby targets. We present a controlled experiment supporting our hypothesis that the performance of semantic pointing is given by Fitts' index of difficulty in motor rather than visual space. We apply semantic pointing to the redesign of traditional GUI widgets by taking advantage of the independent manipulation of motor and visual widget sizes.

/pdf/semantic-pointing-improving-target-acquisition-with-control-37fzd0ozxj.pdf

Semantic pointing: improving target acquisition with control-display ratio adaptation

This paper presents a survey of the main results obtained in the field of “pseudo-haptic feedback”: a technique meant to simulate haptic sensations in virtual environments using visual feedback and properties of human visuo-haptic perception. Pseudo-haptic feedback uses vision to distort haptic perception and verges on haptic illusions. Pseudo-haptic feedback has been used to simulate various haptic properties such as the stiffness of a virtual spring, the texture of an image, or the mass of a virtual object. This paper describes the several experiments in which these haptic properties were simulated. It assesses the definition and the properties of pseudo-haptic feedback. It also describes several virtual reality applications in which pseudo-haptic feedback has been successfully implemented, such as a virtual environment for vocational training of milling machine operations, or a medical simulator for training in regional anesthesia procedures.

/pdf/simulating-haptic-feedback-using-vision-a-survey-of-research-4b3p0h5i0p.pdf

Simulating haptic feedback using vision: A survey of research and applications of pseudo-haptic feedback

How do users generate an illusion of presence in a rich and consistent virtual environment from an impoverished, incomplete, and often inconsistent set of sensory cues? We conducted an experiment to explore how multimodal perceptual cues are integrated into a coherent experience of virtual objects and spaces. Specifically, we explored whether inter-modal integration contributes to generating the illusion of presence in virtual environments.

To discover whether intermodal integration might play a role in presence, we looked for evidence of intermodal integration in the form of cross-modal interactions---perceptual illusions in which users use sensory cues in one modality to “fill in” the “missing” components of perceptual experience. One form of cross-modal interaction, a cross-modal transfer, is defined as a form of synesthesia, that is, a perceptual illusion in which stimulation to a sensory modality connected to the interface (such as the visual modality) is accompanied by perceived stimulation to an unconnected sensory modality that receives no apparent stimulation from the virtual environment (such as the haptic modality). Users of our experimental virtual environment who manipulated the visual analog of a physical force, a virtual spring, reported haptic sensations of “physical resistance”, even though the interface included no haptic displays. A path model of the data suggested that this cross-modal illusion was correlated with and dependent upon the sensation of spatial and sensory presence.

We conclude that this is evidence that presence may derive from the process of multi-modal integration and, therefore, may be associated with other illusions, such as cross-modal transfers, that result from the process of creating a coherent mental model of the space. Finally, we suggest that this perceptual phenomenon might be used to improve user experiences with multimodal interfaces, specifically by supporting limited sensory displays (such as haptic displays) with appropriate synesthetic stimulation to other sensory modalities (such as visual and auditory analogs of haptic forces).

Visual Touch in Virtual Environments: An Exploratory Study of Presence, Multimodal Interfaces, and Cross-Modal Sensory Illusions

For the perception of the hardness of compliant materials, several cues are available. In this paper, the relative roles of force/displacement and surface deformation cues are investigated. We have measured discrimination thresholds with silicone rubber stimuli of differing thickness and compliance. Also, the influence of the finger span is assessed. When compliance is expressed as the Young's modulus, the thresholds in the different conditions follow Weber's law with a Weber fraction of 15 percent. When the surface deformation cue was removed, thresholds more than trebled. Under the assumption of optimal cue combination, this suggests that a large fraction of the information comes from the surface deformation cue. Using a matching experiment, we found that differences in object thickness are correctly taken into account. When cues appear to contradict each other, the conflict is resolved by means of a compromise.

/pdf/cues-for-haptic-perception-of-compliance-5awmjn28e2.pdf

Cues for Haptic Perception of Compliance

This paper considers whether a passive isometric input device, such as a Spaceball/sup TM/, used together with visual feedback, could provide the operator with a pseudo-haptic feedback. For this aim, two psychophysical experiments have been conducted. The first experiment consisted of a compliance discrimination, between two virtual springs hand-operated by means of the Spaceball/sup TM/. In this experiment, the stiffness (or compliance) JND turned out to be 6%. The second experiment assessed stiffness discrimination between a virtual spring and the equivalent spring in reality. In this case, the stiffness (or compliance) JND was found to be 13.4%. These results are consistent with previous outcomes on manual discrimination of compliance. Consequently, this consistency reveals that the passive apparatus that was used can, to some extent, simulate haptic information. In addition, a final test indicated that the proprioceptive sense of the subjects was blurred by visual feedback. This gave them the illusion of using a nonisometric device.

/pdf/pseudo-haptic-feedback-can-isometric-input-devices-simulate-58ppv00sgm.pdf

S. Coquillart

Papers

Pseudo-haptic feedback: can isometric input devices simulate force feedback?