Effect of auditory cues on the haptic perception of stiffness in virtual environments

TLDR: Psychophysical experiments examining the influence of sound on the haptic perception of stiffness indicate that when the physical stiffnesses of the surfaces were the same, subjects consistently ranked the surfaces according to sound, i.e., surfaces paired with sound cues that are typically associated with tapping harder surfaces were generally perceived as stiffer.

Abstract: To explore the possibility that multisensory information may be useful in expanding the range of haptic experiences in virtual environments, psychophysical experiments examining the influence of sound on the haptic perception of stiffness were carried out. In these experiments, subjects utilized the PHANToM, a six-degree-of-freedom haptic interface device with force-reflection along three axes, to feel the stiffness of various virtual surfaces. As subjects tapped on the different virtual surfaces, they were simultaneously presented with various impact sounds. The subjects were asked to rank the surfaces based on their perceived stiffness. The results indicate that when the physical stiffnesses of the surfaces were the same, subjects consistently ranked the surfaces according to sound, i.e., surfaces paired with sound cues that are typically associated with tapping harder surfaces were generally perceived as stiffer. However, when sound cues were randomly paired with surfaces of different mechanical stiffnesses, the results were more equivocal: naive subjects who had not used the PHANToM previously tended to be more affected by sound cues than another group of subjects who had previously completed a set of stiffness discrimination experiments without sound cues. The possible implications of this result for the design of multimodal virtual environments and its comparison to prior work by some of the authors on the effects of vision on haptic perception are discussed.