Creating Realistic Virtual Textures from Contact Acceleration Data
TL;DR: This paper employs a sensorized handheld tool to capture the feel of a given texture, reduces the three-dimensional acceleration signals to a perceptually equivalent one-dimensional signal, and uses linear predictive coding to distill this raw haptic information into a database of frequency-domain texture models.
Abstract: Modern haptic interfaces are adept at conveying the large-scale shape of virtual objects, but they often provide unrealistic or no feedback when it comes to the microscopic details of surface texture. Direct texture-rendering challenges the state of the art in haptics because it requires a finely detailed model of the surface's properties, real-time dynamic simulation of complex interactions, and high-bandwidth haptic output to enable the user to feel the resulting contacts. This paper presents a new, fully realized solution for creating realistic virtual textures. Our system employs a sensorized handheld tool to capture the feel of a given texture, recording three-dimensional tool acceleration, tool position, and contact force over time. We reduce the three-dimensional acceleration signals to a perceptually equivalent one-dimensional signal, and then we use linear predictive coding to distill this raw haptic information into a database of frequency-domain texture models. Finally, we render these texture models in real time on a Wacom tablet using a stylus augmented with small voice coil actuators. The resulting virtual textures provide a compelling simulation of contact with the real surfaces, which we verify through a human subject study.
Citations
More filters
TL;DR: Performance of 99.6% in correctly discriminating pairs of similar textures was found to exceed human capabilities, and the method of Bayesian exploration developed and tested in this paper may generalize well to other cognitive problems.
Abstract: In order to endow robots with humanlike abilities to characterize and identify objects, they must be provided with tactile sensors and intelligent algorithms to select, control and interpret data from useful exploratory movements. Humans make informed decisions on the sequence of exploratory movements that would yield the most information for the task, depending on what the object may be and prior knowledge of what to expect from possible exploratory movements. This study is focused on texture discrimination, a subset of a much larger group of exploratory movements and percepts that humans use to discriminate, characterize, and identify objects. Using a testbed equipped with a biologically inspired tactile sensor (the BioTac®) we produced sliding movements similar to those that humans make when exploring textures. Measurement of tactile vibrations and reaction forces when exploring textures were used to extract measures of textural properties inspired from psychophysical literature (traction, roughness, and fineness). Different combinations of normal force and velocity were identified to be useful for each of these three properties. A total of 117 textures were explored with these three movements to create a database of “prior experience” to use for identifying these same textures in future encounters. When exploring a texture, the discrimination algorithm adaptively selects the optimal movement to make and property to measure based on previous experience to differentiate the texture from a set of plausible candidates, a process we call Bayesian exploration. Performance of 99.6% in correctly discriminating pairs of similar textures was found to exceed human capabilities. Absolute classification from the entire set of 117 textures generally required a small number of well-chosen exploratory movements (median=5) and yielded a 95.4% success rate. The method of “Bayesian exploration” developed and tested in this paper may generalize well to other cognitive problems.
335 citations
Cites background from "Creating Realistic Virtual Textures..."
...Compelling artificial texture percepts can be recreated based on only these two parameters of an exploratory movement (Romano and Kuchenbecker, 2011)....
[...]
01 Sep 2013
TL;DR: The relevant human vibrotactile perceptual capabilities are explained, the main types of commercial vib rotactile actuators are detailed, and how to build both monolithic and localized vibrotACTile displays are described.
Abstract: This paper reviews the technology and applications of vibrotactile display, an effective information transfer modality for the emerging area of haptic media. Our emphasis is on summarizing foundational knowledge in this area and providing implementation guidelines for application designers who do not yet have a background in haptics. Specifically, we explain the relevant human vibrotactile perceptual capabilities, detail the main types of commercial vibrotactile actuators, and describe how to build both monolithic and localized vibrotactile displays. We then identify exemplary vibrotactile display systems in application areas ranging from the presentation of physical object properties to broadcasting vibrotactile media content.
296 citations
Cites background from "Creating Realistic Virtual Textures..."
..., canvas) on a touchscreen [48], [49] [Fig....
[...]
...Designers wishing to quantify the vibration-output performance of such a system can attach a small high-bandwidth three-axis accelerometer near the finger placement locations and record the vibrations that occur when the device is activated in a user’s hands, as done in [48]....
[...]
TL;DR: This survey postulates this as a principle for robot perception and collects evidence in its support by analyzing and categorizing existing work in this area, and provides an overview of the most important applications of IP.
Abstract: Recent approaches in robot perception follow the insight that perception is facilitated by interaction with the environment. These approaches are subsumed under the term Interactive Perception (IP). This view of perception provides the following benefits. First, interaction with the environment creates a rich sensory signal that would otherwise not be present. Second, knowledge of the regularity in the combined space of sensory data and action parameters facilitates the prediction and interpretation of the sensory signal. In this survey, we postulate this as a principle for robot perception and collect evidence in its support by analyzing and categorizing existing work in this area. We also provide an overview of the most important applications of IP. We close this survey by discussing remaining open questions. With this survey, we hope to help define the field of Interactive Perception and to provide a valuable resource for future research.
258 citations
Cites background from "Creating Realistic Virtual Textures..."
...For example, in [92] and [42], it has been shown that surface and material properties of objects can be more accurately estimated if the robot’s haptic sensor is moved along the surface of the object....
[...]
TL;DR: The technology behind creating artificial touch sensations and the relevant aspects of human touch are reviewed and the need to consider the neuroscience and perception behind the human sense of touch in the design and control of haptic devices is addressed.
Abstract: This article reviews the technology behind creating artificial touch sensations and the relevant aspects of human touch We focus on the design and control of haptic devices and discuss the best practices for generating distinct and effective touch sensations Artificial haptic sensations can present information to users, help them complete a task, augment or replace the other senses, and add immersiveness and realism to virtual interactions We examine these applications in the context of different haptic feedback modalities and the forms that haptic devices can take We discuss the prior work, limitations, and design considerations of each feedback modality and individual haptic technology We also address the need to consider the neuroscience and perception behind the human sense of touch in the design and control of haptic devices
214 citations
Cites methods from "Creating Realistic Virtual Textures..."
...Researchers have also created haptic texture models using autoregressive models that depended on the user’s normal force and scanning speed (107, 108)....
[...]
08 Oct 2013
TL;DR: A psychophysical model is established that relates the perceived friction force to the controlled voltage applied to the tactile feedback device, and it is demonstrated that participants are three times more likely to prefer gradient force profiles than other commonly used rendering profiles.
Abstract: We present a tactile-rendering algorithm for simulating 3D geometric features, such as bumps, on touch screen surfaces. This is achieved by modulating friction forces between the user's finger and the touch screen, instead of physically moving the touch surface. We proposed that the percept of a 3D bump is created when local gradients of the rendered virtual surface are mapped to lateral friction forces. To validate this approach, we first establish a psychophysical model that relates the perceived friction force to the controlled voltage applied to the tactile feedback device. We then use this model to demonstrate that participants are three times more likely to prefer gradient force profiles than other commonly used rendering profiles. Finally, we present a generalized algorithm and conclude the paper with a set of applications using our tactile rendering technology.
172 citations
Cites methods from "Creating Realistic Virtual Textures..."
...“Record and Play” strategies have also been used, where physical features of the object were first measured and then re-played at a later time when the user interacted with virtual objects [22]....
[...]
References
More filters
TL;DR: A broad class of up‐down methods used in psychoacoustics with due emphasis on the related problems of parameter estimation and the efficient placing of observations is described, including examples where conventional techniques are inapplicable.
Abstract: During the past decade a number of variations in the simple up‐down procedure have been used in psychoacoustic testing. A broad class of these methods is described with due emphasis on the related problems of parameter estimation and the efficient placing of observations. The advantages of up‐down methods are many, including simplicity, high efficiency, robustness, small‐sample reliability, and relative freedom from restrictive assumptions. Several applications of these procedures in psychoacoustics are described, including examples where conventional techniques are inapplicable.
5,306 citations
01 Feb 1990
TL;DR: The force display technology used in the Sandpaper system is a motor-driven two-degree of freedo m joystick, which computes the appropriate forces for the joystick's motors in real-time.
546 citations
TL;DR: In inspection and exploration tasks the detection of vibrations can be the fundamental goal of the task, while in some manipulation tasks vibrations can enhance performance by reducing reaction times or permitting minimization of ferees.
Abstract: This paper investigates the use of tactile displays for conveying task-related vibrations in teleoperation and virtual environments. Vibration displays can be implemented with inexpensive, open loop devices that can be added to many existing systems to improve performance. We describe the design of our prototype vibration sensing and display system, and experimentally demonstrate the utility of this type of tactile feedback. We also delineate the kinds of tasks where high-frequency vibratory feedback is important. In inspection and exploration tasks the detection of vibrations can be the fundamental goal of the task, while in some manipulation tasks vibrations can enhance performance by reducing reaction times or permitting minimization of ferees. Design guidelines for implementation of vibration displays, based on simple mechanical models, are also presented.
367 citations
"Creating Realistic Virtual Textures..." refers background or methods in this paper
...Beyond these recent examples, we have taken inspiration from the experimental designs of both Kontarinis and Howe [12] and Wall and Harwin [13] to develop a highfidelity vibration output system that can realistically render our tool-mediated texture models....
[...]
...However, no useful data can be recorded under zero force or speed conditions....
[...]
...However, they are seldom capable of accurately reproducing the high-frequency accelerations that occur during real-world contacts [2], [3]....
[...]
...In parallel, older research showed that high-frequency information is critical for high-fidelity haptic feedback in teleoperation [12], where the user controls the movement of a remote robot....
[...]
...However, little research has been done to relate these textures to real-world surfaces....
[...]
TL;DR: This paper reviews some of the recent research on haptic exploration, perception and recognition of multidimensional objects, and discusses several applications of the research programme to the development of tangible graphics displays for the blind, autonomous and teleoperated haptic robotic systems, and food evaluation in the food industry.
351 citations
"Creating Realistic Virtual Textures..." refers background in this paper
...Texture information is so important to how people interact with objects that researchers have labeled it as one of the primary means of human object exploration [6]....
[...]
TL;DR: A new method for generating appropriate transients inverts a dynamic model of the haptic device to determine the motor forces required to create prerecorded acceleration profiles at the user's fingertips, providing an important new avenue for increasing the realism of contact in haptic interactions.
Abstract: Tapping on surfaces in a typical virtual environment feels like contact with soft foam rather than a hard object. The realism of such interactions can be dramatically improved by superimposing event-based, high-frequency transient forces over traditional position-based feedback. When scaled by impact velocity, hand-tuned pulses and decaying sinusoids produce haptic cues that resemble those experienced during real impacts. Our new method for generating appropriate transients inverts a dynamic model of the haptic device to determine the motor forces required to create prerecorded acceleration profiles at the user's fingertips. After development, the event-based haptic paradigm and the method of acceleration matching were evaluated in a carefully controlled user study. Sixteen individuals blindly tapped on nine virtual and three real samples, rating the degree to which each felt like real wood. Event-based feedback achieved significantly higher realism ratings than the traditional rendering method. The display of transient signals made virtual objects feel similar to a real sample of wood on a foam substrate, while position feedback alone received ratings similar to those of foam. This work provides an important new avenue for increasing the realism of contact in haptic interactions.
343 citations
"Creating Realistic Virtual Textures..." refers methods in this paper
...For example, real-world contact data have been used directly for playback of cutting [27], tapping [3], and clicking [11], and it has been used indirectly to set the parameters of models for a toggle switch [28] and the texture of soil [29]....
[...]
...that occur during real-world contacts [2], [3]....
[...]