Quadcopters have been used as hovering encountered-type haptic devices in virtual reality. We suggest that quadcopters can facilitate rich haptic interactions beyond force feedback by appropriating physical objects and the environment. We present HoverHaptics, an autonomous safe-to-touch quadcopter and its integration with a virtual shopping experience. HoverHaptics highlights three affordances of quadcopters that enable these rich haptic interactions: (1) dynamic positioning of passive haptics, (2) texture mapping, and (3) animating passive props. We identify inherent challenges of hovering encountered-type haptic devices, such as their limited speed, inadequate control accuracy, and safety concerns. We then detail our approach for tackling these challenges, including the use of display techniques, visuo-haptic illusions, and collision avoidance. We conclude by describing a preliminary study (n = 9) to better understand the subjective user experience when interacting with a quadcopter in virtual reality using these techniques.

/pdf/beyond-the-force-using-quadcopters-to-appropriate-objects-3k9sga7yg2.pdf

Beyond The Force: Using Quadcopters to Appropriate Objects and the Environment for Haptics in Virtual Reality

Standard controllers for virtual reality (VR) lack sophisticated means to convey a realistic, kinesthetic impression of size, resistance or inertia. We present the concept and implementation of Drag:on, an ungrounded shape-changing VR controller that provides dynamic passive haptic feedback based on drag, i.e. air resistance, and weight shift. Drag:on leverages the airflow occurring at the controller during interaction. By dynamically adjusting its surface area, the controller changes the drag and rotational inertia felt by the user. In a user study, we found that Drag:on can provide distinguishable levels of haptic feedback. Our prototype increases the haptic realism in VR compared to standard controllers and when rotated or swung improves the perception of virtual resistance. By this, Drag:on provides haptic feedback suitable for rendering different virtual mechanical resistances, virtual gas streams, and virtual objects differing in scale, material and fill state.

Drag:on: A Virtual Reality Controller Providing Haptic Feedback Based on Drag and Weight Shift

In this work, we utilize visuo-haptic illusions to improve the perceived performance of encountered-type haptic devices, specifically shape displays, in virtual reality. Shape displays are matrices of actuated pins that travel vertically to render physical shapes; however, they have limitations such as low resolution, small display size, and low pin speed. To address these limitations, we employ illusions such as redirection, scaling, and retargeting that take advantage of the visual dominance effect, the idea that vision often dominates when senses conflict. Our evaluation of these techniques suggests that redirecting sloped lines with angles less than 40 degrees onto a horizontal line is an effective technique for increasing the perceived resolution of the display. Scaling up the virtual object onto the shape display by a factor less than 1.8x can also increase the perceived resolution. Finally, using vertical redirection a perceived 3x speed increase can be achieved.

Visuo-Haptic Illusions for Improving the Perceived Performance of Shape Displays

RoomShift is a room-scale dynamic haptic environment for virtual reality, using a small swarm of robots that can move furniture. RoomShift consists of nine shape-changing robots: Roombas with mechanical scissor lifts. These robots drive beneath a piece of furniture to lift, move and place it. By augmenting virtual scenes with physical objects, users can sit on, lean against, place and otherwise interact with furniture with their whole body; just as in the real world. When the virtual scene changes or users navigate within it, the swarm of robots dynamically reconfigures the physical environment to match the virtual content. We describe the hardware and software implementation, applications in virtual tours and architectural design and interaction techniques.

RoomShift: Room-scale Dynamic Haptics for VR with Furniture-moving Swarm Robots

We present GridDrones, a self-levitating programmable matter platform that can be used for representing 2.5D voxel grid relief maps capable of rendering unsupported structures and 3D transformations. GridDrones consists of cube-shaped nanocopters that can be placed in a volumetric 1xnxn mid-air grid, which is demonstrated here with 15 voxels. The number of voxels and scale is only limited by the size of the room and budget. Grid deformations can be applied interactively to this voxel lattice by manually selecting a set of voxels, then assigning a continuous topological relationship between voxel sets that determines how voxels move in relation to each other and manually drawing out selected voxels from the lattice structure. Using this simple technique, it is possible to create unsupported structures that can be translated and oriented freely in 3D. Shape transformations can also be recorded to allow for simple physical shape morphing animations. This work extends previous work on selection and editing techniques for 3D user interfaces.

GridDrones: A Self-Levitating Physical Voxel Lattice for Interactive 3D Surface Deformations

We explore interactions enabled by 2D spatial manipulation and self-actuation of a tabletop shape display. To explore these interactions, we developed shapeShift, a compact, high-resolution (7 mm pitch), mobile tabletop shape display. shapeShift can be mounted on passive rollers allowing for bimanual interaction where the user can freely manipulate the system while it renders spatially relevant content. shapeShift can also be mounted on an omnidirectional-robot to provide both vertical and lateral kinesthetic feedback, display moving objects, or act as an encountered-type haptic device for VR. We present a study on haptic search tasks comparing spatial manipulation of a shape display for egocentric exploration of a map versus exploration using a fixed display and a touch pad. Results show a 30% decrease in navigation path lengths, 24% decrease in task time, 15% decrease in mental demand and 29% decrease in frustration in favor of egocentric navigation.

https://dl.acm.org/doi/pdf/10.1145/3173574.3173865

shapeShift: 2D Spatial Manipulation and Self-Actuation of Tabletop Shape Displays for Tangible and Haptic Interaction

shapeShift is a compact, high-resolution (7 mm pitch), mobile tabletop shape display. We explore potential interaction techniques in both passive and active mobile scenarios. In the passive case, the user is able to freely move and spin the display as it renders elements. We introduce use cases for rendering lateral I/O elements, exploring volumetric datasets, and grasping and manipulating objects. On an active omnidirectional-robot platform, shapeShift can display moving objects and provide both vertical and lateral kinesthetic feedback. We use the active platform as an encounter-type haptic device combined with a head-mounted display to dynamically simulate the presence of virtual content.

https://dl.acm.org/doi/pdf/10.1145/3131785.3131792

shapeShift: A Mobile Tabletop Shape Display for Tangible and Haptic Interaction

In this paper, we extend the field of view (FoV) of a glasses-free multilayer light field display, which we have assembled. By integrating a camera into the rendering and display pipeline, we constrain our light field display to a narrow FoV at the center of the display. As the viewing cone follows the viewer’s gaze, we optimize our iterative tomographic reconstruction to be utilized more efficiently and extend the apparent FoV from 10° to 100°. Finally, we demonstrate the realtime generation of dynamic light field scenes on the graphics processing unit.

Jason Ginsberg

Papers

shapeShift: 2D Spatial Manipulation and Self-Actuation of Tabletop Shape Displays for Tangible and Haptic Interaction

shapeShift: A Mobile Tabletop Shape Display for Tangible and Haptic Interaction

Dynamic Field of View in a Tomographic Light Field Display