Articulated hand-tracking systems have been widely used in virtual reality but are rarely deployed in consumer applications due to their price and complexity. In this paper, we propose an easy-to-use and inexpensive system that facilitates 3-D articulated user-input using the hands. Our approach uses a single camera to track a hand wearing an ordinary cloth glove that is imprinted with a custom pattern. The pattern is designed to simplify the pose estimation problem, allowing us to employ a nearest-neighbor approach to track hands at interactive rates. We describe several proof-of-concept applications enabled by our system that we hope will provide a foundation for new interactions in modeling, animation control and augmented reality.

Real-time hand-tracking with a color glove

In this paper, we suggest a new approach for authoring tangible augmented reality applications, called 'immersive authoring.' The approach allows the user to carry out the authoring tasks within the AR application being built, so that the development and testing of the application can be done concurrently throughout the development process. We describe the functionalities and the interaction design for the proposed authoring system that are specifically targeted for intuitive specification of scenes and various object behaviors. Several cases of applications developed using the authoring system are presented. A small pilot user study was conducted to compare the proposed method to a non-immersive approach, and the results have shown that the users generally found it easier and faster to carry out authoring tasks in the immersive environment.

/pdf/immersive-authoring-of-tangible-augmented-reality-2xhbup6hgq.pdf

Immersive Authoring of Tangible Augmented Reality Applications

Spacedesign is an innovative mixed reality (MR) application addressed to aesthetic design of free form curves and surfaces. It is a unique and comprehensive approach which uses task-specific configurations to support the design workflow from concept to mock-up evaluation and review. The first-phase conceptual design benefits from a workbench-like 3-D display for free hand sketching, surfacing and engineering visualization. Semitransparent stereo glasses augment the pre-production physical prototype by additional shapes, textures and annotations. Both workspaces share a common interface and allow collaboration and cooperation between different experts, who can configure the system for the specific task. A faster design workflow and CAD data consistency can be thus naturally achieved. Tests and collaborations with designers, mainly from automotive industry, are providing systematic feedback for this ongoing research. As far as the authors are concerned, there is no known similar approach that integrates the creation and editing phase of 3D curves and surfaces in virtual and augmented reality (VR/AR). Herein we see the major contribution of our new application.

Spacedesign: a mixed reality workspace for aesthetic industrial design

Three-dimensional user interfaces (3D UIs) let users interact with virtual objects, environments, or information using direct 3D input in the physical and/or virtual space. In this article, the founders and organizers of the IEEE Symposium on 3D User Interfaces reflect on the state of the art in several key aspects of 3D UIs and speculate on future research.

3D User Interfaces: New Directions and Perspectives

Sketching in immersive 3D virtual reality (VR) environments has great potential for a variety of interactive 3D design applications. Precisely sketching the intended strokes in mid-air, however, can be a challenge. In this paper, we present a set of controlled studies to analyze the factors affecting human ability to sketch freely in a 3D VR environment. In our first study, we directly compare traditional sketching on a physical surface to sketching in VR, with and without a physical surface to rest the stylus on. Our results indicate that the lack of a physical drawing surface is a major cause of inaccuracies in VR drawing, and that the effect is dependent on the orientation of the drawing surface. In a second experiment, we evaluate the extent to which visual guidance can compensate for the loss of sketching precision in VR. We found that while additional visual guidance improves positional accuracy, it can be detrimental to the aesthetic quality of strokes. We conclude by distilling our experimental findings into design guidelines for sketching tools in immersive 3D environments.

Experimental Evaluation of Sketching on Surfaces in VR

A sketching system for spline-based free-form surfaces on the Responsive Workbench is presented. We propose 3D tools for curve drawing and deformation techniques for curves and surfaces, adapted to the needs of designers. The user directly draws curves in the virtual environment, using a tracked stylus as an input device. A curve network can be formed, describing the skeleton of a virtual model. The non-dominant hand positions and orients the model while the dominant hand uses the editing tools. The curves and the resulting skinning surfaces can interactively be deformed.

FreeDrawer: a free-form sketching system on the responsive workbench

This paper presents a collaborative pointing technique for colocated multi-user interaction in projection-based virtual environments. Our approach uses bent pick rays to allow users to collaboratively work together without locking objects. Moreover, a user can manipulate distant objects in immediate reach, using a Scaled- Grab technique. The main purpose of the bent pick ray is to provide continuous visual user feedback, keeping a user informed about the collaborative manipulation.

/pdf/the-bent-pick-ray-an-extended-pointing-technique-for-multi-5ci9m18w06.pdf

The Bent Pick Ray: An Extended Pointing Technique for Multi-User Interaction

A two-handed 3D styling system for free-form surfaces in a table-like Virtual Environment, the Responsive Workbench (RWB)TM, is described. Intuitive curve and surface deformation tools based on variational modeling and interaction techniques adapted to 3D VR modeling applications are proposed. The user draws curves (cubic B-splines) directly in the Virtual Environment using a stylus as an input device. The curves are connected automatically, such that a curve network develops. A combination of automatic and user-controlled topology extraction modules creates the connectivity information. The underlying surface model is based on B-spline surfaces, or, alternatively, uses multisided patches [20] bounded by closed loops of curve pieces.

Conceptual free-form styling on the responsive workbench

Virtual environments (VEs) such as the Responsive Workbench were designed to support interactive, complex applications. The possibility of direct manual interaction with virtual objects, using both hands in a natural, intuitive way is one of the great advantages of such configurations. The related work on interaction techniques for these environments is mainly concentrated on hand-oriented, spatial manipulation. With increasing complexity of applications, application control issues become more relevant, including the problem of tool selection. In this paper, we propose a new interaction technique, optimized for handling tool selection tasks comfortably, called the ToolFinger. The ToolFinger is a hand held 3D widget, resembling a pen, to which a set of tools is assigned in a way that selecting and applying a tool are no longer two separated tasks. The ToolFinger concept allows a fluent integration of tool selections and tool switches with the workflow of geometry editing. It benefits complex manipulation tasks in applications such as immersive geometric modelling, or interactive visualisation.

The ToolFinger: supporting complex direct manipulation in virtual environments

Recent studies have shown that through a careful combination of multiple sensory channels, so called multisensory binding effects can be achieved that can be beneficial for collision detection and texture recognition feedback. During the design of a new pen-input device called Tactylus, specific focus was put on exploring multisensory effects of audiotactile cues to create a new, but effective way to interact in virtual environments with the purpose to overcome several of the problems noticed in current devices.

Gerold Wesche

Papers

FreeDrawer: a free-form sketching system on the responsive workbench

The Bent Pick Ray: An Extended Pointing Technique for Multi-User Interaction

Conceptual free-form styling on the responsive workbench

The ToolFinger: supporting complex direct manipulation in virtual environments

Tactylus, a pen-input device exploring audiotactile sensory binding