Human-Computer Interaction (HCI) exists ubiquitously in our daily lives. It is usually achieved by using a physical controller such as a mouse, keyboard or touch screen. It hinders Natural User Interface (NUI) as there is a strong barrier between the user and computer. There are various hand tracking systems available on the market, but they are complex and expensive. In this paper, we present the design and development of a robust marker-less hand/finger tracking and gesture recognition system using low-cost hardware. We propose a simple but efficient method that allows robust and fast hand tracking despite complex background and motion blur. Our system is able to translate the detected hands or gestures into different functional inputs and interfaces with other applications via several methods. It enables intuitive HCI and interactive motion gaming. We also developed sample applications that can utilize the inputs from the hand tracking system. Our results show that an intuitive HCI and motion gaming system can be achieved with minimum hardware requirements.

Hand tracking and gesture recognition system for human-computer interaction using low-cost hardware

Traditional Tangible Augmented Reality (Tangible AR) interfaces combine a mixture of tangible user interface and augmented reality technology, complementing each other for novel interaction methods and real world anchored visualization. However, well known conventional one and two dimensional interaction methods such as pressing buttons, changing slider values, or menu selections are often quite difficult to apply to Tangible AR interfaces. In this paper we suggest a new approach, occlusion based interaction, in which visual occlusion of physical markers are used to provide intuitive two dimensional interaction in Tangible AR environments. We describe how to implement occlusion based interfaces for Tangible AR environments, give several examples of applications and describe results from informal user studies.

Occlusion based interaction methods for tangible augmented reality environments

This article presents an application of Augmented Reality technology for interior design. Plus, an Educational Interior Design Project is reviewed. Along with the dramatic progress of digital technology, virtual information techniques are also required for architectural projects. Thus, the new technology of Augmented Reality offers many advantages for digital architectural design and construction fields. AR is also being considered as a new design approach for interior design. In an AR environment, the virtual furniture can be displayed and modified in real-time on the screen, allowing the user to have an interactive experience with the virtual furniture in a real-world environment. Here, AR environment is exploited as the new working environment for architects in architectural design works, and then they can do their work conveniently as such collaborative discussion through AR environment. Finally, this study proposes a new method for applying AR technology to interior design work, where a user can view virtual furniture and communicate with 3D virtual furniture data using a dynamic and flexible user interface. Plus, all the properties of the virtual furniture can be adjusted using occlusionbased interaction method for a Tangible Augmented Reality.

/pdf/interior-design-in-augmented-reality-environment-1qhv49w3mi.pdf

Interior Design in Augmented Reality Environment

The control of computers and electronics through hand gestures has gained significant industry and academic attention lately for the usability benefits and convenience that it offers users. Of particular research interest has been the control of living room environments containing televisions and set-top boxes. However, existing research has failed to provide a flexible solution for controlling such devices by hand gestures. They have used cameras that are sensitive to environmental factors such as lighting or that have unreasonable calibration demands. Additionally, the gesture processing techniques used so far have imposed considerable computational burden and have not provided a consistent and compelling TV control experience for a large variety of users and their homes. In this paper, the data returned from a custom 3D depth camera and a customizable gesture language is used to create an intelligent gesture interface for the control of TVs and set-top boxes. By using an infrared blaster to emit the commands typical of a physical remote, any television set or set-top box can be controlled to perform actions such as turning the TV on, changing the volume, muting the sound or changing the channel. Finally, a test setup is presented where a common television and a satellite receiver are controlled exclusively through hand gestures.

An intelligent gesture interface for controlling TV sets and set-top boxes

Tracking is an important section in a gesture recognition system. Numerous techniques for segmentation, tracking, modeling and recognition have been proposed during several past years. A few papers comparing different approaches have been published [31,32]. However, a comprehensive survey on tracking is still missing. We try to fill this vacuum by reviewing most widely used methods and techniques and collecting their numerical evaluation results.

Visual Hand Tracking Algorithms

In this paper, we describe a hand gesture recognition engine based on computer vision (CV), as a computing platform to support gesture interaction between humans and computers. Presenting a simple approach to recognizing gestures through image processing techniques and a single video camera, we address the problem of generic hand gestures recognition, especially of spelled sign language hand poses, introducing a preliminary study to its kinetic component. In our methodology, the system initially removes the background of captured images, eliminating irrelevant pixel information. The human hand is then detected, segmented and its contours localized. From these contours significant metrics are derived, allowing a search in a pre-defined hand poses' library, where each pose is previously converted into a set of metric values. We discuss several algorithmic options to support our methodology and present experimental results, regarding the recognition of Portuguese sign language signs. We discuss future directions of our work.

OGRE - open gestures recognition engine

In this paper, mixed reality (MR) is proposed as a new framework that supplements the definition of augmented reality (AR) MR includes the continuum transitions from real environments (RE), to AR, passing through augmented virtuality (AV) towards fully virtual environments (VE) In the context of MR and according to H Kato et al (2001), tangible interfaces are those in which: 1) each virtual object is registered to a (tangible) physical object; and 2) the user interacts with virtual objects by manipulating the corresponding tangible object Tangible interfaces are described in the literature as being intuitive, since physical object manipulations are mapped to virtual object operations Users generally require intuitive real-time interaction in MR Having this in mind, we have come up with a series of novel sensor-less and cable-less tangible interfaces In the present work, we add novel uses for these interfaces since they are the only physical means of user interaction that triggers the functionalities of the system

http://ieeexplore.ieee.org/iel5/9228/29246/01320422.pdf

A. Correia

Papers

OGRE - open gestures recognition engine

In your hand computing: tangible interfaces for mixed reality