The use of hand gestures provides an attractive alternative to cumbersome interface devices for human-computer interaction (HCI). In particular, visual interpretation of hand gestures can help in achieving the ease and naturalness desired for HCI. This has motivated a very active research area concerned with computer vision-based analysis and interpretation of hand gestures. We survey the literature on visual interpretation of hand gestures in the context of its role in HCI. This discussion is organized on the basis of the method used for modeling, analyzing, and recognizing gestures. Important differences in the gesture interpretation approaches arise depending on whether a 3D model of the human hand or an image appearance model of the human hand is used. 3D hand models offer a way of more elaborate modeling of hand gestures but lead to computational hurdles that have not been overcome given the real-time requirements of HCI. Appearance-based models lead to computationally efficient "purposive" approaches that work well under constrained situations but seem to lack the generality desirable for HCI. We also discuss implemented gestural systems as well as other potential applications of vision-based gesture recognition. Although the current progress is encouraging, further theoretical as well as computational advances are needed before gestures can be widely used for HCI. We discuss directions of future research in gesture recognition, including its integration with other natural modes of human-computer interaction.

/pdf/visual-interpretation-of-hand-gestures-for-human-computer-f24iuo372e.pdf

Visual interpretation of hand gestures for human-computer interaction: a review

A gesture recognition interface for use in controlling self-service machines and other devices is disclosed. A gesture is defined as motions and kinematic poses generated by humans, animals, or machines. Specific body features are tracked, and static and motion gestures are interpreted. Motion gestures are defined as a family of parametrically delimited oscillatory motions, modeled as a linear-in-parameters dynamic system with added geometric constraints to allow for real-time recognition using a small amount of memory and processing time. A linear least squares method is preferably used to determine the parameters which represent each gesture. Feature position measure is used in conjunction with a bank of predictor bins seeded with the gesture parameters, and the system determines which bin best fits the observed motion. Recognizing static pose gestures is preferably performed by localizing the body/object from the rest of the image, describing that object, and identifying that description. The disclosure details methods for gesture recognition, as well as the overall architecture for using gesture recognition to control of devices, including self-service machines.

/pdf/gesture-controlled-interfaces-for-self-service-machines-and-1e3rgeg8o4.pdf

Gesture-controlled interfaces for self-service machines and other applications

Three-dimensional position information is used to identify the gesture created by a body part of interest. At one or more instances of an interval, the posture of a body part is recognized, based on the shape of the body part and its position and orientation. The posture of the body part over each of the one or more instances in the interval are recognized as a combined gesture. The gesture is classified for determining an input into a related electronic device.

Gesture recognition system using depth perceptive sensors

A system and method are disclosed for providing a gesture recognition system for recognizing gestures made by a moving subject within an image and performing an operation based on the semantic meaning of the gesture. A subject, such as a human being, enters the viewing field of a camera connected to a computer and performs a gesture, such as flapping of the arms. The gesture is then examined by the system one image frame at a time. Positional data is derived from the input frames and compared to data representing gestures already known to the system. The comparisons are done in real-time and the system can be trained to better recognize known gestures or to recognize new gestures. A frame of the input image containing the subject is obtained after a background image model has been created. An input frame is used to derive a frame data set that contains particular coordinates of the subject at a given moment in time. This series of frame data sets is examined to determine whether it conveys a gesture that is known to the system. If the subject gesture is recognizable to the system, an operation based on the semantic meaning of the gesture can be performed by a computer.

Method and apparatus for real-time gesture recognition

A gesture recognition system includes: elements for detecting and generating a signal corresponding a number of markers arranged on an object, elements for processing the signal from the detecting elements, members for detecting position of the markers in the signal. The markers are divided into first and second set of markers, the first set of markers constituting a reference position and the system comprises elements for detecting movement of the second set of markers and generating a signal as a valid movement with respect to the reference position.

Gesture recognition system

A 3D human interface apparatus using a motion recognition based on a dynamic image processing in which the motion of an operator operated object as an imaging target can be recognized accurately and stably. The apparatus includes: an image input unit for entering a plurality of time series images of an object operated by the operator into a motion representing a command; a feature point extraction unit for extracting at least four feature points including at least three reference feature points and one fiducial feature point on the object, from each of the images; a motion recognition unit for recognizing the motion of the object by calculating motion parameters, according to an affine transformation determined from changes of positions of the reference feature points on the images, and a virtual parallax for the fiducial feature point expressing a difference between an actual position change on the images and a virtual position change according to the affine transformation; and a command input unit for inputting the command indicated by the motion of the object recognized by the motion recognition unit.

3D Human interface apparatus using motion recognition based on dynamic image processing

An efficient and geometrically intuitive algorithm for reliably interpreting the image velocities of moving objects in 3-D is presented. It is well known that under a weak perspective the image motion of points on a plane can be characterized by an affine transformation. It is shown that the relative image motion of a nearby non-coplanar point and its projection on the plane is equivalent to motion parallax, and because it is independent of view rotations it is a reliable geometric cue to 3-D shape and viewer/object motion. The authors summarize why structure from motion algorithms are often very sensitive to errors in the measured image velocities and then show how to efficiently and reliably extract an incomplete qualitative solution. They also show how to augment this into a complete solution if additional constraints or views are available. A real-time example is presented in which the 3-D visual interpretation of hand gestures or a hand-held object is used as part of a man-machine interface. This is an alternative to the Polhemus coil instrumented Dataglove commonly used in sensing manual gestures. >

/pdf/robust-structure-from-motion-using-motion-parallax-1072m2vlrd.pdf

Robust structure from motion using motion parallax

We present an eficient and gmrnetricdly intuitive Problems with this approach algorithm to reliably interpret the image velocities of moving objects in 31). rt is well known that in a small neighbourbood the image motion of points on a plane ia characterised by an afine transformation. We show th& the relative image motion of a nearby non-coplanar point and its projection on the plane is equivalent to motion pamE1nr and that it is a sehble geometric cue to 3D shape and viewer/ot>ject motion. In particular we ahow how to interpret the motion parallax vector of a fourth point and the turf, diue~gence and defownalion compw nents of the affine transformation (defined by the Ehree points of the plane) in order to recover the projection of the axis of rotation of a moving object: the change in relative poaition of the object; the rotation about the say; the till of the surface and a one parameter family of solutions for the slant as a function of the magnitude af tllc rotation of the object. The latter is a manifmtation of the bas-relief ambiguity. These measurements, allhough representing an incomplete solution to structure from motion, can be relierhly extracted from 2 views evcn when perspective effects are small. We present a real-time example in which the 3D vie sun1 interpretat ion of hand gestures or a hand-held object is used as part of a man-machine interface. Thin is an alternative to the Polhemua coil instrumented Datagfav~ commonly uspd in sensing manual gestures.

/pdf/qualitative-visual-interpretation-of-3d-hand-gestures-using-2f3qhg161a.pdf

Qualitative visual interpretation of 3D hand gestures using motion parallax

A visual navigation system for a robot working in a nuclear power plant is presented. Such a system should have two functions. One is to provide a rough environment map or directions towards a goal, such as turning left at the second corner. The other is to avoid obstacles and to find a passage for continuous movement of the robot. The operation of the system is divided into two stages. In the first stage, a multi-information local map (MIL map) describing information necessary for self-location measurement is created interactively from stereo images collected during remote-controlled navigation. The second stage is autonomous navigation. The robot moves autonomously by using the map with the aid of the second function of the system. The system consists of five subsystems: an environment teaching subsystem, to create the MIL map; a self-location measurement subsystem, to find the robot's own location; an obstacle detection subsystem; a path planning subsystem; and a path following subsystem. Experimental results from a mobile robot show the usefulness of the system. >

A visual navigation system using a multi-information local map

This paper proposes a vision processor for moving-object analysis in time-varying images. The process of motion analysis can be divided into three stages: moving-object candidate detection, object tracking, and final motion analysis. The processor Consists of three components corresponding to these three stages. The first isan overall image processing unitwith local parallel architecture. It locates candidate regions for moving objects. The second is a multimicroprocessor system consisting of 16 local modules. Each module tracks one candidate region. The third is the host workstation. In this paper, we describe both the architecture and the software of the vision processor.

Yasukazu Okamoto

Papers

3D Human interface apparatus using motion recognition based on dynamic image processing

Robust structure from motion using motion parallax

Qualitative visual interpretation of 3D hand gestures using motion parallax

A visual navigation system using a multi-information local map

Vision processor system for moving-object analysis