This paper presents a method for extracting distinctive invariant features from images that can be used to perform reliable matching between different views of an object or scene. The features are invariant to image scale and rotation, and are shown to provide robust matching across a substantial range of affine distortion, change in 3D viewpoint, addition of noise, and change in illumination. The features are highly distinctive, in the sense that a single feature can be correctly matched with high probability against a large database of features from many images. This paper also describes an approach to using these features for object recognition. The recognition proceeds by matching individual features to a database of features from known objects using a fast nearest-neighbor algorithm, followed by a Hough transform to identify clusters belonging to a single object, and finally performing verification through least-squares solution for consistent pose parameters. This approach to recognition can robustly identify objects among clutter and occlusion while achieving near real-time performance.

/pdf/distinctive-image-features-from-scale-invariant-keypoints-3waurjqzke.pdf

Distinctive Image Features from Scale-Invariant Keypoints

An object recognition system has been developed that uses a new class of local image features. The features are invariant to image scaling, translation, and rotation, and partially invariant to illumination changes and affine or 3D projection. These features share similar properties with neurons in inferior temporal cortex that are used for object recognition in primate vision. Features are efficiently detected through a staged filtering approach that identifies stable points in scale space. Image keys are created that allow for local geometric deformations by representing blurred image gradients in multiple orientation planes and at multiple scales. The keys are used as input to a nearest neighbor indexing method that identifies candidate object matches. Final verification of each match is achieved by finding a low residual least squares solution for the unknown model parameters. Experimental results show that robust object recognition can be achieved in cluttered partially occluded images with a computation time of under 2 seconds.

/pdf/object-recognition-from-local-scale-invariant-features-5887b1cgwf.pdf

Object recognition from local scale-invariant features

The Scale-Invariant Feature Transform (or SIFT) algorithm is a highly robust method to extract and consequently match distinctive invariant features from images. These features can then be used to reliably match objects in diering images. The algorithm was rst proposed by Lowe [12] and further developed to increase performance resulting in the classic paper [13] that served as foundation for SIFT which has played an important role in robotic and machine vision in the past decade.

Computer vision is moving into a new era in which the aim is to develop visual skills for robots that allow them to interact with a dynamic, unconstrained environment. To achieve this aim, new kinds of vision algorithms need to be developed which run in real time and subserve the robot's goals. Two fundamental goals are determining the identity of an object with a known location, and determining the location of a known object. Color can be successfully used for both tasks.
This dissertation demonstrates that color histograms of multicolored objects provide a robust, efficient cue for indexing into a large database of models. It shows that color histograms are stable object representations in the presence of occlusion and over change in view, and that they can differentiate among a large number of objects. For solving the identification problem, it introduces a technique called Histogram Intersection, which matches model and image histograms and a fast incremental version of Histogram Intersection which allows real-time indexing into a large database of stored models. It demonstrates techniques for dealing with crowded scenes and with models with similar color signatures. For solving the location problem it introduces an algorithm called Histogram Backprojection which performs this task efficiently in crowded scenes.

Color indexing

A view-based approach to the representation and recognition of human movement is presented. The basis of the representation is a temporal template-a static vector-image where the vector value at each point is a function of the motion properties at the corresponding spatial location in an image sequence. Using aerobics exercises as a test domain, we explore the representational power of a simple, two component version of the templates: The first value is a binary value indicating the presence of motion and the second value is a function of the recency of motion in a sequence. We then develop a recognition method matching temporal templates against stored instances of views of known actions. The method automatically performs temporal segmentation, is invariant to linear changes in speed, and runs in real-time on standard platforms.

/pdf/the-recognition-of-human-movement-using-temporal-templates-5e8eze1cen.pdf

The recognition of human movement using temporal templates

The recognition of human movements such as walking, running or climbing has been approached previously by tracking a number of feature points and either classifying the trajectories directly or matching them with a high-level model of the movement. A major difficulty with these methods is acquiring and trading the requisite feature points, which are generally specific joints such as knees or angles. This requires previous recognition and/or part segmentation of the actor. We show that the recognition of walking or any repetitive motion activity can be accomplished on the basis of bottom up processing, which does not require the prior identification of specific parts, or classification of the actor. In particular, we demonstrate that repetitive motion is such a strong cue, that the moving actor can be segmented, normalized spatially and temporally, and recognized by matching against a spatiotemporal template of motion features. We have implemented a real-time system that can recognize and classify repetitive motion activities in normal gray-scale image sequences. >

/pdf/low-level-recognition-of-human-motion-or-how-to-get-your-man-dl9lzrhm1u.pdf

Low level recognition of human motion (or how to get your man without finding his body parts)

The use of certain measures of flow field divergence is investigated as a qualitative cue for obstacle avoidance during visual navigation. It is shown that a quantity termed the directional divergence of the 2-D motion field can be used as a reliable indicator of the presence of obstacles in the visual field of an observer undergoing generalized rotational and translational motion. The necessary measurements can be robustly obtained from real image sequences. Experimental results are presented showing that the system responds as expected to divergence in real-world image sequences, and the use of the system to navigate between obstacles is demonstrated. >

Obstacle avoidance using flow field divergence

The recognition of nonrigid motion, particularly that arising from human movement (and by extension from the locomotory activity of animals) has typically made use of high-level parametric models representing the various body parts (legs, arms, trunk, head etc) and their connections to each other Such model-based recognition has been successful in some cases; however, the methods are often difficult to apply to real-world scenes, and are severely limited in their generalizability The first problem arises from the difficulty of acquiring and tracking the requisite model parts, usually specific joints such as knees, elbows or ankles This generally requires some prior high-level understanding and segmentation of the scene, or initialization by a human operator The second problem, with generalization, is due to the fact that the human model is not much good for dogs or birds, and for each new type of motion, a new model must be hand-crafted In this paper, we show that the recognition of human or animal locomotion, and, in fact, any repetitive activity can be done using low-level, non-parametric representations Such an approach has the advantage that the same underlying representation is used for all examples, and no individual tailoring of models or prior scene understanding is required We show in particular, that repetitive motion is such a strong cue, that the moving actor can be segmented, normalized spatially and temporally, and recognized by matching against a spatio-temporal template of motion features We have implemented a real-time system that can recognize and classify repetitive motion activities in normal gray-scale image sequences Results on a number of real-world sequences are described

Detection and Recognition of Periodic, Nonrigid Motion

We present an experimental evaluation of adaptive and non-adaptive visual servoing in 3, 6 and 12 degrees of freedom (DOF), comparing it to traditional joint feedback control. While the purpose of experiments in most other work has been to show that the particular algorithm presented indeed also works in practice, we do not focus on the algorithm but rather on properties important to visual servoing in general. Our main results are: positioning of a 6 axis PUMA 762 arm is up to 5 times more precise under visual control than under joint control; positioning of a Utah/MIT dextrous hand is better under visual control than under joint control by a factor of 2; and a trust-region-based adaptive visual feedback controller is very robust. For m tracked visual features the algorithm can successfully estimate online the m/spl times/3 (m/spl ges/3) image Jacobian (J) without any prior information, while carrying out a 3 DOF manipulation task. For 6 and higher DOF manipulation, a rough initial estimate of J is beneficial. We also verified that redundant visual information is valuable. Errors due to imprecise tracking and goal specification were reduced as the number of visual features, m, was increased. Furthermore highly redundant systems allow us to detect outliers in the feature vector and deal with partial occlusion.

Experimental evaluation of uncalibrated visual servoing for precision manipulation

We describe a method of visual motion recognition applicable to a range of naturally occurring motions that are characterized by spatial and temporal uniformity. The underlying motivation is the observation that, for objects that typically move, it is frequently easier to identify them when they are moving than when they are stationary. Specifically, we show that certain statistical spatial and temporal features that can be derived from approximations to the motion field have invariant properties, and can be used to classify regional activities such as windblown trees, ripples on water, or chaotic fluid flow, that are characterized by complex, nonrigid motion. We refer to the technique as temporal texture analysis in analogy to the techniques developed to classify grayscale textures. This recognition approach contrasts with the reconstructive approach that has typified most prior work on motion. We demonstrate the technique on a number of real-world image sequences containing complex movement. The work has practical application in monitoring and surveillance, and as a component of a sophisticated visual system.

Randal C. Nelson

Papers

Low level recognition of human motion (or how to get your man without finding his body parts)

Obstacle avoidance using flow field divergence

Detection and Recognition of Periodic, Nonrigid Motion

Experimental evaluation of uncalibrated visual servoing for precision manipulation

Qualitative recognition of motion using temporal texture