An algorithm for tracking a person's head is presented. The head's projection onto the image plane is modeled as an ellipse whose position and size are continually updated by a local search combining the output of a module concentrating on the intensity gradient around the ellipse's perimeter with that of another module focusing on the color histogram of the ellipse's interior. Since these two modules have roughly orthogonal failure modes, they serve to complement one another. The result is a robust, real-time system that is able to track a person's head with enough accuracy to automatically control the camera's pan, tilt, and zoom in order to keep the person centered in the field of view at a desired size. Extensive experimentation shows the algorithm's robustness with respect to full 360-degree out-of-plane rotation, up to 90-degree tilting, severe but brief occlusion, arbitrary camera movement, and multiple moving people in the background.

/pdf/elliptical-head-tracking-using-intensity-gradients-and-color-3ngtd1owpy.pdf

Elliptical head tracking using intensity gradients and color histograms

A method is presented for robust tracking in highly cluttered environments. The method makes effective use of 3D depth sensing technology, resulting in illumination-invariant tracking. A few applications using tracking are presented including face tracking and hand tracking.

Visual Tracking Using Depth Data

A system for estimating orientation of a target based on real-time video data uses depth data included in the video to determine the estimated orientation. The system includes a time-of-flight camera capable of depth sensing within a depth window. The camera outputs hybrid image data (color and depth). Segmentation is performed to determine the location of the target within the image. Tracking is used to follow the target location from frame to frame. During a training mode, a target-specific training image set is collected with a corresponding orientation associated with each frame. During an estimation mode, a classifier compares new images with the stored training set to determine an estimated orientation. A motion estimation approach uses an accumulated rotation/translation parameter calculation based on optical flow and depth constrains. The parameters are reset to a reference value each time the image corresponds to a dominant orientation.

Target orientation estimation using depth sensing

A simple algorithm for tracking a person's head is presented. A two-dimensional model, namely an ellipse, is used to approximate the head's contour. When a new image becomes available, a local search determines the position and size of the best ellipse by maximizing the normalized sum of the image gradient magnitude around the perimeter of the ellipse. The local search begins from a predicted position, using the head's velocity which eliminates the tracker's dependence upon maximum velocity. The tracker operates at 30 Hz and actively controls camera pan and tilt in order to track a person moving in a real environment. The algorithm tolerates full 360-degree rotation of the body as well as moderate amounts of occlusion, and it performs reacquisition of the subject.

/pdf/an-elliptical-head-tracker-3mixju9bc7.pdf

An elliptical head tracker

Robot hand-eye coordination has recently enjoyed much attention. Previous research at MIT has examined combining vision and manipulation applied to the task of tracking and catching tossed balls in controlled environments. Building upon the foundations of this past research, this paper presents work which incorporates a new active vision system which requires a minimally controlled environment, and implements methods for object tracking, robot/camera calibration, and new catching algorithms. Experimental results for real time catching of free-flying spherical balls are presented. The system was tested on under-hand tosses from random locations approximately 1.5–2.5 meters distant from the base of the arm. The best performance results were found to be 70–80% success for similar tosses.

Experiments in Hand-Eye Coordination Using Active Vision

Depth and motion measurements from images have a variety of industrial and non-industrial applications. Underlying these two measurement modalities is the ability to accurately and reliably correlate portions of images separated in space, as in the case of a stereo pair, or time, as in the case of a motion sequence. We have developed a unique correlation algorithm as well as special hardware accelerators to allow it to operate at video frame rate. The ability to make measurements in real time is invaluable in observing algorithm performance on dynamic scenes; real-time feedback has also allowed us to apply these low-level measurements in unique ways to several perceptual tasks. In this paper, we describe our progress in the areas of figure-ground separation and active object tracking using our correlation techniques. We illustrate the use of these capabilities to drive an active camera head and a robot vehicle in person-following demonstrations.© (1994) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

Real-time tracking of people using stereo and motion

A real-time active stereo vision system incorporating gaze control and task directed vision is described. Emphasis is placed on object tracking and object size and shape determination. Techniques include motion-centroid tracking, depth tracking, and contour tracking.

/pdf/object-tracking-with-stereo-vision-4fiw4pyou0.pdf

Object tracking with stereo vision

Computationally economical techniques for employing confidence measures to generate depth maps are presented in this paper. The NASA-JSC PRISM system has been successfully applied to experiments in manipulation (EVA Helper Retriever) and mobility (Mobile Robot Lab). Results from these experiments and future plans are also presented.

Confidence assessment for stereo depth mapping using a PRISM-3

The state-of-the-art in computing technology is rapidly attaining the performance necessary to implement many early vision algorithms at real-time rates. This new capability is helping to accelerate progress in vision research by improving our ability to evaluate the performance of algorithms in dynamic environments. In particular, we are becoming much more aware of the relative stability of various visual measurements in the presence of camera motion and system noise. This new processing speed is also allowing us to raise our sights toward accomplishing much higher-level processing tasks, such as figure-ground separation and active object tracking, in real-time. This paper describes a methodology for using early visual measurements to accomplish higher-level tasks; it then presents an overview of the high-speed accelerators developed at Teleos to support early visual measurements. The final section describes the successful deployment of a real-time vision system to provide visual perception for the Extravehicular Activity Helper/Retriever robotic system in tests aboard NASA's KC135 reduced gravity aircraft.

/pdf/real-time-tracking-using-stereo-and-motion-visual-perception-4yhhu214b2.pdf

Eric L. Huber

Papers

Real-time tracking of people using stereo and motion

Object tracking with stereo vision

Confidence assessment for stereo depth mapping using a PRISM-3

Real-time tracking using stereo and motion: Visual perception for space robotics