An algorithm for tracking a person's head is presented. The head's projection onto the image plane is modeled as an ellipse whose position and size are continually updated by a local search combining the output of a module concentrating on the intensity gradient around the ellipse's perimeter with that of another module focusing on the color histogram of the ellipse's interior. Since these two modules have roughly orthogonal failure modes, they serve to complement one another. The result is a robust, real-time system that is able to track a person's head with enough accuracy to automatically control the camera's pan, tilt, and zoom in order to keep the person centered in the field of view at a desired size. Extensive experimentation shows the algorithm's robustness with respect to full 360-degree out-of-plane rotation, up to 90-degree tilting, severe but brief occlusion, arbitrary camera movement, and multiple moving people in the background.

/pdf/elliptical-head-tracking-using-intensity-gradients-and-color-3ngtd1owpy.pdf

Elliptical head tracking using intensity gradients and color histograms

A method is presented for robust tracking in highly cluttered environments. The method makes effective use of 3D depth sensing technology, resulting in illumination-invariant tracking. A few applications using tracking are presented including face tracking and hand tracking.

Visual Tracking Using Depth Data

A system for estimating orientation of a target based on real-time video data uses depth data included in the video to determine the estimated orientation. The system includes a time-of-flight camera capable of depth sensing within a depth window. The camera outputs hybrid image data (color and depth). Segmentation is performed to determine the location of the target within the image. Tracking is used to follow the target location from frame to frame. During a training mode, a target-specific training image set is collected with a corresponding orientation associated with each frame. During an estimation mode, a classifier compares new images with the stored training set to determine an estimated orientation. A motion estimation approach uses an accumulated rotation/translation parameter calculation based on optical flow and depth constrains. The parameters are reset to a reference value each time the image corresponds to a dominant orientation.

Target orientation estimation using depth sensing

A simple algorithm for tracking a person's head is presented. A two-dimensional model, namely an ellipse, is used to approximate the head's contour. When a new image becomes available, a local search determines the position and size of the best ellipse by maximizing the normalized sum of the image gradient magnitude around the perimeter of the ellipse. The local search begins from a predicted position, using the head's velocity which eliminates the tracker's dependence upon maximum velocity. The tracker operates at 30 Hz and actively controls camera pan and tilt in order to track a person moving in a real environment. The algorithm tolerates full 360-degree rotation of the body as well as moderate amounts of occlusion, and it performs reacquisition of the subject.

/pdf/an-elliptical-head-tracker-3mixju9bc7.pdf

An elliptical head tracker

Robot hand-eye coordination has recently enjoyed much attention. Previous research at MIT has examined combining vision and manipulation applied to the task of tracking and catching tossed balls in controlled environments. Building upon the foundations of this past research, this paper presents work which incorporates a new active vision system which requires a minimally controlled environment, and implements methods for object tracking, robot/camera calibration, and new catching algorithms. Experimental results for real time catching of free-flying spherical balls are presented. The system was tested on under-hand tosses from random locations approximately 1.5–2.5 meters distant from the base of the arm. The best performance results were found to be 70–80% success for similar tosses.

Experiments in Hand-Eye Coordination Using Active Vision

Depth and motion measurements from images have a variety of industrial and non-industrial applications. Underlying these two measurement modalities is the ability to accurately and reliably correlate portions of images separated in space, as in the case of a stereo pair, or time, as in the case of a motion sequence. We have developed a unique correlation algorithm as well as special hardware accelerators to allow it to operate at video frame rate. The ability to make measurements in real time is invaluable in observing algorithm performance on dynamic scenes; real-time feedback has also allowed us to apply these low-level measurements in unique ways to several perceptual tasks. In this paper, we describe our progress in the areas of figure-ground separation and active object tracking using our correlation techniques. We illustrate the use of these capabilities to drive an active camera head and a robot vehicle in person-following demonstrations.© (1994) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

Real-time tracking of people using stereo and motion

: This report documents work carried out at Teleos Research in support of the UGV Demo II mission. This research has been divided between basic studies relating to binocular stereo, shape recognition and technology development. Teleos' approach to the study of visual perception embodies a strategy for focusing on the minimal form of perceptual measurement that is meaningful or useful. Several new stereo techniques were developed under this research program including: a way to automatically set stereo matcher operating parameters, an analysis of stereo disparity gradients, and a method for improving area correlator performance in the presence of disparity gradients. Several new techniques for shape detection and recognition are reported. An active vision system for tracking moving objects applies results from the stereo and shape recognition work. A number of efforts were directed at supporting the UGV mission, including the use of narrow-field-of-view stereo for vehicle navigation; very high resolution wide-field-of-view stereo to support landmark based navigation; test data collection for comparison and performance evaluation; and collaborative efforts with the other UGV stereo contractors at SRI and JPL to foster technology transfer.

/pdf/arpa-unmanned-ground-vehicle-stereo-vision-program-1fnhcfwtls.pdf

ARPA Unmanned Ground Vehicle Stereo Vision Program

: This report describes work accomplished by Teleos Research in support of DARPA's Unmanned Ground Vehicle (UGV) Program. The presentation focuses on making forward- looking projections about opportunities for visual sensing on unmanned military vehicles based on the experience of this research program. A case is made for deploying UGV's following a small system model. Trends in commodity processor technology enable this possibility. The relationship between system cost and operational conservatism, system size and fragility, and speed and effort reguired, all support a shift towards smaller, cheaper implementations. We believe that this development model will rapidly take hold over the coming years. This report also presents a brief review of stereo performance characteristics relevant to the UGV mobility application. Several new techniques for enhancing stereo performance, including soft surface detection and disparity gradient compensation, are described.

/pdf/active-stereo-and-motion-vision-for-vehicle-navigation-1pv3peh779.pdf

Active Stereo and Motion Vision for Vehicle Navigation

: Teleos is developing and integrating active vision technology with intelligent, domain-specific control strategies to provide real-time visual detection, characterization, and tracking of objects in the security and surveillance domain Work to date has focused on theoretical studies and experimentation in the areas of attentional mechanisms, figure-ground discrimination, recognition, and active sensor mechanisms to support visual surveillance The primary research focus is on the detection and tracking of humans in a monitored area This report describes new results in blob-based shape recognition; the use of oriented texture blobs for face-finding; and stereo based real-time figure-ground discrimination techniques

http://www.dtic.mil/cgi-bin/GetTRDoc?AD=ADA351843&Location=U2&doc=GetTRDoc.pdf

H. K. Nishihara

Papers

Real-time tracking of people using stereo and motion

ARPA Unmanned Ground Vehicle Stereo Vision Program

Active Stereo and Motion Vision for Vehicle Navigation

Task-Driven Active Vision for Security and Surveillance.