The goal of this article is to review the state-of-the-art tracking methods, classify them into different categories, and identify new trends. Object tracking, in general, is a challenging problem. Difficulties in tracking objects can arise due to abrupt object motion, changing appearance patterns of both the object and the scene, nonrigid object structures, object-to-object and object-to-scene occlusions, and camera motion. Tracking is usually performed in the context of higher-level applications that require the location and/or shape of the object in every frame. Typically, assumptions are made to constrain the tracking problem in the context of a particular application. In this survey, we categorize the tracking methods on the basis of the object and motion representations used, provide detailed descriptions of representative methods in each category, and examine their pros and cons. Moreover, we discuss the important issues related to tracking including the use of appropriate image features, selection of motion models, and detection of objects.

/pdf/object-tracking-a-survey-3706dclxb5.pdf

Object tracking: A survey

An introduction to the Kalman filter

In 1960, R.E. Kalman published his famous paper describing a recursive solution to the discrete-data linear filtering problem. Since that time, due in large part to advances in digital computing, the Kalman filter has been the subject of extensive research and application, particularly in the area of autonomous or assisted navigation. The Kalman filter is a set of mathematical equations that provides an efficient computational (recursive) means to estimate the state of a process, in a way that minimizes the mean of the squared error. The filter is very powerful in several aspects: it supports estimations of past, present, and even future states, and it can do so even when the precise nature of the modeled system is unknown. The purpose of this paper is to provide a practical introduction to the discrete Kalman filter. This introduction includes a description and some discussion of the basic discrete Kalman filter, a derivation, description and some discussion of the extended Kalman filter, and a relatively simple (tangible) example with real numbers & results.

/pdf/an-introduction-to-the-kalman-filter-4na56gymup.pdf

An Introduction to the Kalman Filter

The goal of this paper is to present a critical survey of existing literature on human and machine recognition of faces. Machine recognition of faces has several applications, ranging from static matching of controlled photographs as in mug shots matching and credit card verification to surveillance video images. Such applications have different constraints in terms of complexity of processing requirements and thus present a wide range of different technical challenges. Over the last 20 years researchers in psychophysics, neural sciences and engineering, image processing analysis and computer vision have investigated a number of issues related to face recognition by humans and machines. Ongoing research activities have been given a renewed emphasis over the last five years. Existing techniques and systems have been tested on different sets of images of varying complexities. But very little synergism exists between studies in psychophysics and the engineering literature. Most importantly, there exists no evaluation or benchmarking studies using large databases with the image quality that arises in commercial and law enforcement applications In this paper, we first present different applications of face recognition in commercial and law enforcement sectors. This is followed by a brief overview of the literature on face recognition in the psychophysics community. We then present a detailed overview of move than 20 years of research done in the engineering community. Techniques for segmentation/location of the face, feature extraction and recognition are reviewed. Global transform and feature based methods using statistical, structural and neural classifiers are summarized. >

Human and machine recognition of faces: a survey

Inferring scene geometry and camera motion from a stream of images is possible in principle, but is an ill-conditioned problem when the objects are distant with respect to their size. We have developed a factorization method that can overcome this difficulty by recovering shape and motion under orthography without computing depth as an intermediate step. An image stream can be represented by the 2FxP measurement matrix of the image coordinates of P points tracked through F frames. We show that under orthographic projection this matrix is of rank 3. Based on this observation, the factorization method uses the singular-value decomposition technique to factor the measurement matrix into two matrices which represent object shape and camera rotation respectively. Two of the three translation components are computed in a preprocessing stage. The method can also handle and obtain a full solution from a partially filled-in measurement matrix that may result from occlusions or tracking failures. The method gives accurate results, and does not introduce smoothing in either shape or motion. We demonstrate this with a series of experiments on laboratory and outdoor image streams, with and without occlusions.

/pdf/shape-and-motion-from-image-streams-under-orthography-a-3mcosr5hgx.pdf

Shape and motion from image streams under orthography: a factorization method

An approach is presented for the estimation of object motion parameters based on a sequence of noisy images. The problem considered is that of a rigid body undergoing unknown rotational and translational motion. The measurement data consists of a sequence of noisy image coordinates of two or more object correspondence points. By modeling the object dynamics as a function of time, estimates of the model parameters (including motion parameters) can be extracted from the data using recursive and/or batch techniques. This permits a desired degree of smoothing to be achieved through the use of an arbitrarily large number of images. Some assumptions regarding object structure are presently made. Results are presented for a recursive estimation procedure: the case considered here is that of a sequence of one dimensional images of a two dimensional object. Thus, the object moves in one transverse dimension, and in depth, preserving the fundamental ambiguity of the central projection image model (loss of depth information). An iterated extended Kalman filter is used for the recursive solution. Noise levels of 5-10 percent of the object image size are used. Approximate Cramer-Rao lower bounds are derived for the model parameter estimates as a function of object trajectory and noise level. This approach may be of use in situations where it is difficult to resolve large numbers of object match points, but relatively long sequences of images (10 to 20 or more) are available.

Estimation of Object Motion Parameters from Noisy Images

Consideration is given to the design and application of a recursive algorithm to a sequence of images of a moving object to estimate both its structure and kinematics. The object is assumed to be rigid, and its motion is assumed to be smooth in the sense that it can be modeled by retaining an arbitrary number of terms in the appropriate Taylor series expansions. Translational motion involves a standard rectilinear model, while rotational motion is described with quaternions. Neglected terms of the Taylor series are modeled as process noise. A state-space model is constructed, incorporating both kinematic and structural states, and recursive techniques are used to estimate the state vector as a function of time. A set of object match points is assumed to be available. The problem is formulated as a parameter estimation and tracking problem which can use an arbitrarily large number of images in a sequence. The recursive estimation is done using an iterated extended Kalman filter (IEKF), initialized with the output of a batch algorithm run on the first few frames. Approximate Cramer-Rao lower bounds on the error covariance of the batch estimate are used as the initial state estimate error covariance of the IEKF. The performance of the recursive estimator is illustrated using both real and synthetic image sequences. >

Recursive 3-D motion estimation from a monocular image sequence

The problem considered here involves the use of a sequence of monocular images of a three-dimensional moving object to estimate both its structure and kinematics. The object is assumed to be rigid, and its motion is assumed to be smooth. A set of object match points is assumed to be available, consisting of fixed features on the object, the image plane coordinates of which have been extracted from successive images in the sequence. The measured data are the noisy image plane coordinates of this set of object match points, taken from each image in the sequence. In previous papers [ IEEE Trans. Pattern Anal. Mach. Intell.PAMI-8, 90 ( 1986); in Proceedings of the IEEE Workshop on Motion: Representation and Analysis ( Institute of Electrical and Electronics Engineers, New York, 1986), p. 95; in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition ( Institute of Electrical and Electronics Engineers, New York, 1986), p. 176] we discussed model-based approaches for motion and structure estimation from a long sequence of images. We examine here the performance of such techniques by using a Cramer–Rao lower bounds on the estimation error variance. This method permits a priori prediction of estimation accuracy as a function of a number of factors, including the number of images in the sequence, the time at which each image is made, the number of feature points used, the image-plane noise level, and the type of motion that is involved. Theoretical performance predictions are compared with the statistics of Monte Carlo simulation, and it is shown that the actual estimation accuracy is close to the Cramer–Rao bounds in most cases. These results also show that noisy sequences with fewer than four images often do not contain enough information to permit accurate estimation of motion and structure parameters. This conclusion is consistent with the observed instability of so-called two-frame estimation methods in the presence of noise.

Ted J. Broida

Papers

Estimation of Object Motion Parameters from Noisy Images

Recursive 3-D motion estimation from a monocular image sequence

Performance bounds for estimating three-dimensional motion parameters from a sequence of noisy images