Showing papers on "Motion estimation published in 2003"

Two-frame motion estimation based on polynomial expansion

Abstract: This paper presents a novel two-frame motion estimation algorithm. The first step is to approximate each neighborhood of both frames by quadratic polynomials, which can be done efficiently using the polynomial expansion transform. From observing how an exact polynomial transforms under translation a method to estimate displacement fields from the polynomial expansion coefficients is derived and after a series of refinements leads to a robust algorithm. Evaluation on the Yosemite sequence shows good results.

Real-time simultaneous localisation and mapping with a single camera

Abstract: Ego-motion estimation for an agile single camera moving through general, unknown scenes becomes a much more challenging problem when real-time performance is required rather than under the off-line processing conditions under which most successful structure from motion work has been achieved. This task of estimating camera motion from measurements of a continuously expanding set of self-mapped visual features is one of a class of problems known as Simultaneous Localisation and Mapping (SLAM) in the robotics community, and we argue that such real-time mapping research, despite rarely being camera-based, is more relevant here than off-line structure from motion methods due to the more fundamental emphasis placed on propagation of uncertainty. We present a top-down Bayesian framework for single-camera localisation via mapping of a sparse set of natural features using motion modelling and an information-guided active measurement strategy, in particular addressing the difficult issue of real-time feature initialisation via a factored sampling approach. Real-time handling of uncertainty permits robust localisation via the creating and active measurement of a sparse map of landmarks such that regions can be re-visited after periods of neglect and localisation can continue through periods when few features are visible. Results are presented of real-time localisation for a hand-waved camera with very sparse prior scene knowledge and all processing carried out on a desktop PC.

Robust online appearance models for visual tracking

Abstract: We propose a framework for learning robust, adaptive, appearance models to be used for motion-based tracking of natural objects. The model adapts to slowly changing appearance, and it maintains a natural measure of the stability of the observed image structure during tracking. By identifying stable properties of appearance, we can weight them more heavily for motion estimation, while less stable properties can be proportionately downweighted. The appearance model involves a mixture of stable image structure, learned over long time courses, along with two-frame motion information and an outlier process. An online EM-algorithm is used to adapt the appearance model parameters over time. An implementation of this approach is developed for an appearance model based on the filter responses from a steerable pyramid. This model is used in a motion-based tracking algorithm to provide robustness in the face of image outliers, such as those caused by occlusions, while adapting to natural changes in appearance such as those due to facial expressions or variations in 3D pose.

An efficient solution to the five-point relative pose problem

Abstract: An efficient algorithmic solution to the classical five-point relative pose problem is presented. The problem is to find the possible solutions for relative camera motion between two calibrated views given five corresponding points. The algorithm consists of computing the coefficients of a tenth degree polynomial and subsequently finding its roots. It is the first algorithm well suited for numerical implementation that also corresponds to the inherent complexity of the problem. The algorithm is used in a robust hypothesis-and-test framework to estimate structure and motion in real-time.

Free-viewpoint video of human actors

Abstract: In free-viewpoint video, the viewer can interactively choose his viewpoint in 3-D space to observe the action of a dynamic real-world scene from arbitrary perspectives. The human body and its motion plays a central role in most visual media and its structure can be exploited for robust motion estimation and efficient visualization. This paper describes a system that uses multi-view synchronized video footage of an actor's performance to estimate motion parameters and to interactively re-render the actor's appearance from any viewpoint.The actor's silhouettes are extracted from synchronized video frames via background segmentation and then used to determine a sequence of poses for a 3D human body model. By employing multi-view texturing during rendering, time-dependent changes in the body surface are reproduced in high detail. The motion capture subsystem runs offline, is non-intrusive, yields robust motion parameter estimates, and can cope with a broad range of motion. The rendering subsystem runs at real-time frame rates using ubiquous graphics hardware, yielding a highly naturalistic impression of the actor. The actor can be placed in virtual environments to create composite dynamic scenes. Free-viewpoint video allows the creation of camera fly-throughs or viewing the action interactively from arbitrary perspectives.

Preemptive RANSAC for Live Structure and Motion Estimation

Abstract: A system capable of performing robust live ego-motion estimation for perspective cameras is presented. The system is powered by random sample consensus with preemptive scoring of the motion hypotheses. A general statement of the problem of efficient preemptive scoring is given. Then a theoretical investigation of preemptive scoring under a simple inlier-outlier model is performed. A practical preemption scheme is proposed and it is shown that the preemption is powerful enough to enable robust live structure and motion estimation.

Preemptive RANSAC for live structure and motion estimation

Abstract: A system capable of performing robust live ego-motion estimation for perspective cameras is presented. The system is powered by random sample consensus with preemptive scoring of the motion hypotheses. A general statement of the problem of efficient preemptive scoring is given. Then a theoretical investigation of preemptive scoring under a simple inlier-outlier model is performed. A practical preemption scheme is proposed and it is shown that the preemption is powerful enough to enable robust live structure and motion estimation.

Shape-from-silhouette of articulated objects and its use for human body kinematics estimation and motion capture

Abstract: Shape-from-silhouette (SFS), also known as visual hull (VH) construction, is a popular 3D reconstruction method, which estimates the shape of an object from multiple silhouette images. The original SFS formulation assumes that the entire silhouette images are captured either at the same time or while the object is static. This assumption is violated when the object moves or changes shape. Hence the use of SFS with moving objects has been restricted to treating each time instant sequentially and independently. Recently we have successfully extended the traditional SFS formulation to refine the shape of a rigidly moving object over time. We further extend SFS to apply to dynamic articulated objects. Given silhouettes of a moving articulated object, the process of recovering the shape and motion requires two steps: (1) correctly segmenting (points on the boundary of) the silhouettes to each articulated part of the object, (2) estimating the motion of each individual part using the segmented silhouette. In this paper, we propose an iterative algorithm to solve this simultaneous assignment and alignment problem. Once we have estimated the shape and motion of each part of the object, the articulation points between each pair of rigid parts are obtained by solving a simple motion constraint between the connected parts. To validate our algorithm, we first apply it to segment the different body parts and estimate the joint positions of a person. The acquired kinematic (shape and joint) information is then used to track the motion of the person in new video sequences.

A comparison of unscented and extended Kalman filtering for estimating quaternion motion

Abstract: The unscented Kalman filter is a superior alternative to the extended Kalman filter for a variety of estimation and control problems. However, its effectiveness for improving human motion tracking for virtual reality applications in the presence of noisy data has been unexplored. In this paper, we present an empirical study comparing the performance of unscented and extended Kalman filtering for improving human head and hand tracking. Specifically, we examine human head and hand orientation motion signals, represented with quaternions, which are critical for correct viewing perspectives in virtual reality. Our experimental results and analysis indicate that unscented Kalman filtering performs equivalently with extended Kalman filtering. However, the additional computational overhead of the unscented Kalman filter and quasi-linear nature of the quaternion dynamics lead to the conclusion that the extended Kalman filter is a better choice for estimating quaternion motion in virtual reality applications.

Using many cameras as one

Abstract: We illustrate how to consider a network of cameras as a single generalized camera in a framework proposed by Nayar (2001). We derive the discrete structure from motion equations for generalized cameras, and illustrate the corollaries to epipolar geometry. This formal mechanism allows one to use a network of cameras as if they were a single imaging device, even when they do not share a common center of projection. Furthermore, an analysis of structure from motion algorithms for this imaging model gives constraints on the optimal design of panoramic imaging systems constructed from multiple cameras.

Using gravity to estimate accelerometer orientation

Abstract: Several wearable computing or ubiquitous computing research projects have detected and distinguished user motion activities by attaching accelerometers in known positions and orientations on the user’s body. This paper observes that the orientation constraint can probably be relaxed. An estimate of the constant gravity vector can be obtained by averaging accelerometer samples. This gravity vector estimate in turn enables estimation of the vertical component and the magnitude of the horizontal component of the user’s motion, independently of how the three-axis accelerometer system is oriented.

Epitomic analysis of appearance and shape

Abstract: We present novel simple appearance and shape models that we call epitomes. The epitome of an image is its miniature, condensed version containing the essence of the textural and shape properties of the image. As opposed to previously used simple image models, such as templates or basis functions, the size of the epitome is considerably smaller than the size of the image or object it represents, but the epitome still contains most constitutive elements needed to reconstruct the image. A collection of images often shares an epitome, e.g., when images are a few consecutive frames from a video sequence, or when they are photographs of similar objects. A particular image in a collection is defined by its epitome and a smooth mapping from the epitome to the image pixels. When the epitomic representation is used within a hierarchical generative model, appropriate inference algorithms can be derived to extract the epitome from a single image or a collection of images and at the same time perform various inference tasks, such as image segmentation, motion estimation, object removal and super-resolution.

Dynamic camera calibration of roadside traffic management cameras for vehicle speed estimation

Abstract: In this paper, we present a new three-stage algorithm to calibrate roadside traffic management cameras and track vehicles to create a traffic speed sensor. The algorithm first estimates the camera position relative to the roadway using the motion and edges of the vehicles. Given the camera position, the algorithm then calibrates the camera by estimating the lane boundaries and the vanishing point of the lines along the roadway. The algorithm transforms the image coordinates from the vehicle tracker into real-world coordinates using our simplified camera model. We present results that demonstrate the ability of our algorithm to produce good estimates of the mean vehicle speed in a lane of traffic.

Lifting-based invertible motion adaptive transform (LIMAT) framework for highly scalable video compression

Abstract: We propose a new framework for highly scalable video compression, using a lifting-based invertible motion adaptive transform (LIMAT). We use motion-compensated lifting steps to implement the temporal wavelet transform, which preserves invertibility, regardless of the motion model. By contrast, the invertibility requirement has restricted previous approaches to either block-based or global motion compensation. We show that the proposed framework effectively applies the temporal wavelet transform along a set of motion trajectories. An implementation demonstrates high coding gain from a finely embedded, scalable compressed bit-stream. Results also demonstrate the effectiveness of temporal wavelet kernels other than the simple Haar, and the benefits of complex motion modeling, using a deformable triangular mesh. These advances are either incompatible or difficult to achieve with previously proposed strategies for scalable video compression. Video sequences reconstructed at reduced frame-rates, from subsets of the compressed bit-stream, demonstrate the visually pleasing properties expected from low-pass filtering along the motion trajectories. The paper also describes a compact representation for the motion parameters, having motion overhead comparable to that of motion-compensated predictive coders. Our experimental results compare favorably to others reported in the literature, however, our principal objective is to motivate a new framework for highly scalable video compression.

Elastic registration in the presence of intensity variations

Abstract: We have developed a general-purpose registration algorithm for medical images and volumes. This method models the transformation between images as locally affine but globally smooth. The model also explicitly accounts for local and global variations in image intensities. This approach is built upon a differential multiscale framework, allowing us to capture both large- and small-scale transformations. We show that this approach is highly effective across a broad range of synthetic and clinical medical images.

A novel video key-frame-extraction algorithm based on perceived motion energy model

Abstract: The key frame is a simple yet effective form of summarizing a long video sequence. The number of key frames used to abstract a shot should be compliant to visual content complexity within the shot and the placement of key frames should represent most salient visual content. Motion is the more salient feature in presenting actions or events in video and, thus, should be the feature to determine key frames. We propose a triangle model of perceived motion energy (PME) to model motion patterns in video and a scheme to extract key frames based on this model. The frames at the turning point of the motion acceleration and motion deceleration are selected as key frames. The key-frame selection process is threshold free and fast and the extracted key frames are representative.

Fast mode decision and motion estimation for JVT/H.264

Abstract: The JVT/H.264 video coding standard achieves considerably higher coding efficiency than previous standards. Unfortunately this comes at a cost in considerably increased complexity at the encoder mainly due to motion estimation and mode decision. In this paper, we propose a new scheme to jointly optimize motion estimation and mode decision. Simulation results show that we achieve up to 90% complexity reduction while maintaining coding efficiency.

Digital image stabilization with sub-image phase correlation based global motion estimation

Abstract: This paper presents digital image stabilization with sub-image phase correlation based global motion estimation and Kalman filtering based motion correction. Global motion is estimated from the local motions of four sub-images each of which is detected using phase correlation based motion estimation. The global motion vector is decided according to the peak values of sub-image phase correlation surfaces, instead of impartial median filtering. The peak values of sub-image phase correlation surfaces reveal reliable local motion vectors, as poorly matched sub images result in considerably lower peaks in the phase correlation surface due to spread. The utilization of sub-images enables fast implementation of phase correlation based motion estimation. The global motion vectors of image frames are accumulated to obtain global displacement vectors, that are Kalman filtered for stabilization.

Probabilistic video stabilization using Kalman filtering and mosaicing

Abstract: The removal of unwanted, parasitic vibrations in a video sequence induced by camera motion is an essential part of video acquisition in industrial, military and consumer applications. In this paper, we present a new image processing method to remove such vibrations and reconstruct a video sequence void of sudden camera movements. Our approach to separating unwanted vibrations from intentional camera motion is based on a probabilistic estimation framework. We treat estimated parameters of interframe camera motion as noisy observations of the intentional camera motion parameters. We construct a physics-based state-space model of these interframe motion parameters and use recursive Kalman filtering to perform stabilized camera position estimation. A six-parameter affine model is used to describe the interframe transformation, allowing quite accurate description of typical scene changes due to camera motion. The model parameters are estimated using a p-norm-based multi-resolution approach. This approach is robust to model mismatch and to object motion within the scene (which are treated as outliers). We use mosaicking in order to reconstruct undefined areas that result from motion compensation applied to each video frame. Registration between distant frames is performed efficiently by cascading interframe affine transformation parameters. We compare our method' s performance with that of a commercial product on real-life video sequences, and show a significant improvement in stabilization quality for our method.

Natural motion animation through constraining and deconstraining at will

Abstract: This paper presents a computational technique for creating whole-body motions of human and animal characters without reference motion. Our work enables animators to generate a natural motion by dragging a link to an arbitrary position with any number of links pinned in the global frame, as well as other constraints such as desired joint angles and joint motion ranges. The method leads to an intuitive pin-and-drag interface where the user can generate whole-body motions by simply switching on or off or strengthening or weakening the constraints. This work is based on a new interactive inverse kinematics technique that allows more flexible attachment of pins and various types of constraints. Editing or retargeting captured motion requires only a small modification to the original method, although it can also create natural motions from scratch. We demonstrate the usefulness and advantage of our method with a number of example motion clips.

Carotid artery wall motion estimated from B-mode ultrasound using region tracking and block matching.

Abstract: The motion of the carotid atheromatous plaque relative to the adjacent wall may be related to the risk of cerebral events. A quantitative method for motion estimation was applied to analyse arterial wall movement from sequences of 2-D B-mode ultrasound (US) images. Image speckle patterns were tracked between successive frames using the correlation coefficient as the matching criterion. The size of the selected region-of-interest (ROI) was shown to affect the motion analysis results; an optimal size of 3.2 x 2.5 mm(2) was suggested for tracking a region at the wall-lumen interface and of 6.3 x 2.5 mm(2) for one within the tissue. The results showed expected cyclical motion in the radial direction and some axial movement of the arterial wall. The method can be used to study further the axial motion of the carotid artery wall and plaque and, thus, provide useful insight into the mechanisms of atherosclerosis.

Formation control of nonholonomic mobile robots with omnidirectional visual servoing and motion segmentation

Abstract: We consider the problem of having a team of nonholonomic mobile robots follow a desired leader-follower formation using omnidirectional vision. By specifying the desired formation in the image plane, we translate the control problem into a separate visual servoing task for each follower. We use a rank constraint on the omnidirectional optical flows across multiple frames to estimate the position and velocities of the leaders in the image plane of each follower. We show that the direct feedback-linearization of the leader-follower dynamics suffers from degenerate configurations due to the nonholonomic constraints of the robots and the nonlinearity of the omnidirectional projection model. We therefore design a nonlinear tracking controller that avoids such degenerate configurations, while preserving the formation input-to-state stability. Our control law naturally incorporates collision avoidance by exploiting the geometry of omnidirectional cameras. We present simulations and experiments evaluating our omnidirectional vision-based formation control scheme.

Pose estimation using line-based dynamic vision and inertial sensors

Abstract: An observer problem from a computer vision application is studied. Rigid body pose estimation using inertial sensors and a monocular camera is considered and it is shown how rotation estimation can be decoupled from position estimation. Orientation estimation is formulated as an observer problem with implicit output where the states evolve on SO(3). A careful observability study reveals interesting group theoretic structures tied to the underlying system structure. A locally convergent observer where the states evolve on SO (3) is proposed and numerical estimates of the domain of attraction is given. Further, it is shown that, given convergent orientation estimates, position estimation can be formulated as a linear implicit output problem. From an applications perspective, it is outlined how delayed low bandwidth visual observations and high bandwidth rate gyro measurements can provide high bandwidth estimates. This is consistent with real-time constraints due to the complementary characteristics of the sensors which are fused in a multirate way.

Articulated soft objects for multiview shape and motion capture

Abstract: We develop a framework for 3D shape and motion recovery of articulated deformable objects. We propose a formalism that incorporates the use of implicit surfaces into earlier robotics approaches that were designed to handle articulated structures. We demonstrate its effectiveness for human body modeling from synchronized video sequences. Our method is both robust and generic. It could easily be applied to other shape and motion recovery problems.

Hidden Markov measure field models for image segmentation

Abstract: Parametric image segmentation consists of finding a label field that defines a partition of an image into a set of nonoverlapping regions and the parameters of the models that describe the variation of some property within each region. A new Bayesian formulation for the solution of this problem is presented, based on the key idea of using a doubly stochastic prior model for the label field, which allows one to find exact optimal estimators for both this field and the model parameters by the minimization of a differentiable function. An efficient minimization algorithm and comparisons with existing methods on synthetic images are presented, as well as examples of realistic applications to the segmentation of Magnetic Resonance volumes and to motion segmentation.

Adaptive dynamic range imaging: optical control of pixel exposures over space and time

Abstract: This paper presents a new approach to imaging that significantly enhances the dynamic range of a camera. The key idea is to adapt the exposure of each pixel on the image detector, based on the radiance value of the corresponding scene point. This adaptation is done in the optical domain, that is, during image formation. In practice, this is achieved using a spatial light modulator whose transmittance can be varied with high resolution over space and time. A real-time control algorithm is developed that uses acquired images to automatically adjust the transmittance function of the spatial modulator. Each captured image and its corresponding transmittance function are used to compute a very high dynamic range image that is linear in scene radiance. We have implemented a video-rate adaptive dynamic range camera that consists of a color CCD detector and a controllable liquid crystal light modulator. Experiments have been conducted in scenarios with complex and harsh lighting conditions. The results indicate that adaptive imaging can have a significant impact on vision applications such as monitoring, tracking, recognition, and navigation.

A generalized divergence measure for robust image registration

Abstract: Entropy-based divergence measures have shown promising results in many areas of engineering and image processing. We define a new generalized divergence measure, namely, the Jensen-Renyi (1996, 1976) divergence. Some properties such as convexity and its upper bound are derived. Based on the Jensen-Renyi divergence, we propose a new approach to the problem of image registration. Some appealing advantages of registration by Jensen-Renyi divergence are illustrated, and its connections to mutual information-based registration techniques are analyzed. As the key focus of this paper, we apply Jensen-Renyi divergence for inverse synthetic aperture radar (ISAR) image registration. The goal is to estimate the target motion during the imaging time. Our approach applies Jensen-Renyi divergence to measure the statistical dependence between consecutive ISAR image frames, which would be maximal if the images are geometrically aligned. Simulation results demonstrate that the proposed method is efficient and effective.

Motion deblurring using hybrid imaging

Abstract: Motion blur due to camera motion can significantly degrade the quality of an image. Since the path of the camera motion can be arbitrary, deblurring of motion blurred images is a hard problem. Previous methods to deal with this problem have included blind restoration of motion blurred images, optical correction using stabilized lenses, and special CMOS sensors that limit the exposure time in the presence of motion. In this paper, we exploit the fundamental tradeoff between spatial resolution and temporal resolution to construct a hybrid camera that can measure its own motion during image integration. The acquired motion information is used to compute a point spread function (PSF) that represents the path of the camera during integration. This PSF is then used to deblur the image. To verify the feasibility of hybrid imaging for motion deblurring, we have implemented a prototype hybrid camera. This prototype system was evaluated in different indoor and outdoor scenes using long exposures and complex camera motion paths. The results show that, with minimal resources, hybrid imaging outperforms previous approaches to the motion blur problem.

Regularized adaptive high-resolution image reconstruction considering inaccurate subpixel registration

Abstract: We propose a high-resolution image reconstruction algorithm considering inaccurate subpixel registration. A regularized iterative reconstruction algorithm is adopted to overcome the ill-posedness problem resulting from inaccurate subpixel registration. In particular, we use multichannel image reconstruction algorithms suitable for applications with multiframe environments. Since the registration error in each low-resolution image has a different pattern, the regularization parameters are determined adaptively for each channel. We propose two methods for estimating the regularization parameter automatically. The proposed algorithms are robust against registration error noise, and they do not require any prior information about the original image or the registration error process. Information needed to determine the regularization parameter and to reconstruct the image is updated at each iteration step based on the available partially reconstructed image. Experimental results indicate that the proposed algorithms outperform conventional approaches in terms of both objective measurements and visual evaluation.