Showing papers on "Homography (computer vision) published in 2012"

Automatic Homographic Registration of a Pair of Images, with A Contrario Elimination of Outliers

Abstract: The RANSAC [2] algorithm (RANdom SAmple Consensus) is a robust method to estimate parameters of a model tting the data, in presence of outliers among the data. Its random nature is due only to complexity considerations. It iteratively extracts a random sample out of all data, of minimal size sucient to estimate the parameters. At each such trial, the number

Single Camera Structure and Motion

Abstract: A reduced order nonlinear observer is proposed for the problem of “structure and motion (SaM)” estimation of a stationary object observed by a moving calibrated camera. In comparison to existing work which requires some knowledge of the Euclidean geometry of an observed object or full knowledge of the camera motion, the developed reduced order observer only requires one camera linear velocity and corresponding acceleration to asymptotically identify the Euclidean coordinates of the feature points attached to an object (with proper scale reconstruction) and the remaining camera velocities. The unknown linear velocities are assumed to be generated using a model with unknown parameters. The unknown angular velocities are determined from a robust estimator which uses a standard Homography decomposition algorithm applied to tracked feature points. A Lyapunov analysis is provided to prove the observer asymptotically estimates the unknown states under a persistency of excitation condition.

PixMix: A real-time approach to high-quality Diminished Reality

Abstract: Diminished Reality (DR) allows to remove objects from a video stream while preseving a frame to frame coherence. Some approaches apply a pseudo-DR, allowing for the removal of objects only, while their background can be observed by a second camera. Most real DR approaches are highly computational expensive, not even allowing for interactive rates and/or apply significant restrictions regarding the uniformity of the background, or allow linear camera movements or even a static camera only. In this paper we will present a real-time capable Diminished Reality approach for high-quality image manipulation. Our approach achieves a significantly better performance and image quality for almost planar but non-trivial image backgrounds. Our Diminished Reality pipeline provides coherent video streams even for nonlinear camera movements due to the integration of homography based object tracking.

Projector Calibration for 3D Scanning Using Virtual Target Images

Abstract: In this work we propose a novel method to calibrate the projector. Calibration of the projector deals with the calculation of geometric parameters of the projector which are intrinsic and extrinsic parameters. An un-calibrated camera assists the process of finding the geometric parameters for projector. This method exploits the fact that a projector can be treated as inverse camera. The camera captures the scene and saves it as an image while the projector projects an image on the scene. Camera to projector transformation is used to make the projector able to see the calibration pattern. Then the projector is calibrated in a similar fashion like camera. Real data is used to evaluate the proposed method of projector calibration and good results are obtained.

Automatic upright adjustment of photographs

Abstract: Man-made structures often appear to be distorted in photos captured by casual photographers, as the scene layout often conflicts with how it is expected by human perception. In this paper we propose an automatic approach for straightening up slanted man-made structures in an input image to improve its perceptual quality. We call this type of correction upright adjustment. We propose a set of criteria for upright adjustment based on human perception studies, and develop an optimization framework which yields an optimal homography for adjustment. We also develop a new optimization-based camera calibration method that performs favorably to previous methods and allows the proposed system to work reliably for a wide variety of images. The effectiveness of our system is demonstrated by both quantitative comparisons and qualitative user studies.

Robust optical-flow based self-motion estimation for a quadrotor UAV

Abstract: Robotic vision has become an important field of research for micro aerial vehicles in the recent years. While many approaches for autonomous visual control of such vehicles rely on powerful ground stations, the increasing availability of small and light hardware allows for the design of more independent systems. In this context, we present a robust algorithm able to recover the UAV ego-motion using a monocular camera and on-board hardware. Our method exploits the continuous homography constraint so as to discriminate among the observed feature points in order to classify those belonging to the dominant plane in the scene. Extensive experiments on a real quadrotor UAV demonstrate that the estimation of the scaled linear velocity in a cluttered environment improved by a factor of 25% compared to previous approaches.

Nonlinear complementary filters on the special linear group

Abstract: This article proposes a nonlinear complementary filter for the special linear Lie-group SL(3) that fuses low-frequency state measurements with partial velocity measurements and adaptive estimation of unmeasured slowly changing velocity components. The obtained results have direct application on the problem of filtering a sequence of image homographies acquired from low-quality video data. The considered application motivates us to derive results that provide adaptive estimation of the full group velocity or part of the group velocity that cannot be measured from sensors attached to the camera. We demonstrate the performance of the proposed filters on real world homography data.

Robust multi-object tracking via cross-domain contextual information for sports video analysis

Abstract: Multiple player tracking is one of the main building blocks needed in a sports video analysis system. In an uncalibrated camera setting, robust mutli-object tracking can be very difficult due to a number of reasons including the presence of noise, occlusion, fast camera motion, low-resolution image capture, varying viewpoints and illumination changes. To address the problem of multi-object tracking in sports videos, we go beyond the video frame domain and make use of information in a homography transform domain that is denoted the homography field domain. We propose a novel particle filter based tracking algorithm that uses both object appearance information (e.g. color and shape) in the image domain and cross-domain contextual information in the field domain to improve object tracking. In the field domain, the effect of fast camera motion is significantly alleviated since the underlying homography transform from each frame to the field domain can be accurately estimated. We use contextual trajectory information (intra-trajectory and inter-trajectory context) to further improve the prediction of object states within an particle filter framework. Here, intra-trajectory contextual information is based on history tracking results in the field domain, while inter-trajectory contextual information is extracted from a compiled trajectory dataset based on tracks computed from videos depicting the same sport. Experimental results on real world sports data show that our system is able to effectively and robustly track a variable number of targets regardless of background clutter, camera motion and frequent mutual occlusion between targets.

Point-less calibration: Camera parameters from gradient-based alignment to edge images

Abstract: Point-based targets, such as checkerboards, are often not practical for outdoor camera calibration, as cameras are usually at significant heights requiring extremely large calibration patterns on the ground. Fortunately, it is possible to make use of existing non-point landmarks in the scene by formulating camera calibration in terms of image alignment. In this paper, we simultaneously estimate the camera intrinsic, extrinsic and lens distortion parameters directly by aligning to a planar schematic of the scene. For cameras with square pixels and known principal point, finding the parameters to such an image warp is equivalent to calibrating the camera. Overhead schematics of many environments resemble edge images. Edge images are difficult to align using image-based algorithms because both the image and its gradient are sparse. We employ a ‘long range’ gradient which enables informative parameter updates at each iteration while maintaining a precise alignment measure. As a result, we are able to calibrate our camera models robustly using regular gradient-based image alignment, given an initial ground to image homography estimate.

Homography-based Vision-aided Inertial Navigation of UAVs in Unknown Environments

Abstract: This paper investigates the navigation of small-scale unmanned aerial vehicles (UAVs) in unknown and GPS-denied environments. We consider a UAV equipped with a low-cost inertial measurement unit (IMU) and a monocular camera. The IMU can measure the specic acceleration and angular rate of the UAV. The IMU measurements are assumed to be corrupted by white noises and unknown constant biases. Hence the position, velocity and attitude of the UAV estimated by pure IMU dead reckoning will all drift over time. The monocular camera takes image sequences of the ground scene during ight. By assuming the ground scene is a level plane, the vision measurement, homography matrices, can be obtained from pairs of consecutive images. We propose a novel approach to fuse IMU and vision measurements by using an extended Kalman lter (EKF). Unlike conventional approaches, homography matrices are not required to be decomposed. Instead, they are converted to vectors and fed into the EKF directly. In the end, we analyze the observability of the proposed navigation system. We show that the velocity and attitude of the UAV and the unknown biases in IMU measurements are all observable when noisy yaw angle can be measured using a magnetometer. Numerical simulations verify our observability analysis and show that all UAV states except the position can be estimated without drift. The position drift is signicantly reduced compared to the IMU dead reckoning.

Optimizing PTZ camera calibration from two images

Abstract: In this paper, we address the problem of calibrating an active pan–tilt–zoom (PTZ) camera. In this regard, we make three main contributions: first, for the general camera rotation, we provide a novel solution that yields four independent constraints from only two images, by directly decomposing the infinite homography using a series of Givens rotations. Second, for a camera varying its focal length, we present a solution for the degenerate cases of pure pan and pure tilt that occur very frequently in practical applications of PTZ cameras. Third, we derive a new optimized error function for pure rotation or pan–tilt rotation, which plays a similar role as the epipolar constraint in a freely moving camera, in terms of characterizing the reprojection error of point correspondences. Our solutions and analysis are thoroughly validated and tested on both synthetic and real data, whereby the new geometric error function is shown to outperform existing methods in terms of accuracy and noise resilience.

Methods and Apparatus for Robust Video Stabilization

Abstract: Methods and apparatus for robust video stabilization. A video stabilization technique applies a feature tracking technique to an input video sequence to generate feature trajectories. The technique applies a video partitioning technique to segment the input video sequence into factorization windows and transition windows. The technique smoothes the trajectories in each of the windows, in sequence. For factorization windows, a subspace-based optimization technique may be used. For transition windows, a direct track optimization technique that uses a similarity motion model may be used. The technique then determines and applies warping models to the frames in the video sequence. In at least some embodiments, the warping models may include a content-preserving warping model, a homography model, a similarity transform model, and a whole-frame translation model. The warped frames may then be cropped according to a cropping technique.

Mapping Crop Status from AN Unmanned Aerial Vehicle for Precision Agriculture Applications

Abstract: . Remote sensing system mounted on unmanned aerial vehicle (UAV) could provide a complementary means to the conventional satellite and aerial remote sensing solutions especially for the applications of precision agriculture. UAV remote sensing offers a great flexibility to quickly acquire field data in sufficient spatial and spectral resolution at low cost. However a major problem of UAV is the high instability due to the low-end equipments and difficult environment situation, and this leads to image sensor being mostly operated under a highly uncertain configuration. Thus UAV images exhibit considerable derivation in spatial orientation, large geometric and spectral distortion, and low signal-to-noise ratio (SNR). To achieve the objectives of agricultural mapping from UAV, we apply a micro-helicopter UAV with a multiple spectral camera mounted and develop a framework to process UAV images. A very important processing is to generate mosaic image which can be aligned with maps for later GIS integration. With appropriate geometric calibration applied, we first decompose a homography of consecutive image pairs into a rotational component and a simple perspective component, and apply a linear interpolation to the angle of the rotational component, followed by a linear matrix interpolation operator to the perspective component, and this results in an equivalent transformation but ensures a smooth evolution between two images. Lastly to demonstrate the potential of UAV images to precision agriculture application, we perform spectral processing to derive vegetation indices (VIs) maps of crop, and also show the comparison with satellite imagery. Through this paper, we demonstrate that it is highly feasible to generate quantitative mapping products such as crop stress maps from UAV images, and suggest that UAV remote sensing is very valuable for the applications of precision agriculture.

Light field compression using homography and 2D warping

Abstract: This paper describes a new method for light field compression that exploits inter-view correlation. The proposed method uses homography and 2D warping to predict views and does not require additional camera parameters or a 3D geometry model. The method utilizes angular shift between views, which is neglected in conventional motion compensation methods. Results indicate improved coding efficiency of the proposed method over traditional motion compensation schemes. A full light field coder based on video-compression demonstrates 1–1.5 dB additional improvement in PSNR, or equivalently a 4–12% additional reduction in bitrate when the new method is introduced as a prediction mode.

Making any planar surface into a touch-sensitive display by a mere projector and camera

Abstract: We address how an HCI (Human-Computer Interface) with small device size, large display, and touch input facility can be made possible by a mere projector and camera. The realization is through the use of a properly embedded structured light sensing scheme that enables a regular light-colored table surface to serve the dual roles of both a projection screen and a touch-sensitive display surface. A random binary pattern is employed to code structured light in pixel accuracy, which is embedded into the regular projection display in a way that the user perceives only regular display but not the structured pattern hidden in the display. With the projection display on the table surface being imaged by a camera, the observed image data, plus the known projection content, can work together to probe the 3D world immediately above the table surface, like deciding if there is a finger present and if the finger touches the table surface, and if so at what position of the table surface the finger tip makes the contact. All the decisions hinge upon a careful calibration of the projector-camera-table surface system, intelligent segmentation of the hand in the image data, and exploitation of the homography mapping existing between the projector's display panel and the camera's image plane. Extensive experimentation including evaluation of the display quality, touch detection accuracy, and system efficiency are shown to illustrate the feasibility of the proposed realization.

Devices, methods, and apparatuses for homography evaluation involving a mobile device

Abstract: Components, methods, and apparatuses are provided that may be used to access information pertaining to a two-dimensional image of a three-dimensional object, to detect homography between said image of said three-dimensional object captured in said two-dimensional image indicative of said three-dimensional object and a reference object image and to determine whether said homography indicates pose suitable for image augmentation based, at least in part, on characteristics of an elliptically-shaped area that encompasses at least some of a plurality of inliers distributed in said two-dimensional image.

Calibrating visual sensors using homography operators

Abstract: A plurality of homography operators define respective mappings between pairs of coordinate spaces, wherein the coordinate spaces include a coordinate space of a first visual sensor, a virtual coordinate space, and a coordinate space of a second visual sensor. Calibration between the first and second visual sensors is provided using the plurality of homography operators.

Visual Control for Multirobot Organized Rendezvous

Abstract: This paper addresses the problem of visual control of a set of mobile robots. In our framework, the perception system consists of an uncalibrated flying camera performing an unknown general motion. The robots are assumed to undergo planar motion considering nonholonomic constraints. The goal of the control task is to drive the multirobot system to a desired rendezvous configuration relying solely on visual information given by the flying camera. The desired multirobot configuration is defined with an image of the set of robots in that configuration without any additional information. We propose a homography-based framework relying on the homography induced by the multirobot system that gives a desired homography to be used to define the reference target, and a new image-based control law that drives the robots to the desired configuration by imposing a rigidity constraint. This paper extends our previous work, and the main contributions are that the motion constraints on the flying camera are removed, the control law is improved by reducing the number of required steps, the stability of the new control law is proved, and real experiments are provided to validate the proposal.

Repetition Maximization based Texture Rectification

Abstract: Many photographs are taken in perspective. Techniques for rectifying resulting perspective distortions typically rely on the existence of parallel lines in the scene. In scenarios where such parallel lines are hard to automatically extract or manually annotate, the unwarping process remains a challenge. In this paper, we introduce an automatic algorithm to rectifying images containing textures of repeated elements lying on an unknown plane. We unwrap the input by maximizing for image self-similarity over the space of homography transformations. We map a set of detected regional descriptors to surfaces in a transformation space, compute the intersection points among triplets of such surfaces, and then use consensus among the projected intersection points to extract the correcting transform. Our algorithm is global, robust, and does not require explicit or accurate detection of similar elements. We evaluate our method on a variety of challenging textures and images. The rectified outputs are directly useful for various tasks including texture synthesis, image completion, etc. © 2012 Wiley Periodicals, Inc.

Accurate distance estimation using camera orientation compensation technique for vehicle driver assistance system

Abstract: This paper proposes a camera orientation compensation technique to help compensating unwanted camera orientation so as to accurately estimate the distance of a preceding vehicle in a vehicle driver assistance system. We propose a method in computing a homography to compensate the unwanted camera orientation misalignment. Distance between the preceding vehicle and the digital camera can then be accurately estimated. We will demonstrate that this approach be most suitable for vehicle driver assistance systems, and it also further expands the extent of automobile safety.

Moving camera video stabilization using homography consistency

Abstract: Videos recorded on moving cameras are often known to be shaky due to unstable carrier motion and the video stabilization problem involves inferring the intended smooth motion to keep and the unintended shaky motion to remove. However, conventional methods typically require proper, scenario-specific parameter setting, which does not generalize well across different scenarios. Moreover, we observe that a stable video should satisfy two conditions: a smooth trajectory and consistent inter-frame transition. While conventional methods only target at the former condition, we address these two issues at the same time. In this paper, we propose a homography consistency based algorithm to directly extract the optimal smooth trajectory and evenly distribute the inter-frame transition. By optimizing in the homography domain, our method does not need further matrix decomposition and parameter adjustment, automatically adapting to all possible types of motion (eg. translational or rotational) and video properties (eg. frame rates). We test our algorithm on translational videos recorded from a car and rotational videos from a hovering aerial vehicle, both of high and low frame rates. Results show our method widely applicable to different scenarios without any need of additional parameter adjustment.

Image matching techniques for vision-based indoor navigation systems: performance analysis for 3D map based approach

Abstract: In this paper, we give an overview of image matching techniques for various vision-based navigation systems: stereo vision, structure from motion and map-based approach. Focused on map-based approach, which generally uses feature-based matching for localization, and based on our early developed system, a performance analysis has been carried out and three major problems have been identified: being vulnerable to illumination changes, drastic viewpoint changes and good percentage of mismatches. By introducing ASIFT into the system, the major improvement takes place on the epoch with large viewpoint changes. In order to deal with mismatches that are unable to be removed by RANSAC, we propose to use cross-correlation information to evaluate the quality of homography model and help select the proper one. The conducted experiments have proved that such an approach can reduce the chances of mismatches being included by RANSAC and final positioning accuracy can be improved.

Registration accuracy and quality of real-life images.

Abstract: Background A common registration problem for the application of consumer device is to align all the acquired image sequences into a complete scene. Image alignment requires a registration algorithm that will compensate as much as possible for geometric variability among images. However, images captured views from a real scene usually produce different distortions. Some are derived from the optic characteristics of image sensors, and others are caused by the specific scenes and objects. Methodology/Principal Findings An image registration algorithm considering the perspective projection is proposed for the application of consumer devices in this study. It exploits a multiresolution wavelet-based method to extract significant features. An analytic differential approach is then proposed to achieve fast convergence of point matching. Finally, the registration accuracy is further refined to obtain subpixel precision by a feature-based modified Levenberg-Marquardt method. Due to its feature-based and nonlinear characteristic, it converges considerably faster than most other methods. In addition, vignette compensation and color difference adjustment are also performed to further improve the quality of registration results. Conclusions/Significance The performance of the proposed method is evaluated by testing the synthetic and real images acquired by a hand-held digital still camera and in comparison with two registration techniques in terms of the squared sum of intensity differences (SSD) and correlation coefficient (CC). The results indicate that the proposed method is promising in registration accuracy and quality, which are statistically significantly better than other two approaches.

Homography estimation from correspondences of local elliptical features

Abstract: We propose a novel unified approach for homography estimation from two or more correspondences of local elliptical features. The method finds a homography defined by first-order Taylor expansions at two (or more) points. The approximations are affine transformations that are constrained by the ellipse-to-ellipse correspondences. Unlike methods based on projective invariants of conics, the proposed method generates only a single homography model per pair of ellipse correspondences. We show experimentally, that the proposed method generates models of precision comparable or better than the state-of-the-art at lower computational costs.

Three-dimensional reconstruction based on multiple virtual planes by using fusion-based camera network

Abstract: A novel approach for three-dimensional (3D) volumetric reconstruction of an object inside a scene is proposed. A camera network is used to observe the scene. Each camera within the network is rigidly coupled with an Inertial Sensor (IS). A virtual camera is defined for each IS-camera couple using the concept of infinite homography, by fusion of inertial and visual information. Using the inertial data and without planar ground assumption, a set of virtual horizontal planes are defined. The intersections of these inertial-based virtual planes with the object are registered using the concept of planar homography. Moreover a method to estimate the translation vectors among virtual cameras is proposed, which just needs the relative heights of two 3D points in the scene with respect to one of the cameras and their correspondences on the image planes. Different experimental results for the proposed 3D reconstruction method are provided on two different types of scenarios. In the first type, a single IS-camera couple is used and placed in different locations around the object. In the second type, the 3D reconstruction of a walking person (dynamic case) is performed where a set of installed cameras in a smart-room is used for the data acquisition. Moreover, a set of experiments are simulated to analyse the accuracy of the translation estimation method. The experimental results show the feasibility and effectiveness of the proposed framework for the purpose of multi-layer data registration and volumetric reconstruction.

Visual Gyroscope and Odometer for Pedestrian Indoor Navigation with a Smartphone

Abstract: A pedestrian navigation system has to be accurate, reasonably priced, easy to use and light to carry to be adopted into use. Smartphones are attractive platforms for the navigation systems due to their small size, low cost and diversity of sensors feasible for positioning. Pedestrian navigation is mostly needed in GNSS degraded and denied areas such as indoors and in urban canyons. Self-contained sensors function independently regardless of the environment by augmenting the radio positioning systems, like GPS or wireless radio sensors, using information of the motion of the user. The measurements from self-contained sensors in a smartphone are, however, noisy and biased and thus need some aiding. This paper presents a method of using measurements from consecutive images as a gyroscope and an odometer. The rotation of the camera is calculated from vanishing points and the change in the heading angle used as a gyroscope measurement. Using the tilt obtained from the vanishing point calculations the distance to the objects in the navigation environment is observed and the translation between consecutive images measured with image homography from the image points of the objects. The translation measurements are then used as measurements from an odometer.

Panoramic Vision System to Eliminate Driver’s Blind Spots using a Laser Sensor and Cameras

Abstract: In this paper, we propose a multipurpose panoramic vision system for eliminating the blind spot and informing the driver of approaching vehicles using three cameras and a laser sensor. A wide-angle camera is attached to the car trunk and two cameras are attached under each side-view mirror to eliminate the blind spot of a vehicle. A laser sensor is attached on the rear-left of the vehicle to gather information from around the vehicle. The proposed system performs a pre-processing to estimate several system parameters. We compute a warping map to compensate the wide-angle lens distortion of the rear camera. We estimate the focus-of-contraction (FOC) in the rear image, and interactively compute the homography between the rear image and the laser sensor data. Homographies between each side-view image and the rear image are also computed in the pre-processing step. Using the system parameters obtained in the pre-processing step, the proposed system generates a panoramic mosaic view to eliminate the blind spot. First we obtain the undistorted rear image using the warping map. Then, we find road boundaries and classify approaching vehicles in laser sensor data, and overlap the laser sensor data on the rear image for further visualization. Next, the system performs the image registration process after segmentation of road and background regions based on road boundaries. Finally, it generates various views such as a cylindrical panorama view, a top view, a side view and an information panoramic mosaic view for displaying varied safety information.

A Coarse-to-Fine Matching Algorithm for FLIR and Optical Satellite Image Registration

Abstract: The registration of a forward-looking infrared (FLIR) image and an optical satellite image (visible image) is challenging but important for image-based navigation systems. To solve this problem effectively, a coarse-to-fine matching algorithm is proposed. First, geometric rectification based on the attitude angles and height parameter is carried out to eliminate the distinct rotation and scale discrepancies between the FLIR and visible images. Then, in the fine registration step, the edges of the visible image and rectified infrared image are extracted, and a robust point set registration algorithm which can deal with the similarity transformation distortion is proposed. Finally, the experiments on both the simulated images and real images show that our algorithm can achieve excellent performance in terms of both robustness and accuracy, and the registration precision of real images can be around one pixel.

Panorama image generating method

Abstract: PURPOSE: A panorama image generating method is provided to be strong against size conversion and rotation. CONSTITUTION: Feature points for size and rotation invariables of input images are extracted by an SURF(Speed-Up Robust Features) algorithm. Inliers and outliers are distinguished from the feature points for the size and rotation invariables of the input images by an RANSAC(Random Sample Consensus) algorithm. A related feature is matched by eliminating the outliers. A plurality of parameters for geometrical distortion is estimated after a perspective transformation formula is modeled based on homography. A panorama image is generated by aligning images matched with the related feature.

A Bayes filter based adaptive floor segmentation with homography and appearance cues

Abstract: This paper proposes a robust approach for image based floor detection and segmentation from sequence of images or video. In contrast to many previous approaches, which uses a priori knowledge of the surroundings, our method uses combination of modified sparse optical flow and planar homography for ground plane detection which is then combined with graph based segmentation for extraction of floor from images. We also propose a probabilistic framework which makes our method adaptive to the changes in the surroundings. We tested our algorithm on several common indoor environment scenarios and were able to extract floor even under challenging circumstances. We obtained extremely satisfactory results in various practical scenarios such as where the floor and non floor areas are of same color, in presence of textured flooring, and where illumination changes are steep.