scispace - formally typeset
Search or ask a question
Journal ArticleDOI

Image Alignment by Piecewise Planar Region Matching

08 Aug 2014-IEEE Transactions on Multimedia (IEEE)-Vol. 16, Iss: 7, pp 2052-2061
TL;DR: A novel method which approximates image regions with planes by incorporating piecewise local geometric models and outperforms state-of-the-art, especially in the case of large appearance variations and it is applicable to web-images which are taken from the same scene with different viewpoints.
Abstract: Robust image registration is a challenging problem, especially when dealing with severe changes in illumination and viewpoint. Previous methods assume a global geometric model (e.g., homography) and, hence, are only able to align images under predefined constraints (e.g., planar scenes and parallax-free camera motion). However, these constraints may not hold for natural scenes and uncontrolled imaging conditions. Therefore, this paper proposes a novel method which approximates image regions with planes by incorporating piecewise local geometric models. The approximated planar regions are obtained by exploiting a hierarchical figure-ground segmentation method. Each such planar region assumes an affine transformation. To achieve the alignment of the planar regions, an energy function is defined which employs intensity, a key-point descriptor, and geometric information under a global constraint. By re-segmenting and re-merging planar regions iteratively in an energy minimization framework, the method is able to align images even under significant changes in illumination and viewpoint. Experiments on two datasets show that the proposed method outperforms state-of-the-art, especially in the case of large appearance variations and it is, therefore, applicable to web-images (i.e., unconstrained setting) which are taken from the same scene with different viewpoints.
Citations
More filters
Journal ArticleDOI
TL;DR: A parallax-tolerant image stitching method based on robust elastic warping, which could achieve accurate alignment and efficient processing simultaneously and is highly compatible with different transformation types.
Abstract: Image stitching aims at generating high-quality panoramas with the lowest computational cost. In this paper, we propose a parallax-tolerant image stitching method based on robust elastic warping, which could achieve accurate alignment and efficient processing simultaneously. Given a group of point matches between images, an analytical warping function is constructed to eliminate the parallax errors. Then, the input images are warped according to the computed deformations over the meshed image plane. The seamless panorama is composed by directly reprojecting the warped images. As an important complement to the proposed method, a Bayesian model of feature refinement is proposed to adaptively remove the incorrect local matches. This ensures a more robust alignment than existing approaches. Moreover, our warp is highly compatible with different transformation types. A flexible strategy of combining it with the global similarity transformation is provided as an example. The performance of the proposed approach is demonstrated using several challenging cases.

134 citations


Cites methods from "Image Alignment by Piecewise Planar..."

  • ...By explicitly segmenting and matching the planar regions, Lou and Gevers proposed the piecewise planar region matching approach [10]...

    [...]

Journal ArticleDOI
TL;DR: In this paper, a quasi-homography warp is proposed to balance the perspective distortion against the projective distortion in the nonoverlapping region to create a more natural-looking panorama.
Abstract: The naturalness of warps is gaining extensive attention in image stitching. Recent warps, such as SPHP and AANAP, use global similarity warps to mitigate projective distortion (which enlarges regions); however, they necessarily bring in perspective distortion (which generates inconsistencies). In this paper, we propose a novel quasi-homography warp, which effectively balances the perspective distortion against the projective distortion in the non-overlapping region to create a more natural-looking panorama. Our approach formulates the warp as the solution of a bivariate system, where perspective distortion and projective distortion are characterized as slope preservation and scale linearization, respectively. Because our proposed warp only relies on a global homography, it is thus totally parameter free. A comprehensive experiment shows that a quasi-homography warp outperforms some state-of-the-art warps in urban scenes, including homography, AutoStitch and SPHP. A user study demonstrates that it wins most users’ favor, compared to homography and SPHP.

86 citations

Journal ArticleDOI
TL;DR: This paper casts the multi-view registration into a clustering problem, which can be solved by the extended K-means clustering algorithm, which has tested on some public data sets and compared with the-state-of-art algorithms.

44 citations


Cites background from "Image Alignment by Piecewise Planar..."

  • ...] can be combined to estimate the optimal transformation. 3D features can be also extracted from the point sets and they can be matched to provide initial transformation for the pair-wise registration[20, 21, 22]. 2 The development of scanning equipment makes the 3D reconstruction of an object possible. Due to the occlusion, however, the object cannot be entirely scanned from a single viewpoint. Therefore, sc...

    [...]

Journal ArticleDOI
Jin Zheng1, Yue Wang1, Hanzi Wang2, Bo Li1, Hai-Miao Hu1 
TL;DR: Compared with the state-of-the-art image stitching methods, the experimental results show that the projective transformation model estimated by the proposed PCPS method for each projective-consistent plane is more accurate, and the achieved stitching results have less seams and projective distortion.
Abstract: When different target surfaces, in three-dimensional space, are mapped onto an image plane, they have different projections. These projections vary with the viewpoint. These local differences have influence on the accuracy of image stitching. Most of the existing image stitching methods divide an input image into a number of fixed-size cells, and the pixels within the same cell are then warped using the same local transformation model for the alignment. These methods are based on the hypothesis that the transformation models in one cell are consistent. However, this hypothesis does not hold in general. In this paper, we propose a novel projective-consistent plane based image stitching method (termed PCPS). It divides the overlapping regions of an input image into some projective-consistent planes according to the normal vectors’ orientations of local regions and the reprojection errors of aligned images. The local projective transformation model is estimated for each projective-consistent plane. And then, a hybrid warping model is estimated. For the pixels in overlapping regions, the local projective transformation models are adopted to achieve a better alignment. While for the pixels in non-overlapping regions, a global projective transformation model is estimated by using the inliers uniformly distributed in the projective-consistent planes to avoid distortion. Compared with the state-of-the-art image stitching methods, the experimental results on a number of challenging image sequences show that the projective transformation model estimated by the proposed PCPS method for each projective-consistent plane is more accurate, and the achieved stitching results have less seams and projective distortion.

27 citations


Cites background from "Image Alignment by Piecewise Planar..."

  • ...as image stabilization [1], high-resolution photo mosaics [2], and so on....

    [...]

Journal ArticleDOI
22 Jan 2016
TL;DR: The proposed real-time 2.5-D change detection method combines the accuracy of a 2-D local image registration with the robustness of 3-D scene alignment, and was found that the resulting change detection system detects small changes of only 18×18×9 at distances of 60 meters under large trajectory deviations of up to2.5 meters.
Abstract: Change detection from mobile platforms is a relevant topic in the field of intelligent vehicles and has many applications, such as countering improvised explosive devices (C-IED). Existing real-time C-IED systems are not robust against large viewpoint differences, which are unavoidable under realistic operating conditions in outdoor environments. To address this, we proposes a new hierarchical 2.5-D scene-alignment algorithm. First, the 3-D ground surface of the historic scene is reconstructed by polygons, onto which historic image-based texture is projected. By estimating the 3-D transformation between historic and live camera views, the historic scene can be rendered as if seen from the live camera viewpoint. To compensate for 3-D alignment and reconstruction imperfections, local pixel-accurate registration refinement is performed in 2-D. The proposed real-time 2.5-D method thereby combines the accuracy of a 2-D local image registration with the robustness of 3-D scene alignment. It was found that the resulting change detection system detects small changes of only $18 \times 18 \times 9 \text{ cm}$ at distances of 60 meters under large trajectory deviations of up to 2.5 meters.

12 citations

References
More filters
Journal ArticleDOI
TL;DR: This paper presents a method for extracting distinctive invariant features from images that can be used to perform reliable matching between different views of an object or scene and can robustly identify objects among clutter and occlusion while achieving near real-time performance.
Abstract: This paper presents a method for extracting distinctive invariant features from images that can be used to perform reliable matching between different views of an object or scene. The features are invariant to image scale and rotation, and are shown to provide robust matching across a substantial range of affine distortion, change in 3D viewpoint, addition of noise, and change in illumination. The features are highly distinctive, in the sense that a single feature can be correctly matched with high probability against a large database of features from many images. This paper also describes an approach to using these features for object recognition. The recognition proceeds by matching individual features to a database of features from known objects using a fast nearest-neighbor algorithm, followed by a Hough transform to identify clusters belonging to a single object, and finally performing verification through least-squares solution for consistent pose parameters. This approach to recognition can robustly identify objects among clutter and occlusion while achieving near real-time performance.

46,906 citations

Proceedings ArticleDOI
20 Jun 2005
TL;DR: It is shown experimentally that grids of histograms of oriented gradient (HOG) descriptors significantly outperform existing feature sets for human detection, and the influence of each stage of the computation on performance is studied.
Abstract: We study the question of feature sets for robust visual object recognition; adopting linear SVM based human detection as a test case. After reviewing existing edge and gradient based descriptors, we show experimentally that grids of histograms of oriented gradient (HOG) descriptors significantly outperform existing feature sets for human detection. We study the influence of each stage of the computation on performance, concluding that fine-scale gradients, fine orientation binning, relatively coarse spatial binning, and high-quality local contrast normalization in overlapping descriptor blocks are all important for good results. The new approach gives near-perfect separation on the original MIT pedestrian database, so we introduce a more challenging dataset containing over 1800 annotated human images with a large range of pose variations and backgrounds.

31,952 citations


"Image Alignment by Piecewise Planar..." refers background in this paper

  • ...In order to achieve this, an affine model is fitted between these two groups of warped points and the parameters of plane Pi are updated: Ti ← Hg(i) ∗ Ti, (11) where Hg(i) is a 3×3 matrix by fitting an affine transformation model between C1(i) and C2(i)....

    [...]

Journal ArticleDOI
TL;DR: New results are derived on the minimum number of landmarks needed to obtain a solution, and algorithms are presented for computing these minimum-landmark solutions in closed form that provide the basis for an automatic system that can solve the Location Determination Problem under difficult viewing.
Abstract: A new paradigm, Random Sample Consensus (RANSAC), for fitting a model to experimental data is introduced. RANSAC is capable of interpreting/smoothing data containing a significant percentage of gross errors, and is thus ideally suited for applications in automated image analysis where interpretation is based on the data provided by error-prone feature detectors. A major portion of this paper describes the application of RANSAC to the Location Determination Problem (LDP): Given an image depicting a set of landmarks with known locations, determine that point in space from which the image was obtained. In response to a RANSAC requirement, new results are derived on the minimum number of landmarks needed to obtain a solution, and algorithms are presented for computing these minimum-landmark solutions in closed form. These results provide the basis for an automatic system that can solve the LDP under difficult viewing

23,396 citations

Book
01 Jan 2000
TL;DR: In this article, the authors provide comprehensive background material and explain how to apply the methods and implement the algorithms directly in a unified framework, including geometric principles and how to represent objects algebraically so they can be computed and applied.
Abstract: From the Publisher: A basic problem in computer vision is to understand the structure of a real world scene given several images of it. Recent major developments in the theory and practice of scene reconstruction are described in detail in a unified framework. The book covers the geometric principles and how to represent objects algebraically so they can be computed and applied. The authors provide comprehensive background material and explain how to apply the methods and implement the algorithms directly.

15,558 citations

01 Jan 2001
TL;DR: This book is referred to read because it is an inspiring book to give you more chance to get experiences and also thoughts and it will show the best book collections and completed collections.
Abstract: Downloading the book in this website lists can give you more advantages. It will show you the best book collections and completed collections. So many books can be found in this website. So, this is not only this multiple view geometry in computer vision. However, this book is referred to read because it is an inspiring book to give you more chance to get experiences and also thoughts. This is simple, read the soft file of the book and you get it.

14,282 citations


"Image Alignment by Piecewise Planar..." refers methods in this paper

  • ...We use a global constraint to ensure that regions are tightly connected reducing the influence of the outliers....

    [...]