GreenWarps: A Two-Stage Warping Model for Stitching Images Using Diffeomorphic Meshes and Green Coordinates
08 Sep 2018-pp 740-744
TL;DR: The proposed method “GreenWarps” aims to accurately align frames/images with large parallax by using a demon-based diffeomorphic warping method for mesh deformation termed “DiffeoMeshes” and the results show superior performance of the method compared to the state-of-the-art methods.
Abstract: Image Stitching is a hard task to solve in the presence of large parallax in the images. Specifically, for a sequence of frames from unconstrained videos which are considerably shaky, recent works fail to align such a sequence of images accurately. The proposed method “GreenWarps” aims to accurately align frames/images with large parallax. The method consists of two novel stages, namely, Prewarping and Diffeomorphic Mesh warping. The first stage warps unaligned image to the reference image using Green Coordinates. The second stage of the model refines the alignment by using a demon-based diffeomorphic warping method for mesh deformation termed “DiffeoMeshes”. The warping is performed using Green Coordinates in both the stages without the assumption of any motion model. The combination of the two stages provide accurate alignment of the images. Experiments were performed on two standard image stitching datasets and one dataset consisting of images created from unconstrained videos. The results show superior performance of our method compared to the state-of-the-art methods.
Citations
More filters
TL;DR: A novel method for large-scale image stitching that is robust against repetitive patterns and featureless regions in the imaginary, and formalizes the method as a weighted multigraph whose nodes represent the individual image transformations from the composite image, and whose sets of multiple edges between two nodes represent all the plausible transformations between the pixel coordinates of the two images.
Abstract: We propose a novel method for large-scale image stitching that is robust against repetitive patterns and featureless regions in the imaginary. In such cases, state-of-the-art image stitching methods easily produce image alignment artifacts, since they may produce false pairwise image registrations that are in conflict within the global connectivity graph. Our method augments the current methods by collecting all the plausible pairwise image registration candidates, among which globally consistent candidates are chosen. This enables the stitching process to determine the correct pairwise registrations by utilizing all the available information from the whole imaginary, such as unambiguous registrations outside the repeating pattern and featureless regions. We formalize the method as a weighted multigraph whose nodes represent the individual image transformations from the composite image, and whose sets of multiple edges between two nodes represent all the plausible transformations between the pixel coordinates of the two images. The edge weights represent the plausibility of the transformations. The image transformations and the edge weights are solved from a non-linear minimization problem with linear constraints, for which a projection method is used. As an example, we apply the method in a large-scale scanning application where the transformations are primarily translations with only slight rotation and scaling component. Despite these simplifications, the state-of-the-art methods do not produce adequate results in such applications, since the image overlap is small, can be featureless or repetitive, and misalignment artifacts and their concealment is unacceptable.
4 citations
Cites methods from "GreenWarps: A Two-Stage Warping Mod..."
...In [9], conformal mappings with small non-conformal deviations are used....
[...]
TL;DR: Experimental results demonstrate that the feature extraction speed with the proposed algorithm is improved by at least 78% compared to SIFT, and the motion ghosting is effectively eliminated, thus achieving a fast and high-quality mosaic.
3 citations
TL;DR: In this article, the authors proposed a method for large-scale image stitching that is robust against repetitive patterns and featureless regions in the imagery, which enables the stitching process to determine the correct pairwise registrations by utilizing all the available information from the whole imagery.
Abstract: We propose a novel method for large-scale image stitching that is robust against repetitive patterns and featureless regions in the imagery. In such cases, state-of-the-art image stitching methods easily produce image alignment artifacts, since they may produce false pairwise image registrations that are in conflict within the global connectivity graph. Our method augments the current methods by collecting all the plausible pairwise image registration candidates, among which globally consistent candidates are chosen. This enables the stitching process to determine the correct pairwise registrations by utilizing all the available information from the whole imagery, such as unambiguous registrations outside the repeating pattern and featureless regions. We formalize the method as a weighted multigraph whose nodes represent the individual image transformations from the composite image, and whose sets of multiple edges between two nodes represent all the plausible transformations between the pixel coordinates of the two images. The edge weights represent the plausibility of the transformations. The image transformations and the edge weights are solved from a non-linear minimization problem with linear constraints, for which a projection method is used. As an example, we apply the method in a large-scale scanning application where the transformations are primarily translations with only slight rotation and scaling component. Despite these simplifications, the state-of-the-art methods do not produce adequate results in such applications, since the image overlap is small, which can be featureless or repetitive, and misalignment artifacts and their concealment are unacceptable.
3 citations
TL;DR: A systematic literature review of image stitching techniques applied on the plane and 3-D models for both feature-based and deep learning methods is provided in this paper , where the authors divide the stitching methods into two categories, namely, mosaic stitching methods for generating stitched plane images and panoramic stitching methods, and light field camera plane stitching methods.
Abstract: Image stitching is a technique in which multiple overlapping images of the scene are stitched together to generate an image with a wide view and high resolution. Image stitching methods can be broadly classified into feature-based and deep learning methods. Feature-based methods use manually designed features to establish transformation relationships between multiple images. This technology has played an important role in medical, industrial, military, and other fields. With the rise of deep learning in computer vision, it has also become the mainstream method in the field of image stitching. This article provides a systematic literature review of image stitching techniques applied on the plane and 3-D models for both feature-based and deep learning methods. We divide the stitching methods into two categories, namely, mosaic stitching methods for generating stitched plane images and panoramic stitching methods for generating stitched panoramic images. Based on the camera type, it is further divided into pinhole camera plane stitching methods, pinhole camera panoramic stitching methods, fisheye camera panoramic stitching methods, and light field camera plane stitching methods. An extensive search was conducted in International Conference on Image Processing (ICIP), IEEE Transactions on Image Processing (TIP), International Conference on Computer Vision (ICCV), European Conference on Computer Vision (ECCV), IEEE Conference on Computer Vision and Pattern Recognition (CVPR), British Machine Vision Conference (BMVC), International Conference on Pattern Recognition (ICPR), International Journal of Computer Vision (IJCV), IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM), IEEE Transactions on Intelligent Transportation Systems (ITS), IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), and ACM SIGGRAPH Computer Graphics (SIGGRAFH) to summarize related image stitching techniques; 89 articles are selected for systematic literature review and are presented in the table. The objective of this systematic literature review is to provide detailed technical progress in image stitching techniques and identify the research gap in this field.
01 Jan 2020
TL;DR: An improved GreenWarps is proposed, which is based on the combination of prewarping using Green coordinates and robust elastic warping, which can effectively stitch images with large parallax and projection distortion,which is difficult to realize using the state-of-the-art methods.
Abstract: In this study, we have proposed an improved GreenWarps, which is based on the combination of prewarping using Green coordinates and robust elastic warping. Firstly, an unaligned image is warped to the reference image by Green coordinates, and then the prewarped image is warped and locally registered using robust elastic warping. Finally, seam estimation is implemented to seamlessly blend the registered warped and reference images together. The experimental results confirm that our method can effectively stitch images with large parallax and projection distortion, which is difficult to realize using the state-of-the-art methods.
Cites methods from "GreenWarps: A Two-Stage Warping Mod..."
...In a recent study, a novel approach called GreenWarps[23] was proposed, which utilizes Green coordinates to warp the unaligned and the prewarped images without computing any transformations and assuming any motion model....
[...]
References
More filters
3,377 citations
TL;DR: This work forms stitching as a multi-image matching problem, and uses invariant local features to find matches between all of the images, and is insensitive to the ordering, orientation, scale and illumination of the input images.
Abstract: This paper concerns the problem of fully automated panoramic image stitching. Though the 1D problem (single axis of rotation) is well studied, 2D or multi-row stitching is more difficult. Previous approaches have used human input or restrictions on the image sequence in order to establish matching images. In this work, we formulate stitching as a multi-image matching problem, and use invariant local features to find matches between all of the images. Because of this our method is insensitive to the ordering, orientation, scale and illumination of the input images. It is also insensitive to noise images that are not part of a panorama, and can recognise multiple panoramas in an unordered image dataset. In addition to providing more detail, this paper extends our previous work in the area (Brown and Lowe, 2003) by introducing gain compensation and automatic straightening steps.
2,550 citations
TL;DR: The main idea is to consider the objects boundaries in one image as semi-permeable membranes and to let the other image, considered as a deformable grid model, diffuse through these interfaces, by the action of effectors situated within the membranes.
2,277 citations
TL;DR: An efficient non-parametric diffeomorphic image registration algorithm based on Thirion's demons algorithm that provides results that are similar to the ones from the demons algorithm but with transformations that are much smoother and closer to the gold standard, available in controlled experiments, in terms of Jacobians.
1,432 citations
TL;DR: A multiresolution spline technique for combining two or more images into a larger image mosaic is defined and coarse features occur near borders are blended gradually over a relatively large distance without blurring or otherwise degrading finer image details in the neighborhood of th e border.
Abstract: We define a multiresolution spline technique for combining two or more images into a larger image mosaic. In this procedure, the images to be splined are first decomposed into a set of band-pass filtered component images. Next, the component images in each spatial frequency hand are assembled into a corresponding bandpass mosaic. In this step, component images are joined using a weighted average within a transition zone which is proportional in size to the wave lengths represented in the band. Finally, these band-pass mosaic images are summed to obtain the desired image mosaic. In this way, the spline is matched to the scale of features within the images themselves. When coarse features occur near borders, these are blended gradually over a relatively large distance without blurring or otherwise degrading finer image details in the neighborhood of th e border.
1,246 citations