Image Stitching based on Feature Extraction Techniques: A Survey
20 Aug 2014-International Journal of Computer Applications (Foundation of Computer Science (FCS))-Vol. 99, Iss: 6, pp 1-8
TL;DR: A framework of a complete image stitching system based on feature based approaches will be introduced and the current challenges of image stitching will be discussed.
Abstract: stitching (Mosaicing) is considered as an active research area in computer vision and computer graphics. Image stitching is concerned with combining two or more images of the same scene into one high resolution image which is called panoramic image. Image stitching techniques can be categorized into two general approaches: direct and feature based techniques. Direct techniques compare all the pixel intensities of the images with each other, whereas feature based techniques aim to determine a relationship between the images through distinct features extracted from the processed images. The last approach has the advantage of being more robust against scene movement, faster, and has the ability to automatically discover the overlapping relationships among an unordered set of images. The purpose of this paper is to present a survey about the feature based image stitching. The main components of image stitching will be described. A framework of a complete image stitching system based on feature based approaches will be introduced. Finally, the current challenges of image stitching will be discussed. Keywordsstitching/mosaicing, panoramic image, features based detection, SIFT, SURF, image blending.
Citations
More filters
TL;DR: The results in this paper explained that the HDR panorama images that resulting from the proposed method is more realistic image and appears as it is a real panorama environment.
Abstract: This paper presents a methodology for enhancement of panorama images environment by calculating high dynamic range. Panorama is constructing by merge of several photographs that are capturing by traditional cameras at different exposure times. Traditional cameras usually have much lower dynamic range compared to the high dynamic range in the real panorama environment, where the images are captured with traditional cameras will have regions that are too bright or too dark. A more details will be visible in bright regions with a lower exposure time and more details will be visible in dark regions with a higher exposure time. Since the details in both bright and dark regions cannot preserve in the images that are creating using traditional cameras, the proposed system have to calculate one using the images that traditional camera can actually produce. The proposed systems start by get LDR panorama image from multiple LDR images using SIFT features technology and then convert this LDR panorama image to the HDR panorama image using inverted local patterns. The results in this paper explained that the HDR panorama images that resulting from the proposed method is more realistic image and appears as it is a real panorama environment.
55Â citations
Cites methods from "Image Stitching based on Feature Ex..."
...are used to find a sequence of linked images transform the image [26]....
[...]
...Lastly, use inliers to re-calculate least squares H if the inliers grow over a specific threshold [26-30]....
[...]
...In which, the feathering image blending method takes the average of pixel values in the blending region from the two overlapped images [26, 33, 34]....
[...]
28 Oct 2020
TL;DR: A path-planning algorithm enables autonomous multidrone aerial surveys of Adélie penguin colonies in Antarctica and can be applied to other domains, such as wildfire surveys in high-risk weather conditions or disaster response.
Abstract: Speed is essential in wildlife surveys due to the dynamic movement of animals throughout their environment and potentially extreme changes in weather. In this work, we present a multirobot path-planning method for conducting aerial surveys over large areas designed to make the best use of limited flight time. Unlike current survey path-planning solutions based on geometric patterns or integer programs, we solve a series of satisfiability modulo theory instances of increasing complexity. Each instance yields a set of feasible paths at each iteration and recovers the set of shortest paths after sufficient time. We implemented our planning algorithm with a team of drones to conduct multiple photographic aerial wildlife surveys of Cape Crozier, one of the largest Adelie penguin colonies in the world containing more than 300,000 nesting pairs. Over 2 square kilometers was surveyed in about 3 hours. In contrast, previous human-piloted single-drone surveys of the same colony required over 2 days to complete. Our method reduces survey time by limiting redundant travel while also allowing for safe recall of the drones at any time during the survey. Our approach can be applied to other domains, such as wildfire surveys in high-risk weather conditions or disaster response.
38Â citations
Cites methods from "Image Stitching based on Feature Ex..."
...Image stitching (23) is normally done via a feature detection algorithm (24), such as scale-invariant feature transform (25), and a bundle adjustment method (26, 27) to align the spatial data....
[...]
01 Oct 2017
TL;DR: This paper designs a quality assessment metric specifically for stitched images, where ghosting and structure inconsistency are the most common visual distortions, and fuse a perceptual geometric error metric and a local structure-guided metric into one.
Abstract: One key enabling component of immersive VR visual experience is the construction of panoramic images-each stitched into one large wide-angle image from multiple smaller viewpoint images captured by different cameras. To better evaluate and design stitching algorithms, a lightweight yet accurate quality metric for stitched panoramic images is desirable. In this paper, we design a quality assessment metric specifically for stitched images, where ghosting and structure inconsistency are the most common visual distortions. Specifically, to efficiently capture these distortion types, we fuse a perceptual geometric error metric and a local structure-guided metric into one. For the geometric error, we compute the local variance of optical flow field energy between the distorted and reference images. For the structure-guided metric, we compute intensity and chrominance gradient in highly-structured patches. The two metrics are content-adaptively combined based on the amount of image structures inherent in the 3D scene. Extensive experiments are conducted on our stitched image quality assessment (SIQA) dataset, which contains 408 groups of examples. Results show that the two parts of metrics complement each other, and the fused metric achieves 94.36% precision with the mean subjective opinion. Our SIQA dataset is made publicly available as part of the submission.
29Â citations
Cites background or methods from "Image Stitching based on Feature Ex..."
...Among the diversity of stitching algorithm literatures, many researchers choose to assess the stitched images by either making comparisons subjectively [24, 14] or using conventional image quality assessment (IQA) metrics [12, 1]....
[...]
...Another way to evaluate stitching algorithm is to adopt classical IQA metrics to stitched images [1, 12] such as MSE (Mean Squared Error) [23], PSNR ((Peak Signalto-Noise Ratio) [17], SSIM (Structural Similarity index) [6] and VSI (Visual Saliency Induced index) [26]....
[...]
01 Nov 2016
TL;DR: A panoramic image stitching algorithm that is able to stitch correctly with a minimum of 5% overlapping area, of images acquired by a wide angle lens, by a horizontal and vertical rotation, and also of a multitude of input images acquired at unfixed axis is presented.
Abstract: In this paper, a panoramic image stitching algorithm is presented. The aim of image stitching is to detect several images of the same scene and merge them to create a larger image. This is achieved by first detecting the overlapping area of the acquired images, and then aligning and blending the seams of the images automatically to create a seamless panoramic image. The experimental testing into the size of the overlapping area, the use of different focal lengths, the use of tilting and panning, and the number of input images were carried out. The results demonstrated that our algorithm is able to stitch correctly with a minimum of 5% overlapping area, of images acquired by a wide angle lens, by a horizontal and vertical rotation, and also of a multitude of input images acquired at unfixed axis.
29Â citations
TL;DR: This paper presents a tile-based adaptive streaming method for 360-degree videos that preserves both the quality and the smoothness of tiles in FoV, thus providing the best QoE for users.
Abstract: The 360-degree video allows users to enjoy the whole scene by interactively switching viewports. However, the huge data volume of the 360-degree video limits its remote applications via network. To provide high quality of experience ( QoE ) for remote web users, this paper presents a tile-based adaptive streaming method for 360-degree videos. First, we propose a simple yet effective rate adaptation algorithm to determine the requested bitrate for downloading the current video segment by considering the balance between the buffer length and video quality. Then, we propose to use a Gaussian model to predict the field of view at the beginning of each requested video segment. To deal with the circumstance that the view angle is switched during the display of a video segment, we propose to download all the tiles in the 360-degree video with different priorities based on a Zipf model. Finally, in order to allocate bitrates for all the tiles, a two-stage optimization algorithm is proposed to preserve the quality of tiles in FoV and guarantee the spatial and temporal smoothness. Experimental results demonstrate the effectiveness and advantage of the proposed method compared with the state-of-the-art methods. That is, our method preserves both the quality and the smoothness of tiles in FoV, thus providing the best QoE for users.
25Â citations
Cites methods from "Image Stitching based on Feature Ex..."
...After aggregating and stitching [14] the images captured by different cameras, a panoramic image can be generated, and finally formed to be a 360-degree video....
[...]
References
More filters
TL;DR: This paper presents a method for extracting distinctive invariant features from images that can be used to perform reliable matching between different views of an object or scene and can robustly identify objects among clutter and occlusion while achieving near real-time performance.
Abstract: This paper presents a method for extracting distinctive invariant features from images that can be used to perform reliable matching between different views of an object or scene. The features are invariant to image scale and rotation, and are shown to provide robust matching across a substantial range of affine distortion, change in 3D viewpoint, addition of noise, and change in illumination. The features are highly distinctive, in the sense that a single feature can be correctly matched with high probability against a large database of features from many images. This paper also describes an approach to using these features for object recognition. The recognition proceeds by matching individual features to a database of features from known objects using a fast nearest-neighbor algorithm, followed by a Hough transform to identify clusters belonging to a single object, and finally performing verification through least-squares solution for consistent pose parameters. This approach to recognition can robustly identify objects among clutter and occlusion while achieving near real-time performance.
46,906Â citations
20 Jun 2005
TL;DR: It is shown experimentally that grids of histograms of oriented gradient (HOG) descriptors significantly outperform existing feature sets for human detection, and the influence of each stage of the computation on performance is studied.
Abstract: We study the question of feature sets for robust visual object recognition; adopting linear SVM based human detection as a test case. After reviewing existing edge and gradient based descriptors, we show experimentally that grids of histograms of oriented gradient (HOG) descriptors significantly outperform existing feature sets for human detection. We study the influence of each stage of the computation on performance, concluding that fine-scale gradients, fine orientation binning, relatively coarse spatial binning, and high-quality local contrast normalization in overlapping descriptor blocks are all important for good results. The new approach gives near-perfect separation on the original MIT pedestrian database, so we introduce a more challenging dataset containing over 1800 annotated human images with a large range of pose variations and backgrounds.
31,952Â citations
"Image Stitching based on Feature Ex..." refers methods in this paper
...There are many features descriptors such as SIFT [22], SURF [14], HOG [17], GLOH [23], PCA-SIFT [16], Pyramidal HOG (PHOG), and Pyramidal Histogram Of visual Words (PHOW)....
[...]
TL;DR: New results are derived on the minimum number of landmarks needed to obtain a solution, and algorithms are presented for computing these minimum-landmark solutions in closed form that provide the basis for an automatic system that can solve the Location Determination Problem under difficult viewing.
Abstract: A new paradigm, Random Sample Consensus (RANSAC), for fitting a model to experimental data is introduced. RANSAC is capable of interpreting/smoothing data containing a significant percentage of gross errors, and is thus ideally suited for applications in automated image analysis where interpretation is based on the data provided by error-prone feature detectors. A major portion of this paper describes the application of RANSAC to the Location Determination Problem (LDP): Given an image depicting a set of landmarks with known locations, determine that point in space from which the image was obtained. In response to a RANSAC requirement, new results are derived on the minimum number of landmarks needed to obtain a solution, and algorithms are presented for computing these minimum-landmark solutions in closed form. These results provide the basis for an automatic system that can solve the LDP under difficult viewing
23,396Â citations
01 Jan 1988
TL;DR: The problem the authors are addressing in Alvey Project MMI149 is that of using computer vision to understand the unconstrained 3D world, in which the viewed scenes will in general contain too wide a diversity of objects for topdown recognition techniques to work.
Abstract: The problem we are addressing in Alvey Project MMI149 is that of using computer vision to understand the unconstrained 3D world, in which the viewed scenes will in general contain too wide a diversity of objects for topdown recognition techniques to work. For example, we desire to obtain an understanding of natural scenes, containing roads, buildings, trees, bushes, etc., as typified by the two frames from a sequence illustrated in Figure 1. The solution to this problem that we are pursuing is to use a computer vision system based upon motion analysis of a monocular image sequence from a mobile camera. By extraction and tracking of image features, representations of the 3D analogues of these features can be constructed.
13,993Â citations
"Image Stitching based on Feature Ex..." refers methods in this paper
...Harris corner is not invariant to scale changes and cross correlation....
[...]
...Harris corner detector [18] is used to detect the features....
[...]
...The processing method involves feature extraction, image matching based on Harris corner detectors method as the feature detection and neighboring pairs alignment using RANSAC (RANdom Sample Consensus) algorithm....
[...]
...[18] Harris, C., & Stephens, M. (1988)....
[...]
...The closest system to ORB proposes a multi-scale Harris keypoint and oriented patch descriptor....
[...]
TL;DR: A flexible technique to easily calibrate a camera that only requires the camera to observe a planar pattern shown at a few (at least two) different orientations is proposed and advances 3D computer vision one more step from laboratory environments to real world use.
Abstract: We propose a flexible technique to easily calibrate a camera. It only requires the camera to observe a planar pattern shown at a few (at least two) different orientations. Either the camera or the planar pattern can be freely moved. The motion need not be known. Radial lens distortion is modeled. The proposed procedure consists of a closed-form solution, followed by a nonlinear refinement based on the maximum likelihood criterion. Both computer simulation and real data have been used to test the proposed technique and very good results have been obtained. Compared with classical techniques which use expensive equipment such as two or three orthogonal planes, the proposed technique is easy to use and flexible. It advances 3D computer vision one more step from laboratory environments to real world use.
13,200Â citations
"Image Stitching based on Feature Ex..." refers background in this paper
...Intrinsic camera parameters link the pixel coordinates of an image point with the corresponding coordinates in the camera reference frame [9]....
[...]