Distinctive Image Features from Scale-Invariant Keypoints
01 Jan 2011-
TL;DR: The Scale-Invariant Feature Transform (or SIFT) algorithm is a highly robust method to extract and consequently match distinctive invariant features from images that can then be used to reliably match objects in diering images.
Abstract: The Scale-Invariant Feature Transform (or SIFT) algorithm is a highly robust method to extract and consequently match distinctive invariant features from images. These features can then be used to reliably match objects in diering images. The algorithm was rst proposed by Lowe [12] and further developed to increase performance resulting in the classic paper [13] that served as foundation for SIFT which has played an important role in robotic and machine vision in the past decade.
Citations
More filters
08 Sep 2018
TL;DR: DetNet is proposed, which is a novel backbone network specifically designed for object detection that includes the extra stages against traditional backbone network for image classification, while maintains high spatial resolution in deeper layers.
Abstract: Recent CNN based object detectors, either one-stage methods like YOLO, SSD, and RetinaNet, or two-stage detectors like Faster R-CNN, R-FCN and FPN, are usually trying to directly finetune from ImageNet pre-trained models designed for the task of image classification. However, there has been little work discussing the backbone feature extractor specifically designed for the task of object detection. More importantly, there are several differences between the tasks of image classification and object detection. (i) Recent object detectors like FPN and RetinaNet usually involve extra stages against the task of image classification to handle the objects with various scales. (ii) Object detection not only needs to recognize the category of the object instances but also spatially locate them. Large downsampling factors bring large valid receptive field, which is good for image classification, but compromises the object location ability. Due to the gap between the image classification and object detection, we propose DetNet in this paper, which is a novel backbone network specifically designed for object detection. Moreover, DetNet includes the extra stages against traditional backbone network for image classification, while maintains high spatial resolution in deeper layers. Without any bells and whistles, state-of-the-art results have been obtained for both object detection and instance segmentation on the MSCOCO benchmark based on our DetNet (4.8G FLOPs) backbone. Codes will be released (https://github.com/zengarden/DetNet).
233 citations
Cites methods from "Distinctive Image Features from Sca..."
...Old detectors extract image features by using hand-engineered object component descriptors, such as HOG [5], SIFT [26], Selective Search [37], Edge Box [41]....
[...]
01 Dec 2013
TL;DR: A match kernel is proposed that takes the best of existing techniques by combining an aggregation procedure with a selective match kernel, providing a large scale image search both precise and scalable, as shown by the experiments on several benchmarks.
Abstract: This paper considers a family of metrics to compare images based on their local descriptors. It encompasses the VLAD descriptor and matching techniques such as Hamming Embedding. Making the bridge between these approaches leads us to propose a match kernel that takes the best of existing techniques by combining an aggregation procedure with a selective match kernel. Finally, the representation underpinning this kernel is approximated, providing a large scale image search both precise and scalable, as shown by our experiments on several benchmarks.
233 citations
TL;DR: It is demonstrated that computer vision can support ultra-low-cost, user-deployed high spatial resolution 3D remote sensing of vegetation structure and spectral characteristics from ordinary digital photographs acquired using inexpensive hobbyist aerial platforms.
Abstract: High spatial resolution measurements of vegetation structure in three-dimensions (3D) are essential for accurate estimation of vegetation biomass, carbon accounting, forestry, fire hazard evaluation and other land management and scientific applications. Light Detection and Ranging (LiDAR) is the current standard for these measurements but requires bulky instruments mounted on commercial aircraft. Here we demonstrate that high spatial resolution 3D measurements of vegetation structure and spectral characteristics can be produced by applying open-source computer vision algorithms to ordinary digital photographs acquired using inexpensive hobbyist aerial platforms. Digital photographs were acquired using a kite aerial platform across two 2.25 ha test sites in Baltimore, MD, USA. An open-source computer vision algorithm generated 3D point cloud datasets with RGB spectral attributes from the photographs and these were geocorrected to a horizontal precision of 0.82) than computer vision (R2 > 0.64), primarily because of difficulties observing terrain under closed canopy forest. Results confirm that computer vision can support ultra-low-cost, user-deployed high spatial resolution 3D remote sensing of vegetation structure.
232 citations
Cites methods from "Distinctive Image Features from Sca..."
...Bundler is a new open-source SfM software package [19] that combines the SIFT algorithm (Scale Invariant Feature Transform) [20] for keypoint extraction with bundle adjustment using the Sparse Remote Sens. 2010, 2 1159 Bundle Adjustment package (SBA) [21]....
[...]
...Bundler is a new open-source SfM software package [19] that combines the SIFT algorithm (Scale Invariant Feature Transform) [20] for keypoint extraction with bundle adjustment using the Sparse...
[...]
23 Jun 2013
TL;DR: This work proposes a joint Bayesian adaptation algorithm to adapt the universally trained GMM to better model the pose variations between the target pair of faces/face tracks, which consistently improves face verification accuracy.
Abstract: Pose variation remains to be a major challenge for real-world face recognition. We approach this problem through a probabilistic elastic matching method. We take a part based representation by extracting local features (e.g., LBP or SIFT) from densely sampled multi-scale image patches. By augmenting each feature with its location, a Gaussian mixture model (GMM) is trained to capture the spatial-appearance distribution of all face images in the training corpus. Each mixture component of the GMM is confined to be a spherical Gaussian to balance the influence of the appearance and the location terms. Each Gaussian component builds correspondence of a pair of features to be matched between two faces/face tracks. For face verification, we train an SVM on the vector concatenating the difference vectors of all the feature pairs to decide if a pair of faces/face tracks is matched or not. We further propose a joint Bayesian adaptation algorithm to adapt the universally trained GMM to better model the pose variations between the target pair of faces/face tracks, which consistently improves face verification accuracy. Our experiments show that our method outperforms the state-of-the-art in the most restricted protocol on Labeled Face in the Wild (LFW) and the YouTube video face database by a significant margin.
232 citations
Cites background from "Distinctive Image Features from Sca..."
...Each face image is densely partitioned into overlapping patches at multiple scales, from each of which a local feature such as Local Binary Pattern (LBP) [1] or SIFT [19] is extracted....
[...]
...Our method only employed simple visual features such as LBP and SIFT....
[...]
...As shown in Figure 1, SIFT and LBP features are extracted over each scale for a 3-scale Gaussian image pyramid with scaling factor 0.9....
[...]
...We take a part based representation by extracting local features (e.g., LBP or SIFT) from densely sampled multi-scale image patches....
[...]
...SIFT features are extracted from patches from a 8x8 sliding window with 4-pixel spacing, and LBP features2 are extracted from a 32x32 sliding window with 4-pixel spacing....
[...]
29 Oct 2012
TL;DR: The article presents an investigation of automated image orientation packages in order to clarify potentialities and performances when dealing with large and complex datasets.
Abstract: The recent developments in automated image processing for 3D reconstruction purposes have led to the diffusion of low-cost and open-source solutions which can be nowadays used by everyone to produce 3D models. The level of automation is so high that many solutions are black-boxes with poor repeatability and low reliability. The article presents an investigation of automated image orientation packages in order to clarify potentialities and performances when dealing with large and complex datasets.
231 citations
Cites background from "Distinctive Image Features from Sca..."
...- Bundler: developed by the University of Washington & Microsoft, it was created with the aim of reconstructing 3D scene using a huge number of images downloaded approach, with RANSAC to estimate the F matrix from the extracted SIFT features and reject possible outlier for every couple of images....
[...]
...Nowadays SIFT [28] and SURF [29] algorithms provide highly distinctive features invariant to image scaling and rotations with an associate descriptor (64- or 128-dimensional vector) to each extracted image feature....
[...]
...This software uses a modified SIFT++ feature extractor [35] and allows to choose between several camera model (Brown’s, fisheye, etc.)....
[...]
...The numerical solution to the problem of function minimization is generally sough with methods like Levenberg-Marquardt, Gauss-Newton or Gauss-Markov. quadratic matching or kd-tree feature extraction: SIFT-like, SURF, etc. descriptor comparison and pairwise correspondences extraction son robust outlier rejection: E/F matrix or T tensor with RANSAC, MAPSAC or LMedS method bundle adjustment concatenation of all image combinations and extraction of image correspondences for the entire image block the datum (or gauge) definition problem....
[...]
References
More filters
TL;DR: This paper presents a method for extracting distinctive invariant features from images that can be used to perform reliable matching between different views of an object or scene and can robustly identify objects among clutter and occlusion while achieving near real-time performance.
Abstract: This paper presents a method for extracting distinctive invariant features from images that can be used to perform reliable matching between different views of an object or scene. The features are invariant to image scale and rotation, and are shown to provide robust matching across a substantial range of affine distortion, change in 3D viewpoint, addition of noise, and change in illumination. The features are highly distinctive, in the sense that a single feature can be correctly matched with high probability against a large database of features from many images. This paper also describes an approach to using these features for object recognition. The recognition proceeds by matching individual features to a database of features from known objects using a fast nearest-neighbor algorithm, followed by a Hough transform to identify clusters belonging to a single object, and finally performing verification through least-squares solution for consistent pose parameters. This approach to recognition can robustly identify objects among clutter and occlusion while achieving near real-time performance.
46,906 citations
20 Sep 1999
TL;DR: Experimental results show that robust object recognition can be achieved in cluttered partially occluded images with a computation time of under 2 seconds.
Abstract: An object recognition system has been developed that uses a new class of local image features. The features are invariant to image scaling, translation, and rotation, and partially invariant to illumination changes and affine or 3D projection. These features share similar properties with neurons in inferior temporal cortex that are used for object recognition in primate vision. Features are efficiently detected through a staged filtering approach that identifies stable points in scale space. Image keys are created that allow for local geometric deformations by representing blurred image gradients in multiple orientation planes and at multiple scales. The keys are used as input to a nearest neighbor indexing method that identifies candidate object matches. Final verification of each match is achieved by finding a low residual least squares solution for the unknown model parameters. Experimental results show that robust object recognition can be achieved in cluttered partially occluded images with a computation time of under 2 seconds.
16,989 citations
01 Jan 1988
TL;DR: The problem the authors are addressing in Alvey Project MMI149 is that of using computer vision to understand the unconstrained 3D world, in which the viewed scenes will in general contain too wide a diversity of objects for topdown recognition techniques to work.
Abstract: The problem we are addressing in Alvey Project MMI149 is that of using computer vision to understand the unconstrained 3D world, in which the viewed scenes will in general contain too wide a diversity of objects for topdown recognition techniques to work. For example, we desire to obtain an understanding of natural scenes, containing roads, buildings, trees, bushes, etc., as typified by the two frames from a sequence illustrated in Figure 1. The solution to this problem that we are pursuing is to use a computer vision system based upon motion analysis of a monocular image sequence from a mobile camera. By extraction and tracking of image features, representations of the 3D analogues of these features can be constructed.
13,993 citations
TL;DR: It is observed that the ranking of the descriptors is mostly independent of the interest region detector and that the SIFT-based descriptors perform best and Moments and steerable filters show the best performance among the low dimensional descriptors.
Abstract: In this paper, we compare the performance of descriptors computed for local interest regions, as, for example, extracted by the Harris-Affine detector [Mikolajczyk, K and Schmid, C, 2004]. Many different descriptors have been proposed in the literature. It is unclear which descriptors are more appropriate and how their performance depends on the interest region detector. The descriptors should be distinctive and at the same time robust to changes in viewing conditions as well as to errors of the detector. Our evaluation uses as criterion recall with respect to precision and is carried out for different image transformations. We compare shape context [Belongie, S, et al., April 2002], steerable filters [Freeman, W and Adelson, E, Setp. 1991], PCA-SIFT [Ke, Y and Sukthankar, R, 2004], differential invariants [Koenderink, J and van Doorn, A, 1987], spin images [Lazebnik, S, et al., 2003], SIFT [Lowe, D. G., 1999], complex filters [Schaffalitzky, F and Zisserman, A, 2002], moment invariants [Van Gool, L, et al., 1996], and cross-correlation for different types of interest regions. We also propose an extension of the SIFT descriptor and show that it outperforms the original method. Furthermore, we observe that the ranking of the descriptors is mostly independent of the interest region detector and that the SIFT-based descriptors perform best. Moments and steerable filters show the best performance among the low dimensional descriptors.
7,057 citations
TL;DR: The high utility of MSERs, multiple measurement regions and the robust metric is demonstrated in wide-baseline experiments on image pairs from both indoor and outdoor scenes.
3,422 citations