Showing papers on "Scale-invariant feature transform published in 2008"

SIFT Flow: Dense Correspondence across Different Scenes

Abstract: While image registration has been studied in different areas of computer vision, aligning images depicting different scenes remains a challenging problem, closer to recognition than to image matching Analogous to optical flow, where an image is aligned to its temporally adjacent frame, we propose SIFT flow, a method to align an image to its neighbors in a large image collection consisting of a variety of scenes For a query image, histogram intersection on a bag-of-visual-words representation is used to find the set of nearest neighbors in the database The SIFT flow algorithm then consists of matching densely sampled SIFT features between the two images, while preserving spatial discontinuities The use of SIFT features allows robust matching across different scene/object appearances and the discontinuity-preserving spatial model allows matching of objects located at different parts of the scene Experiments show that the proposed approach is able to robustly align complicated scenes with large spatial distortions We collect a large database of videos and apply the SIFT flow algorithm to two applications: (i) motion field prediction from a single static image and (ii) motion synthesis via transfer of moving objects

CenSurE: Center Surround Extremas for Realtime Feature Detection and Matching

Abstract: We explore the suitability of different feature detectors for the task of image registration, and in particular for visual odometry, using two criteria: stability (persistence across viewpoint change) and accuracy (consistent localization across viewpoint change). In addition to the now-standard SIFT, SURF, FAST, and Harris detectors, we introduce a suite of scale-invariant center-surround detectors (CenSurE) that outperform the other detectors, yet have better computational characteristics than other scale-space detectors, and are capable of real-time implementation.

A fast local descriptor for dense matching

Abstract: We introduce a novel local image descriptor designed for dense wide-baseline matching purposes. We feed our descriptors to a graph-cuts based dense depth map estimation algorithm and this yields better wide-baseline performance than the commonly used correlation windows for which the size is hard to tune. As a result, unlike competing techniques that require many high-resolution images to produce good reconstructions, our descriptor can compute them from pairs of low-quality images such as the ones captured by video streams. Our descriptor is inspired from earlier ones such as SIFT and GLOH but can be computed much faster for our purposes. Unlike SURF which can also be computed efficiently at every pixel, it does not introduce artifacts that degrade the matching performance. Our approach was tested with ground truth laser scanned depth maps as well as on a wide variety of image pairs of different resolutions and we show that good reconstructions are achieved even with only two low quality images.

Salient local visual features for shape-based 3D model retrieval

Abstract: In this paper, we describe a shape-based 3D model retrieval method based on multi-scale local visual features. The features are extracted from 2D range images of the model viewed from uniformly sampled locations on a view sphere. The method is appearance-based, and accepts all the models that can be rendered as a range image. For each range image, a set of 2D multi-scale local visual features is computed by using the scale invariant feature transform [22] algorithm. To reduce cost of distance computation and feature storage, a set of local features describing a 3D model is integrated into a histogram using the bag-of-features approach. Our experiments using two standard benchmarks, one for articulated shapes and the other for rigid shapes, showed that the methods achieved the performance comparable or superior to some of the most powerful 3D shape retrieval methods.

Keypoint Detection and Local Feature Matching for Textured 3D Face Recognition

Abstract: Holistic face recognition algorithms are sensitive to expressions, illumination, pose, occlusions and makeup. On the other hand, feature-based algorithms are robust to such variations. In this paper, we present a feature-based algorithm for the recognition of textured 3D faces. A novel keypoint detection technique is proposed which can repeatably identify keypoints at locations where shape variation is high in 3D faces. Moreover, a unique 3D coordinate basis can be defined locally at each keypoint facilitating the extraction of highly descriptive pose invariant features. A 3D feature is extracted by fitting a surface to the neighborhood of a keypoint and sampling it on a uniform grid. Features from a probe and gallery face are projected to the PCA subspace and matched. The set of matching features are used to construct two graphs. The similarity between two faces is measured as the similarity between their graphs. In the 2D domain, we employed the SIFT features and performed fusion of the 2D and 3D features at the feature and score-level. The proposed algorithm achieved 96.1% identification rate and 98.6% verification rate on the complete FRGC v2 data set.

Multi-spectral remote image registration based on SIFT

Abstract: The number of incorrect matches of SIFT keypoints will sharply increase when the SIFT-based registration algorithm is adopted for pairs of multi-spectral remote images, owing to the significant difference in the image intensity between multi-spectral images. Scale restriction criteria for keypoint matching are proposed, and experimental results demonstrate that it will greatly improve match performance.

Circular Object Detection Using a Modified Hough Transform

Abstract: A practical modification of the Hough transform is proposed that improves the detection of low-contrast circular objects. The original circular Hough transform and its numerous modifications are discussed and compared in order to improve both the efficiency and computational complexity of the algorithm. Medical images are selected to verify the algorithm. In particular, the algorithm is applied to localize cell nuclei of cytological smears visualized using a phase contrast microscope.

Automated insect identification through concatenated histograms of local appearance features: feature vector generation and region detection for deformable objects

Abstract: This paper describes a computer vision approach to automated rapid-throughput taxonomic identification of stonefly larvae. The long-term objective of this research is to develop a cost-effective method for environmental monitoring based on automated identification of indicator species. Recognition of stonefly larvae is challenging because they are highly articulated, they exhibit a high degree of intraspecies variation in size and color, and some species are difficult to distinguish visually, despite prominent dorsal patterning. The stoneflies are imaged via an apparatus that manipulates the specimens into the field of view of a microscope so that images are obtained under highly repeatable conditions. The images are then classified through a process that involves (a) identification of regions of interest, (b) representation of those regions as SIFT vectors (Lowe, in Int J Comput Vis 60(2):91–110, 2004) (c) classification of the SIFT vectors into learned “features” to form a histogram of detected features, and (d) classification of the feature histogram via state-of-the-art ensemble classification algorithms. The steps (a) to (c) compose the concatenated feature histogram (CFH) method. We apply three region detectors for part (a) above, including a newly developed principal curvature-based region (PCBR) detector. This detector finds stable regions of high curvature via a watershed segmentation algorithm. We compute a separate dictionary of learned features for each region detector, and then concatenate the histograms prior to the final classification step. We evaluate this classification methodology on a task of discriminating among four stonefly taxa, two of which, Calineuria and Doroneuria, are difficult even for experts to discriminate. The results show that the combination of all three detectors gives four-class accuracy of 82% and three-class accuracy (pooling Calineuria and Doro-neuria) of 95%. Each region detector makes a valuable contribution. In particular, our new PCBR detector is able to discriminate Calineuria and Doroneuria much better than the other detectors.

Image Feature Extraction Using Gradient Local Auto-Correlations

Abstract: In this paper, we propose a method for extracting image features which utilizes 2nd order statistics, i.e., spatial and orientational auto-correlations of local gradients. It enables us to extract richer information from images and to obtain more discriminative power than standard histogram based methods. The image gradients are sparsely described in terms of magnitude and orientation. In addition, normal vectors on the image surface are derived from the gradients and these could also be utilized instead of the gradients. From a geometrical viewpoint, the method extracts information about not only the gradients but also the curvatures of the image surface. Experimental results for pedestrian detection and image patch matching demonstrate the effectiveness of the proposed method compared with other methods, such as HOG and SIFT.

Full orientation invariance and improved feature selectivity of 3D SIFT with application to medical image analysis

Abstract: This paper presents a comprehensive extension of the Scale Invariant Feature Transform (SIFT), originally introduced in 2D, to volumetric images. While tackling the significant computational efforts required by such multiscale processing of large data volumes, our implementation addresses two important mathematical issues related to the 2D-to-3D extension. It includes efficient steps to filter out extracted point candidates that have low contrast or are poorly localized along edges or ridges. In addition, it achieves, for the first time, full 3D orientation invariance of the descriptors, which is essential for 3D feature matching. An application of this technique is demonstrated to the feature-based automated registration and segmentation of clinical datasets in the context of radiation therapy.

Improved SIFT-features matching for object recognition

Abstract: The SIFT algorithm (Scale Invariant Feature Transform) proposed by Lowe [1] is an approach for extracting distinctive invariant features from images. It has been successfully applied to a variety of computer vision problems based on feature matching including object recognition, pose estimation, image retrieval and many others. However, in real-world applications there is still a need for improvement of the algorithm's robustness with respect to the correct matching of SIFT features. In this paper, an improvement of the original SIFT algorithm providing more reliable feature matching for the purpose of object recognition is proposed. The main idea is to divide the features extracted from both the test and the model object image into several sub-collections before they are matched. The features are divided into several sub-collections considering the features arising from different octaves, that is from different frequency domains. To evaluate the performance of the proposed approach, it was applied to real images acquired with the stereo camera system of the rehabilitation robotic system FRIEND II. The experimental results show an increase in the number of correct features matched and, at the same time, a decrease in the number of outliers in comparison with the original SIFT algorithm. Compared with the original SIFT algorithm, a 40% reduction in processing time was achieved for the matching of the stereo images.

Robust Affine Invariant Feature Extraction for Image Matching

Abstract: A new approach is presented to extract more robust affine invariant features for image matching. The novelty of our approach is a hierarchical filtering strategy for affine invariant feature detection, which is based on information entropy and spatial dispersion quality constraints. The concept of spatial dispersion quality is introduced to quantify the spatial distribution of features. Moreover, an integrated algorithm combined by the filtering strategy, maximally stable extremal region (MSER) and scale invariant feature transform, is introduced for affine invariant feature extraction. Since Mikolajczyk et al. identified that MSER is the best detector in many cases, we design an experiment to compare our approach (ED-MSER) with the standard MSER. By using two stereo pairs and an image sequence with different types of imagery, the experiment indicates that ED-MSER can always get much higher repeatability and matching score compared to the standard MSER and other algorithms, thus benefiting the subsequent image matching and many other applications.

Mobile museum guide based on fast SIFT recognition

Abstract: This article explores the feasibility of a market-ready, mobile pattern recognition system based on the latest findings in the field of object recognition and currently available hardware and network technology. More precisely, an innovative, mobile museum guide system is presented, which enables camera phones to recognize paintings in art galleries. After careful examination, the algorithms Scale-Invariant Feature Transform (SIFT) and Speeded Up Robust Features (SURF) were found most promising for this goal. Consequently, both have been integrated in a fully implemented prototype system and their performance has been thoroughly evaluated under realistic conditions. In order to speed up the matching process for finding the corresponding sample in the feature database, an approximation to Nearest Neighbor Search was investigated. The k-means based clustering approach was found to significantly improve the computational time.

Automatic image stitching using SIFT

Abstract: This paper concerns the problem of automatic image stitching which mainly applies to the image sequence even those including noise images. And it uses a method based on invariant features to realize fully automatic image stitching, in which it includes two main parts: image matching and image blending. As the noises images have large differences between the other images, when using SIFT features to realize correct and robust matching, it supplies a probabilistic model to verify the panorama image sequence. Addison to have a more satisfied panorama image, it uses a simple and fast blending method which is weighted average method. Finally, the experiment results confirm the feasibility of our methods.

Robust 2D Ear Registration and Recognition Based on SIFT Point Matching

Abstract: Significant recent progress has shown ear recognition to be a viable biometric. Good recognition rates have been demonstrated under controlled conditions, using manual registration or with specialised equipment. This paper describes a new technique which improves the robustness of ear registration and recognition, addressing issues of pose variation, background clutter and occlusion. By treating the ear as a planar surface and creating a homography transform using SIFT feature matches, ears can be registered accurately. The feature matches reduce the gallery size and enable a precise ranking using a simple 2D distance algorithm. When applied to the XM2VTS database it gives results comparable to PCA with manual registration. Further analysis on more challenging datasets demonstrates the technique to be robust to background clutter, viewing angles up to plusmn13 degrees and with over 20% occlusion.

Traffic sign recognition using Scale Invariant Feature Transform and color classification

Abstract: In this paper, we propose a traffic sign detection and recognition technique by augmenting the scale invariant feature transform (SIFT) with new features related to the color of local regions. SIFT finds local invariant features in a given image and matches these features to the features of images that exist in the training set. Recognition is performed by finding out the training image that gives the maximum number of matches. In this study, performance of SIFT in traffic sign detection and recognition issue is investigated. Afterwards, new features which increase the performance are added. Those are color inspection by using proposed color classification method and inspecting the orientations of SIFT features. These features check the accuracy of matches which are found by SIFT. Color classification method finds out true colors of the pixels by applying some classification rules. It is observed that adding color and orientation inspections raises the recognition performance of SIFT significantly. Obtained results are very good and satisfying even for the images containing traffic signs which are rotated, have undergone affine transformations, have been damaged, occluded, overshadowed, had alteration in color, pictured in different weather conditions and different illumination conditions.

Video object matching based on SIFT algorithm

Abstract: SIFT (scale invariant feature transform) is used to solve visual tracking problem, where the appearances of the tracked object and scene background change during tracking. The implementation of this algorithm has five major stages: scale-space extrema detection; keypoint localization; orientation assignment; keypoint descriptor; keypoint matching. From the beginning frame, object is selected as the template, its SIFT features are computed. Then in the following frames, the SIFT features are computed. Euclidean distance between the object's SIFT features and the frames' SIFT features can be used to compute the accurate position of the matched object. The experimental results on real video sequences demonstrate the effectiveness of this approach and show this algorithm is of higher robustness and real-time performance. It can solve the matching problem with translation, rotation and affine distortion between images. It plays an important role in video object tracking and video object retrieval.

Object recognition and segmentation using SIFT and Graph Cuts

Abstract: In this paper, we propose a method of object recognition and segmentation using scale-invariant feature transform (SIFT) and graph cuts. SIFT feature is invariant for rotations, scale changes, and illumination changes and it is often used for object recognition. However, in previous object recognition work using SIFT, the object region is simply presumed by the affine-transformation and the accurate object region was not segmented. On the other hand, graph cuts is proposed as a segmentation method of a detail object region. But it was necessary to give seeds manually. By combing SIFT and graph cuts, in our method, the existence of objects is recognized first by vote processing of SIFT keypoints. After that, the object region is cut out by graph cuts using SIFT keypoints as seeds. Thanks to this combination, both recognition and segmentation are performed automatically under cluttered backgrounds including occlusion.

Near duplicate image identification with patially Aligned Pyramid Matching

Abstract: A new framework, termed spatially aligned pyramid matching, is proposed for near duplicate image identification. The proposed method robustly handles spatial shifts as well as scale changes. Images are divided into both overlapped and non-overlapped blocks over multiple levels. In the first matching stage, pairwise distances between blocks from the examined image pair are computed using SIFT features and Earth Moverpsilas distance (EMD). In the second stage, multiple alignment hypotheses that consider piecewise spatial shifts and scale variation are postulated and resolved using integer-flow EMD. Two application scenarios are addressed - retrieval ranking and binary classification. For retrieval ranking, a pyramid-based scheme is constructed to fuse matching results from different partition levels. For binary classification, a novel generalized neighborhood component analysis method is formulated that can be effectively used in tandem with SVMs to select the most critical matching components. The proposed methods are shown to clearly outperform existing methods through extensive testing on the Columbia near duplicate image database and another new dataset.

Parallelizing the Hough Transform Computation

Abstract: The Hough transform, a widely used operation in image processing, suffers from being computationally intensive. In this letter, we propose a method called additive Hough transform (AHT) to accelerate Hough transform computation. It is based on parallel processing of k2 points, obtained by dividing the edge map into uniform blocks using a k times k grid. It has been shown that AHT on verification gives the same Hough profile as the conventional Hough transform. Further, AHT reduces the total computation time by at least k2 times as compared to existing Hough transform architectures.

Image Mosaic Based on SIFT

Abstract: The traditional feature-based algorithm was found to be sensitive to rotations and scales. In this paper, an automatic image mosaic technique based on SIFT (Scale Invariant Feature Transform) was proposed by using the rotation and scale invariant property of SIFT. Keypoints are first extracted by searching over all scales and image locations, then the descriptors defined on the keypoint neighborhood are computed, through to compare the Euclidean distance of their descriptors to extract the initial feature points pair, then eliminate spurious feature points pair by applying RANSAC, finally transform the input image with the correct mapping model for image fusion and complete image stitching. Experimental results demonstrate the proposed algorithm is robust to translation, rotation, noise and scaling.

Image Matching for Geomorphic Measurement Based on SIFT and RANSAC Methods

Abstract: To effectively realize the image feature matching for geomorphic reverse measurement and rebuilding, a new matching scheme is presented, where the SIFT method are adopted to implement initial geomorphic image matching by going through five stages: scale-space construction, scale-space extrema detection, orientation assignment, keypoint descriptor and feature vector matching. Then, in order to eliminate the wrong matching features existing in the initial matching process, RANSAC algorithm is applied. The experimental results show that this algorithm can effectively improve the accuracy and efficiency of geomorphic image matching.

Creating Panoramic Images for Bladder Fluorescence Endoscopy

Abstract: The medical diagnostic analysis and therapy of urinary bladder cancer based on endoscopes are state of the art in urological medicine. Due to the limited field of view of endoscopes, the physician can examine only a small part of the whole operating field at once. This constraint makes visual control and navigation difficult, especially in hollow organs. A panoramic image, covering a larger field of view, can overcome this difficulty. Directly motivated by a physician we developed an image mosaicing algorithm for endoscopic bladder fluorescence video sequences. In this paper, we present an approach which is capable of stitching single endoscopic video images to a combined panoramic image. Based on SIFT features we estimate a 2-D homography for each image pair, using an affine model and an iterative model-fitting algorithm. We then apply the stitching process and perform a mutual linear interpolation. Our panoramic image results show a correct stitching and lead to a better overview and understanding of the operation field.

Feature group matching for appearance-based localization

Abstract: Local feature matching has become a commonly used method to compare images. For mobile robots, a reliable method for comparing images can constitute a key component for localization tasks. In this paper, we address the issues of appearance-based topological and metric localization by introducing a novel group matching approach to select less but more robust features to match the current robot view with reference images. Feature group matching is based on the consideration that feature descriptors together with spatial relations are more robust than classical approaches. Our datasets, each consisting of a large number of omnidirectional images, have been acquired over different day times (different lighting conditions) both in indoor and outdoor environments. The feature group matching outperforms the SIFT in indoor localization showing better performances both in the case of topological and metric localization. In outdoor SURF remains the best feature extraction method, as reported in literature.

Super-resolution mosaicking of UAV surveillance video

Abstract: This paper explains and implements our efficient multi-frame super-resolution mosaicking algorithm. In this algorithm, feature points between images are matched using SIFT, and then random M-least squares is used to estimate the homography between frames. Next, separate frames are registered and the overlapping region is extracted. A generative model is then adopted and combined with maximum a posteriori estimation to construct the underdetermined sparse linear system. To solve the ill-posed large-scale inverse system, we derive and implement a new hybrid regularization (bilateral total variance Hubert) method. Cross validation is utilized to estimate the derivative of the blur kernel as well as the regularization parameter. Super-resolution is then applied to the individual sub-frames from the overlapping region. Finally, multi-band blending is used to stitch these resolution-enhanced frames to form the final image. The whole process is semi-real time (roughly 30 seconds for 35 frames) and the effectiveness of our algorithm is validated by applying it to real and synthetic UAV video frames.

Design and Implementation of a Dual-Camera Wireless Sensor Network for Object Retrieval

Abstract: This paper presents the design and implementation of a dual-camera sensor network that can be used as a memory assistant tool for assisted living. Our system performs energy-efficient object detection and recognition of commonly misplaced objects. The novelty in our approach is the ability to tradeoff between recognition accuracy and computational efficiency by employing a combination of low complexity but less precise color histogram-based image recognition together with more complex image recognition using SIFT descriptors. In addition, our system can seamlessly integrate feedback from the user to improve the robustness of object recognition. Experimental results reveal that our system is computation-efficient and adaptive to slow changes of environmental conditions.

Local 3D structure recognition in range images

Abstract: A feature detector and a feature descriptor are presented, which are applicable to 3D range data. The feature detector is used to identify locations in the range data at which the feature descriptor is applied. The feature descriptor, or feature transform, calculates a signature for each identified location on the basis of local shape information. The approach used in both the feature detector and the descriptor is motivated by the success of the scale invariant feature transform and speeded up robust features approaches in the 2D case. Using synthetic data, the authors evaluate the repeatability of the detector and robustness of the descriptor to global transformations and image noise. The complete system is then applied to the problem of automatic detection of repeated structure in real range images.

Facial feature detection with optimal pixel reduction SVM

Abstract: Automatic facial feature localization has been a long-standing challenge in the field of computer vision for several decades. This can be explained by the large variation a face in an image can have due to factors such as position, facial expression, pose, illumination, and background clutter. Support Vector Machines (SVMs) have been a popular statistical tool for facial feature detection. Traditional SVM approaches to facial feature detection typically extract features from images (e.g. multiband filter, SIFT features) and learn the SVM parameters. Independently learning features and SVM parameters might result in a loss of information related to the classification process. This paper proposes an energy-based framework to jointly perform relevant feature weighting and SVM parameter learning. Preliminary experiments on standard face databases have shown significant improvement in speed with our approach.

System and method for sift implementation and optimization

Abstract: A method is to implement a Scale Invariant Feature Transform algorithm in a shared memory multiprocessing system. The method comprises building differences of Gaussian (DoG) images for an input image, detecting keypoints in the DoG images; assigning orientations to the keypoints and computing keypoints descriptors and performing matrix operations. In the method, building differences of Gaussian (DoG) images for an input image and detecting keypoints in the DoG images are executed for all scales of the input image in parallel. And, orientation assignment and keypoints descriptions computation are executed for all octaves of the input image in parallel.