This paper presents a method for extracting distinctive invariant features from images that can be used to perform reliable matching between different views of an object or scene. The features are invariant to image scale and rotation, and are shown to provide robust matching across a substantial range of affine distortion, change in 3D viewpoint, addition of noise, and change in illumination. The features are highly distinctive, in the sense that a single feature can be correctly matched with high probability against a large database of features from many images. This paper also describes an approach to using these features for object recognition. The recognition proceeds by matching individual features to a database of features from known objects using a fast nearest-neighbor algorithm, followed by a Hough transform to identify clusters belonging to a single object, and finally performing verification through least-squares solution for consistent pose parameters. This approach to recognition can robustly identify objects among clutter and occlusion while achieving near real-time performance.

/pdf/distinctive-image-features-from-scale-invariant-keypoints-3waurjqzke.pdf

Distinctive Image Features from Scale-Invariant Keypoints

An object recognition system has been developed that uses a new class of local image features. The features are invariant to image scaling, translation, and rotation, and partially invariant to illumination changes and affine or 3D projection. These features share similar properties with neurons in inferior temporal cortex that are used for object recognition in primate vision. Features are efficiently detected through a staged filtering approach that identifies stable points in scale space. Image keys are created that allow for local geometric deformations by representing blurred image gradients in multiple orientation planes and at multiple scales. The keys are used as input to a nearest neighbor indexing method that identifies candidate object matches. Final verification of each match is achieved by finding a low residual least squares solution for the unknown model parameters. Experimental results show that robust object recognition can be achieved in cluttered partially occluded images with a computation time of under 2 seconds.

/pdf/object-recognition-from-local-scale-invariant-features-5887b1cgwf.pdf

Object recognition from local scale-invariant features

The Scale-Invariant Feature Transform (or SIFT) algorithm is a highly robust method to extract and consequently match distinctive invariant features from images. These features can then be used to reliably match objects in diering images. The algorithm was rst proposed by Lowe [12] and further developed to increase performance resulting in the classic paper [13] that served as foundation for SIFT which has played an important role in robotic and machine vision in the past decade.

In this paper, we present a novel scale- and rotation-invariant interest point detector and descriptor, coined SURF (Speeded Up Robust Features). It approximates or even outperforms previously proposed schemes with respect to repeatability, distinctiveness, and robustness, yet can be computed and compared much faster.

This is achieved by relying on integral images for image convolutions; by building on the strengths of the leading existing detectors and descriptors (in casu, using a Hessian matrix-based measure for the detector, and a distribution-based descriptor); and by simplifying these methods to the essential. This leads to a combination of novel detection, description, and matching steps. The paper presents experimental results on a standard evaluation set, as well as on imagery obtained in the context of a real-life object recognition application. Both show SURF's strong performance.

/pdf/surf-speeded-up-robust-features-1fje09jteo.pdf

SURF: speeded up robust features

https://www.cs.jhu.edu/~misha/ReadingSeminar/Papers/Bay08.pdf

Speeded-Up Robust Features (SURF)

Generalized Gaussian scale-space axiomatics comprising linear scale-space, affine scale-space and sp

Elementary techniques from real analysis, and singularity theory are applied to derive analytical results for the behaviour in scale-space of critical points and related entities. The main results of the treatment comprise


• a description of the general nature of trajectories of critical points in scale-space


• estimates of the drift velocity of critical points and straight edges


• an analysis of the qualitative behaviour of critical points in bifurcation situations


• a classification of what types of blod events are possible

/pdf/scale-space-behaviour-of-local-extrema-and-blobs-4w9mw8ren4.pdf

Scale-space behaviour of local extrema and blobs

Scale-space theoryA theory of multi-scale representation of sensory data developed by the image processing and computer vision communities. The purpose is to represent signals at multiple scales in such a way that fine scale structures are successively suppressed, and a scale parameter is associated with each level in the multi-scale representation.For a given signal , a linear scale-space representation is a family of derived signals , defined by andfor some family of convolution kernels [a1], [a2] (cf. also Integral equation of convolution type). An essential requirement on the scale-space family is that the representation at a coarse scale constitutes a simplification of the representations at finer scales. Several different ways of formalizing this requirement about non-creation of new structures with increasing scales show that the Gaussian kernelconstitutes a canonical choice for generating a scale-space representation [a3], [a4], [a5], [a6]. Equivalently, the scale-space family satisfies the diffusion equationThe motivation for generating a scale-space representation of a given data set originates from the basic fact that real-world objects are composed of different structures at different scales and may appear in different ways depending on the scale of observation. For example, the concept of a "tree" is appropriate at the scale of meters, while concepts such as leaves and molecules are more appropriate at finer scales. For a machine vision system analyzing an unknown scene, there is no way to know what scales are appropriate for describing the data. Thus, the only reasonable approach is to consider descriptions at all scales simultaneously [a1], [a2].From the scale-space representation, at any level of scale one can define scale-space derivatives bywhere and constitute multi-index notation for the derivative operator . Such Gaussian derivative operators provide a compact way to characterize the local image structure around a certain image point at any scale. Specifically, the output from scale-space derivatives can be combined into multi-scale differential invariants, to serve as feature detectors (see Edge detection and Corner detection for two examples).More generally, a scale-space representation with its Gaussian derivative operators can serve as a basis for expressing a large number of early visual operations, including feature detection, stereo matching, computation of motion descriptors and the computation of cues to surface shape [a3], [a4]. Neuro-physiological studies have shown that there are receptive field profiles in the mammalian retina and visual cortex, which can be well modeled by the scale-space framework [a7].Pyramid representation [a8] is a predecessor to scale-space representation, constructed by simultaneously smoothing and subsampling a given signal. In this way, computationally highly efficient algorithms can be obtained. A problem noted with pyramid representations, however, is that it is usually algorithmically hard to relate structures at different scales, due to the discrete nature of the scale levels. In a scale-space representation, the existence of a continuous scale parameter makes it conceptually much easier to express this deep structure [a2]. For features defined as zero-crossings of differential invariants, the implicit function theorem (cf. Implicit function) directly defines trajectories across scales, and at those scales where a bifurcation occurs, the local behaviour can be modeled by singularity theory [a3], [a5].Extensions of linear scale-space theory concern the formulation of non-linear scale-space concepts more committed to specific purposes [a9]. There are strong relations between scale-space theory and wavelet theory (cf. also Wavelet analysis), although these two notions of multi-scale representation have been developed from slightly different premises.References[a1] A.P. Witkin, "Scale-space filtering" , Proc. 8th Internat. Joint Conf. Art. Intell. Karlsruhe, West Germany Aug. 1983 (1983) pp. 1019–1022[a2] J.J. Koenderink, "The structure of images" Biological Cybernetics , 50 (1984) pp. 363–370[a3] T. Lindeberg, "Scale-space theory in computer vision" , Kluwer Acad. Publ. (1994)[a4] L.M.J. Florack, "Image structure" , Kluwer Acad. Publ. (1997)[a5]J. Sporring, et al., "Gaussian scale-space theory" , Kluwer Acad. Publ. (1997)[a6] B.M ter Haar Romeny, et al., "Proc. First Internat. Conf. scale-space" , Lecture Notes Computer Science , 1252 , Springer (1997)[a7] R.A. Young, "The Gaussian derivative model for spatial vision: Retinal mechanisms" Spatial Vision , 2 (1987) pp. 273–293[a8] P.J. Burt, E.H. Adelson, "The Laplacian Pyramid as a Compact Image Code" IEEE Trans. Commun. , 9 : 4 (1983) pp. 532–540[a9] "Geometry-driven diffusion in computer vision" B.M ter Haar Romeny (ed.) , Kluwer Acad. Publ. (1994)

Scale-space theory

This paper presents an approach for simultaneous tracking and recognition of hierarchical object representations in terms of multiscale image features. A scale-invariant dissimilarity measure is proposed for comparing scale-space features at different positions and scales. Based on this measure, the likelihood of hierarchical, parameterized models can be evaluated in such a way that maximization of the measure over different models and their parameters allows for both model selection and parameter estimation. Then, within the framework of particle filtering, we consider the area of hand gesture analysis, and present a method for simultaneous tracking and recognition of hand models under variations in the position, orientation, size and posture of the hand. In this way, qualitative hand states and quantitative hand motions can be captured, and be used for controlling different types of computerised equipment.

/pdf/tracking-of-multi-state-hand-models-using-particle-filtering-3hxkdwz32q.pdf

Tracking of Multi-state Hand Models Using Particle Filtering and a Hierarchy of Multi-scale Image Features

Local scale information extracted from visual data in a bottom-up manner constitutes an important cue for a large number of visual tasks. This article presents a framework for how the computation of such scale descriptors can be performed in real time on a standard computer.

The proposed scale selection framework is expressed within a novel type of multi-scale representation, referred to as hybrid multi-scale representation, which aims at integrating and providing variable trade-offs between the relative advantages of pyramids and scale-space representation, in terms of computational efficiency and computational accuracy. Starting from binomial scale-space kernels of different widths, we describe a family pyramid representations, in which the regular pyramid concept and the regular scale-space representation constitute limiting cases. In particular, the steepness of the pyramid as well as the sampling density in the scale direction can be varied.

It is shown how the definition of γ-normalized derivative operators underlying the automatic scale selection mechanism can be transferred from a regular scale-space to a hybrid pyramid, and two alternative definitions are studied in detail, referred to as variance normalization and lp-normalization. The computational accuracy of these two schemes is evaluated, and it is shown how the choice of sub-sampling rate provides a trade-off between the computational efficiency and the accuracy of the scale descriptors. Experimental evaluations are presented for both synthetic and real data. In a simplified form, this scale selection mechanism has been running for two years, in a real-time computer vision system.

/pdf/real-time-scale-selection-in-hybrid-multi-scale-49lfexqjcg.pdf

Tony Lindeberg

Papers

Generalized Gaussian scale-space axiomatics comprising linear scale-space, affine scale-space and sp

Scale-space behaviour of local extrema and blobs

Scale-space theory

Tracking of Multi-state Hand Models Using Particle Filtering and a Hierarchy of Multi-scale Image Features

Real-time scale selection in hybrid multi-scale representations