We have witnessed great interest and a wealth of promise in content-based image retrieval as an emerging technology. While the last decade laid foundation to such promise, it also paved the way for a large number of new techniques and systems, got many new people involved, and triggered stronger association of weakly related fields. In this article, we survey almost 300 key theoretical and empirical contributions in the current decade related to image retrieval and automatic image annotation, and in the process discuss the spawning of related subfields. We also discuss significant challenges involved in the adaptation of existing image retrieval techniques to build systems that can be useful in the real world. In retrospect of what has been achieved so far, we also conjecture what the future may hold for image retrieval research.

/pdf/image-retrieval-ideas-influences-and-trends-of-the-new-age-51whw4w7su.pdf

Image retrieval: Ideas, influences, and trends of the new age

IEEE Transactions on Pattern Analysis and Machine Intelligence

Computer and Robot Vision Vol. 1, by R.M. Haralick and Linda G. Shapiro, Addison-Wesley, 1992, ISBN 0-201-10887-1.

Computer and Robot Vision

While research on articulated human motion and pose estimation has progressed rapidly in the last few years, there has been no systematic quantitative evaluation of competing methods to establish the current state of the art. We present data obtained using a hardware system that is able to capture synchronized video and ground-truth 3D motion. The resulting HumanEva datasets contain multiple subjects performing a set of predefined actions with a number of repetitions. On the order of 40,000 frames of synchronized motion capture and multi-view video (resulting in over one quarter million image frames in total) were collected at 60 Hz with an additional 37,000 time instants of pure motion capture data. A standard set of error measures is defined for evaluating both 2D and 3D pose estimation and tracking algorithms. We also describe a baseline algorithm for 3D articulated tracking that uses a relatively standard Bayesian framework with optimization in the form of Sequential Importance Resampling and Annealed Particle Filtering. In the context of this baseline algorithm we explore a variety of likelihood functions, prior models of human motion and the effects of algorithm parameters. Our experiments suggest that image observation models and motion priors play important roles in performance, and that in a multi-view laboratory environment, where initialization is available, Bayesian filtering tends to perform well. The datasets and the software are made available to the research community. This infrastructure will support the development of new articulated motion and pose estimation algorithms, will provide a baseline for the evaluation and comparison of new methods, and will help establish the current state of the art in human pose estimation and tracking.

/pdf/humaneva-synchronized-video-and-motion-capture-dataset-and-4dx3ly40om.pdf

HumanEva: Synchronized Video and Motion Capture Dataset and Baseline Algorithm for Evaluation of Articulated Human Motion

In this paper the main problems and the available solutions are addressed for the generation of 3D models from terrestrial images. Close range photogrammetry has dealt for many years with manual or automatic image measurements for precise 3D modelling. Nowadays 3D scanners are also becoming a standard source for input data in many application areas, but image-based modelling still remains the most complete, economical, portable, flexible and widely used approach. In this paper the full pipeline is presented for 3D modelling from terrestrial image data, considering the different approaches and analysing all the steps involved.

/pdf/image-based-3d-modelling-a-review-5msqumyd5o.pdf

Image-based 3d modelling: a review

This paper presents a novel method for the recognition of ancient characters in historical documents. The method proposed is especially designed for degraded documents in which the character recognition based on state of the art methods is hard to achieve due to faded out ink, stain and background noise. The method proposed deals with such degradations by making use of the Dense SIFT features and the nearest neighbor distance maps. The maps encode the distances between the features of the documents and the training set. This results in local minima in the nearest neighbor distance maps which help in localization and recognition of characters in the documents. The experiments on three datasets show that the method proposed achieves a better character recognition performance compared to another method designed for similar historical documents.

Recognition of degraded ancient characters based on dense SIFT

The proposed binarization algorithm uses a scale space to avoid the estimation of script size dependent parameters. Due to the continous smoothing from finer to coarse scales, noise such as background clutter is suppressed since coarse scales characterize homogeneous regions of the image. Thus, coarser scales of the scale space can be used as a foreground estimation to apply a weigthing scheme robust against noise present in, for instance carbon copies or ancient and degraded documents. Additionally the information of filled regions is propagated through the scales. The use of integral images for the calculation of the mean, standard deviation and morphological operations allow for an efficient implementation of the method presented. The binarization of each scale is based on changes of the local intensity as proposed by Su et al.

Scale Space Binarization Using Edge Information Weighted by a Foreground Estimation

In this paper a document form classification and retrieval method using Bag of Words and newly introduced local shape features of form lines is proposed. In a preprocessing step the document is binarized and the form lines (solid and dotted) are detected. The shape features are based on the line information describing local line structures, e.g. line endings, crossings, boxes. The dominant line structures build a vocabulary for each form class. According to the vocabulary an occurrence histogram of structures of form documents can be calculated for the classification and retrieval. The proposed method has been tested on a set of 489 documents and 9 different form classes.

Form classification and retrieval using bag of words with shape features of line structures

This paper presents a hybrid approach towards self- localization of tiny autonomous mobile robots in a known but highly dynamic environment. The proposed algorithm is intended for two-wheeled differential drive robots which are equipped with a pivoted stereo vision system, two digital encoders, a gyro sensor, two 10g accelerometers and a magnetic compass. The global position of the robot can be estimated by extracting two distinct landmarks from the robot environment and measuring their range and orientation using the stereo vision system. However, distinct landmarks are not available through the entire state space and it is required to track the robot position once a global estimate is available. Tracking of the globally estimated position is performed within the framework of extended Kalman filter. Constant monitoring of the robot observation enables it to detect any unexpected situation. Simulation results show that robot can successfully localize itself at startup and is capable of detecting and recovering from localization failures.

A hybrid approach towards vision based self-localization of autonomous mobile robots

Pencil drawings like portraits or landscapes comprise dozens of strokes. The segmentation and identification of
individual strokes is an interesting question in analyzing the drawings since it allows art historians to analyze
the development of the stroke formations in the picture in more detail. In this study we are going to identify
individual strokes in stroke formations and to reconstruct the original drawing trace of the artist. The method
is based on a thinning algorithm and a following analysis of the accrued skeleton. In order to detect the original
stroke and the natural drawing trace we use the curvilinearity information of the thinned sub-strokes. A sub-stroke
runs from either a real end point to a crossing point, or between two crossing points. The selection of
corresponding strokes in crossing points is based on the angle at the end points of the sub-strokes. The individual
strokes drawn through are represented by a one pixel wide line which approximates the original drawing trace
of the artist by a cubic B-spline. The whole process is parameter free: we use the automatic calculated stroke
width for the skeleton pruning process, for the calculation of the angles at the sub-stroke endings and as the
distance for the spline control points.

Robert Sablatnig

Papers

Recognition of degraded ancient characters based on dense SIFT

Scale Space Binarization Using Edge Information Weighted by a Foreground Estimation

Form classification and retrieval using bag of words with shape features of line structures

A hybrid approach towards vision based self-localization of autonomous mobile robots

Estimating the original drawing trace of painted strokes