We have witnessed great interest and a wealth of promise in content-based image retrieval as an emerging technology. While the last decade laid foundation to such promise, it also paved the way for a large number of new techniques and systems, got many new people involved, and triggered stronger association of weakly related fields. In this article, we survey almost 300 key theoretical and empirical contributions in the current decade related to image retrieval and automatic image annotation, and in the process discuss the spawning of related subfields. We also discuss significant challenges involved in the adaptation of existing image retrieval techniques to build systems that can be useful in the real world. In retrospect of what has been achieved so far, we also conjecture what the future may hold for image retrieval research.

/pdf/image-retrieval-ideas-influences-and-trends-of-the-new-age-51whw4w7su.pdf

Image retrieval: Ideas, influences, and trends of the new age

IEEE Transactions on Pattern Analysis and Machine Intelligence

Computer and Robot Vision Vol. 1, by R.M. Haralick and Linda G. Shapiro, Addison-Wesley, 1992, ISBN 0-201-10887-1.

Computer and Robot Vision

While research on articulated human motion and pose estimation has progressed rapidly in the last few years, there has been no systematic quantitative evaluation of competing methods to establish the current state of the art. We present data obtained using a hardware system that is able to capture synchronized video and ground-truth 3D motion. The resulting HumanEva datasets contain multiple subjects performing a set of predefined actions with a number of repetitions. On the order of 40,000 frames of synchronized motion capture and multi-view video (resulting in over one quarter million image frames in total) were collected at 60 Hz with an additional 37,000 time instants of pure motion capture data. A standard set of error measures is defined for evaluating both 2D and 3D pose estimation and tracking algorithms. We also describe a baseline algorithm for 3D articulated tracking that uses a relatively standard Bayesian framework with optimization in the form of Sequential Importance Resampling and Annealed Particle Filtering. In the context of this baseline algorithm we explore a variety of likelihood functions, prior models of human motion and the effects of algorithm parameters. Our experiments suggest that image observation models and motion priors play important roles in performance, and that in a multi-view laboratory environment, where initialization is available, Bayesian filtering tends to perform well. The datasets and the software are made available to the research community. This infrastructure will support the development of new articulated motion and pose estimation algorithms, will provide a baseline for the evaluation and comparison of new methods, and will help establish the current state of the art in human pose estimation and tracking.

/pdf/humaneva-synchronized-video-and-motion-capture-dataset-and-4dx3ly40om.pdf

HumanEva: Synchronized Video and Motion Capture Dataset and Baseline Algorithm for Evaluation of Articulated Human Motion

In this paper the main problems and the available solutions are addressed for the generation of 3D models from terrestrial images. Close range photogrammetry has dealt for many years with manual or automatic image measurements for precise 3D modelling. Nowadays 3D scanners are also becoming a standard source for input data in many application areas, but image-based modelling still remains the most complete, economical, portable, flexible and widely used approach. In this paper the full pipeline is presented for 3D modelling from terrestrial image data, considering the different approaches and analysing all the steps involved.

/pdf/image-based-3d-modelling-a-review-5msqumyd5o.pdf

Image-based 3d modelling: a review

An algorithm for the automatic segmentation of strokes in underdrawings - the basic concept of the artist - in ancient panel paintings is presented. The purpose of the stroke analysis is the determination of the drawing tool used to draft the painting. This information allows significant support for a systematic stylistic approach in the analysis of paintings. Up to now, this analysis has been made by naked eye examination only, and the restricted human optical retentiveness complicated the comparison of different underdrawings with respect to drawing tools and stroke characteristics. Stroke segmentation in painting is related to the extraction and recognition of handwritings, therefore similar techniques to segment the strokes from the background incorporating boundary information are used. Following the segmentation, the approximation of the stroke boundary by a closed polygon done based on active contours. Results of the algorithms developed are presented for both test panels and real reflectograms.

/pdf/stroke-segmentation-in-infrared-reflectograms-3nxpg0bg2v.pdf

Stroke segmentation in infrared reflectograms

The activation of ancient vessel-fragments is a time consuming but important task for archaeologists. The basis for classification and reconstruction is the profile which is the cross-section of the fragment in the direction of the rotational axis of symmetry. Hence the position of a fragment (orientation) on a vessel is important. In this work the estimation of the axis of rotation out of range data by using a Hough inspired method is proposed. In order to avoid outliers a robust method for estimation of the axis is used. Classification and reconstruction are performed in a bottom-up manner using a description language, which holds all features of the fragment as primitives and all properties among features as relations. Classification of newly found fragments of unknown type is performed by comparing the description of the new fragment with the description of already classified fragments by completing graph similarity. The sub-graph with the highest similarity is then used to reconstruct the complete vessel out of the fragment.

On estimating the position of fragments on rotational symmetric pottery

Archaeology is at a point where it can benefit greatly from the application of computer vision methods, and in turn provides a large number of new, challenging and interesting conceptual problems and data for computer science. This is true in particular in the study of ceramics - the most abundant and widespread of all archaeological finds. The traditional way of documenting archaeological sherds is to draw the profile line, which is the intersection of a sherd along the axis of symmetry. A profilograph is a mechanical device, which can directly acquire and transfer a profile line by pin-pointing the profile on a sherd to a computer. We developed a fully automated vision system, which is able to compute the profile line out of the acquired 3D model of the fragment. In this paper we want to give a thorough comparison between the traditional manual approach, the profilograph and our system and present an improvement of the robustness of our approach by finding circular rills on the fragments. Practical experiments have been undertaken at the excavation Tel Dor in Israel.

Investigation on traditional and modern ceramic documentation

This work is concerned with the digitization and analysis of historical documents. The investigation of the documents has been conducted in three successive interdisciplinary projects. The team involved in the projects consists of philologists, chemists and computer scientists specialized in the field of digital image processing. The manuscripts investigated are partially degraded since they have been infected by mold, are corrupted by background clutter or contain faded-out or even erased writings. Since these degradations impede a transcription by scholars and worsen the performance of automated document image analysis techniques, the documents have been imaged with a portable multispectral imaging system. By using this non-invasive investigation technique, the contrast of the faded out characters can be increased, compared to ordinary white light illumination. Post-processing techniques, such as dimension reduction tools, can be used to gain a further legibility increase. The resulting images are used as a basis for further document analysis methods. These methods have been especially designed for the historical documents investigated and involve Optical Character Recognition and writer identification. This paper presents an overview on selected methods that have been developed in the projects.

Investigation of Ancient Manuscripts based on Multispectral Imaging

Document analysis is done to analyze entire forms (eg intelligent form analysis, table detection) or to describe
the layout/structure of a document Also skew detection of scanned documents is performed to support OCR
algorithms that are sensitive to skew In this paper document analysis is applied to snippets of torn documents to
calculate features for the reconstruction Documents can either be destroyed by the intention to make the printed
content unavailable (eg tax fraud investigation, business crime) or due to time induced degeneration of ancient
documents (eg bad storage conditions) Current reconstruction methods for manually torn documents deal
with the shape, inpainting and texture synthesis techniques In this paper the possibility of document analysis
techniques of snippets to support the matching algorithm by considering additional features are shown This
implies a rotational analysis, a color analysis and a line detection As a future work it is planned to extend the
feature set with the paper type (blank, checked, lined), the type of the writing (handwritten vs machine printed)
and the text layout of a snippet (text size, line spacing) Preliminary results show that these pre-processing steps
can be performed reliably on a real dataset consisting of 690 snippets

Robert Sablatnig

Papers

Stroke segmentation in infrared reflectograms

On estimating the position of fragments on rotational symmetric pottery

Investigation on traditional and modern ceramic documentation

Investigation of Ancient Manuscripts based on Multispectral Imaging

Document reconstruction by layout analysis of snippets