We have witnessed great interest and a wealth of promise in content-based image retrieval as an emerging technology. While the last decade laid foundation to such promise, it also paved the way for a large number of new techniques and systems, got many new people involved, and triggered stronger association of weakly related fields. In this article, we survey almost 300 key theoretical and empirical contributions in the current decade related to image retrieval and automatic image annotation, and in the process discuss the spawning of related subfields. We also discuss significant challenges involved in the adaptation of existing image retrieval techniques to build systems that can be useful in the real world. In retrospect of what has been achieved so far, we also conjecture what the future may hold for image retrieval research.

/pdf/image-retrieval-ideas-influences-and-trends-of-the-new-age-51whw4w7su.pdf

Image retrieval: Ideas, influences, and trends of the new age

IEEE Transactions on Pattern Analysis and Machine Intelligence

Computer and Robot Vision Vol. 1, by R.M. Haralick and Linda G. Shapiro, Addison-Wesley, 1992, ISBN 0-201-10887-1.

Computer and Robot Vision

While research on articulated human motion and pose estimation has progressed rapidly in the last few years, there has been no systematic quantitative evaluation of competing methods to establish the current state of the art. We present data obtained using a hardware system that is able to capture synchronized video and ground-truth 3D motion. The resulting HumanEva datasets contain multiple subjects performing a set of predefined actions with a number of repetitions. On the order of 40,000 frames of synchronized motion capture and multi-view video (resulting in over one quarter million image frames in total) were collected at 60 Hz with an additional 37,000 time instants of pure motion capture data. A standard set of error measures is defined for evaluating both 2D and 3D pose estimation and tracking algorithms. We also describe a baseline algorithm for 3D articulated tracking that uses a relatively standard Bayesian framework with optimization in the form of Sequential Importance Resampling and Annealed Particle Filtering. In the context of this baseline algorithm we explore a variety of likelihood functions, prior models of human motion and the effects of algorithm parameters. Our experiments suggest that image observation models and motion priors play important roles in performance, and that in a multi-view laboratory environment, where initialization is available, Bayesian filtering tends to perform well. The datasets and the software are made available to the research community. This infrastructure will support the development of new articulated motion and pose estimation algorithms, will provide a baseline for the evaluation and comparison of new methods, and will help establish the current state of the art in human pose estimation and tracking.

/pdf/humaneva-synchronized-video-and-motion-capture-dataset-and-4dx3ly40om.pdf

HumanEva: Synchronized Video and Motion Capture Dataset and Baseline Algorithm for Evaluation of Articulated Human Motion

In this paper the main problems and the available solutions are addressed for the generation of 3D models from terrestrial images. Close range photogrammetry has dealt for many years with manual or automatic image measurements for precise 3D modelling. Nowadays 3D scanners are also becoming a standard source for input data in many application areas, but image-based modelling still remains the most complete, economical, portable, flexible and widely used approach. In this paper the full pipeline is presented for 3D modelling from terrestrial image data, considering the different approaches and analysing all the steps involved.

/pdf/image-based-3d-modelling-a-review-5msqumyd5o.pdf

Image-based 3d modelling: a review

MultiSpectral (MS) imaging enriches document digitization by increasing the spectral resolution. We present a methodology which detects a target ink in document images by taking into account this additional information. The proposed method performs a rough foreground estimation to localize possible ink regions. Then, the Adaptive Coherence Estimator (ACE), a target detection algorithm, transforms the MS input space into a single gray-scale image where values close to one indicate ink. A spatial segmentation using GrabCut on the target detection's output is computed to create the final binary image. To find a baseline performance, the method is evaluated on the three most recent Document Image Binarization COntests (DIBCO) despite the fact that they only provide RGB images. In addition, an evaluation on three publicly available MS datasets is carried out. The presented methodology achieved the highest performance at the MultiSpectral Text Extraction (MS-TEx) contest 2015.

MSIO: MultiSpectral Document Image BinarizatIOn

This work is concerned with the legibility enhancement of ancient and degraded handwritings. The writings are partially barely visible under normal white light and hence they have been imaged with a MultiSpectral Imaging (MSI) system in order to increase their legibility. Dimension reduction techniques - like Principal Component Analysis (PCA) - can be used to further enhance the contrast of the faded-out characters. In this work the dimensionality of the multispectral scan is lowered, by applying Linear Discriminant Analysis (LDA). Since LDA is a supervised dimension reduction method, it is necessary to label a subset of the multispectral samples as belonging to the fore-or background. For this purpose, an approach is suggested that uses spatial information. The enhancement method is evaluated by Optical Character Recognition (OCR). By applying the enhancement method the OCR performance is increased in the case of degraded writings, compared to OCR results gained on unprocessed multispectral images and to OCR results achieved on images, which have been produced by applying unsupervised dimension reductions.

Improving OCR Accuracy by Applying Enhancement Techniques on Multispectral Images

Architectural elements are the components and details of buildings. Their unique set, combination, design, construction technique form the architectural style of buildings. Building facade classification by architectural styles is viewed as a task of classifying separate architectural structural elements. In the scope of building facade architectural style classification the current paper targets the problem of classification of Gothic and Baroque architectural elements called tracery, pediment and balustrade. Since certain gradient directions dominate on the shape of each architectural element, discrimination between dominating gradients means classification of architectural elements and thus architectural styles. We use local features to describe gradient directions. Our approach is based on clustering and learning of local features and yields a high classification rate.

Classification of gothic and baroque architectural elements

In this paper we propose a method for segmenting characters in multispectral images of ancient documents. Due to the low quality of the document images the main idea of our study is to combine the multispectral behavior and contextual spatial information. Therefore we utilize a Markov Random Field model using the spectral information of the images and stroke properties to include spatial dependencies of the characters. The whole process is parameter free since we calculate the stroke properties and the Gaussian parameters for the imaging model automatically. The study shows the effectiveness of using multispectral data for a computer aided analysis of ancient text documents. We compare the results of the proposed segmentation method to traditional methods based on color images as well as gray level images.

Multispectral imaging for analyzing ancient manuscripts

While single-molecule localization microscopy (SMLM) offers the invaluable prospect to visualize cellular structures below the diffraction limit of light microscopy, its potential has not yet been fully capitalized due to its inherent susceptibility to blinking artifacts. Particularly, overcounting of single molecule localizations has impeded a reliable and sensitive detection of biomolecular nanoclusters. Here we introduce a 2-Color Localization microscopy And Significance Testing Approach (2-CLASTA), providing a parameter-free statistical framework for the qualitative analysis of two-dimensional SMLM data via significance testing methods. 2-CLASTA yields p-values for the null hypothesis of random biomolecular distributions, independent of the blinking behavior of the chosen fluorescent labels. The method is parameter-free and does not require any additional measurements nor grouping of localizations. We validated the method both by computer simulations as well as experimentally, using protein concatemers as a mimicry of biomolecular clustering. As the new approach is not affected by overcounting artifacts, it is able to detect biomolecular clustering of various shapes at high sensitivity down to a level of dimers.

/pdf/verifying-molecular-clusters-by-2-color-localization-ygcg32jpb4.pdf

Robert Sablatnig

Papers

MSIO: MultiSpectral Document Image BinarizatIOn

Improving OCR Accuracy by Applying Enhancement Techniques on Multispectral Images

Classification of gothic and baroque architectural elements

Multispectral imaging for analyzing ancient manuscripts

Verifying molecular clusters by 2-color localization microscopy and significance testing.