We have witnessed great interest and a wealth of promise in content-based image retrieval as an emerging technology. While the last decade laid foundation to such promise, it also paved the way for a large number of new techniques and systems, got many new people involved, and triggered stronger association of weakly related fields. In this article, we survey almost 300 key theoretical and empirical contributions in the current decade related to image retrieval and automatic image annotation, and in the process discuss the spawning of related subfields. We also discuss significant challenges involved in the adaptation of existing image retrieval techniques to build systems that can be useful in the real world. In retrospect of what has been achieved so far, we also conjecture what the future may hold for image retrieval research.

/pdf/image-retrieval-ideas-influences-and-trends-of-the-new-age-51whw4w7su.pdf

Image retrieval: Ideas, influences, and trends of the new age

IEEE Transactions on Pattern Analysis and Machine Intelligence

Computer and Robot Vision Vol. 1, by R.M. Haralick and Linda G. Shapiro, Addison-Wesley, 1992, ISBN 0-201-10887-1.

Computer and Robot Vision

While research on articulated human motion and pose estimation has progressed rapidly in the last few years, there has been no systematic quantitative evaluation of competing methods to establish the current state of the art. We present data obtained using a hardware system that is able to capture synchronized video and ground-truth 3D motion. The resulting HumanEva datasets contain multiple subjects performing a set of predefined actions with a number of repetitions. On the order of 40,000 frames of synchronized motion capture and multi-view video (resulting in over one quarter million image frames in total) were collected at 60 Hz with an additional 37,000 time instants of pure motion capture data. A standard set of error measures is defined for evaluating both 2D and 3D pose estimation and tracking algorithms. We also describe a baseline algorithm for 3D articulated tracking that uses a relatively standard Bayesian framework with optimization in the form of Sequential Importance Resampling and Annealed Particle Filtering. In the context of this baseline algorithm we explore a variety of likelihood functions, prior models of human motion and the effects of algorithm parameters. Our experiments suggest that image observation models and motion priors play important roles in performance, and that in a multi-view laboratory environment, where initialization is available, Bayesian filtering tends to perform well. The datasets and the software are made available to the research community. This infrastructure will support the development of new articulated motion and pose estimation algorithms, will provide a baseline for the evaluation and comparison of new methods, and will help establish the current state of the art in human pose estimation and tracking.

/pdf/humaneva-synchronized-video-and-motion-capture-dataset-and-4dx3ly40om.pdf

HumanEva: Synchronized Video and Motion Capture Dataset and Baseline Algorithm for Evaluation of Articulated Human Motion

In this paper the main problems and the available solutions are addressed for the generation of 3D models from terrestrial images. Close range photogrammetry has dealt for many years with manual or automatic image measurements for precise 3D modelling. Nowadays 3D scanners are also becoming a standard source for input data in many application areas, but image-based modelling still remains the most complete, economical, portable, flexible and widely used approach. In this paper the full pipeline is presented for 3D modelling from terrestrial image data, considering the different approaches and analysing all the steps involved.

/pdf/image-based-3d-modelling-a-review-5msqumyd5o.pdf

Image-based 3d modelling: a review

In this paper a public database for writer retrieval, writer identification and word spotting is presented. The CVL-Database consists of 7 different handwritten texts (1 German and 6 English Texts) and 311 different writers. For each text an RGB color image (300 dpi) comprising the handwritten text and the printed text sample are available as well as a cropped version (only handwritten). A unique ID identifies the writer, whereas the bounding boxes for each single word are stored in an XML file. An evaluation of the best algorithms of the ICDAR and ICHFR writer identification contest has been performed on the CVL-database.

CVL-DataBase: An Off-Line Database for Writer Retrieval, Writer Identification and Word Spotting

With the recent evolution of technology, the number of image archives has increased exponentially. In Content-Based Image Retrieval (CBIR), high-level visual information is represented in the form of low-level features. The semantic gap between the low-level features and the high-level image concepts is an open research problem. In this paper, we present a novel visual words integration of Scale Invariant Feature Transform (SIFT) and Speeded-Up Robust Features (SURF). The two local features representations are selected for image retrieval because SIFT is more robust to the change in scale and rotation, while SURF is robust to changes in illumination. The visual words integration of SIFT and SURF adds the robustness of both features to image retrieval. The qualitative and quantitative comparisons conducted on Corel-1000, Corel-1500, Corel-2000, Oliva and Torralba and Ground Truth image benchmarks demonstrate the effectiveness of the proposed visual words integration.

/pdf/a-novel-image-retrieval-based-on-visual-words-integration-of-4nkbztiyoi.pdf

A Novel Image Retrieval Based on Visual Words Integration of SIFT and SURF.

In this paper a novel method for writer identification and retrieval is presented. Writer identification is the process of finding the author of a specific document by comparing it to documents in a database where writers are known, whereas retrieval is the task of finding similar handwritings or all documents of a specific writer. The method presented is using Convolutional Neural Networks CNN to generate a feature vector for each writer, which is then compared with the precalculated feature vectors stored in the database. For the generation of this vector the CNN is trained on a database with known writers and after training the classification layer is cut off and the output of the second last fully connected layer is used as feature vector. For the identification a nearest neighbor classification is used. The evaluation is performed on the ICDAR2013 Competition on Writer Identification, ICDAR 2011 Writer Identification Contest, and the CVL-Database datasets. Experiments show, that this novel approach achieves better results to previously presented writer identification approaches.

Writer Identification and Retrieval Using a Convolutional Neural Network

In this paper a method for writer identification and writer retrieval is presented. Writer identification is the task of identifying the writer of a document out of a database of known writers. In contrast to identification, writer retrieval is the task of finding documents in a database according to the similarity of handwritings. The approach presented in this paper uses local features for this task. First a vocabulary is calculated by clustering features using a Gaussian Mixture Model and applying the Fisher kernel. For each document image the features are calculated and the Fisher Vector is generated using the vocabulary. The distance of this vector is then used as similarity measurement for the handwriting and can be used for writer identification and writer retrieval. The proposed method is evaluated on two datasets, namely the ICDAR 2011 Writer Identification Contest dataset which consists of 208 documents from 26 writers, and the CVL Database which contains 1539 documents from 309 writers. Experiments show that the proposed methods performs slightly better than previously presented writer identification approaches.

Writer Identification and Writer Retrieval Using the Fisher Vector on Visual Vocabularies

This paper introduces the 3D Measurement and Virtual Reconstruction of Ancient Lost Worlds of Europe system (3D MURALE). It consists of a set of tools for recording, reconstructing, encoding, visualising and database searching/querying that operate on buildings, building parts, statues, statue parts, pottery, stratigraphy, terrain geometry and texture and material texture. The tools are loosely linked together by a common database on which they all have the facility to store and access data. The paper describes the overall architecture of the 3D MURALE system and then briefly describes the functionality of the tools provided by the project. The paper compares the multimedia studio architecture adopted in this project with other multimedia studio architectures.

/pdf/3d-murale-a-multimedia-system-for-archaeology-3vynrq3ny9.pdf

Robert Sablatnig

Papers

CVL-DataBase: An Off-Line Database for Writer Retrieval, Writer Identification and Word Spotting

A Novel Image Retrieval Based on Visual Words Integration of SIFT and SURF.

Writer Identification and Retrieval Using a Convolutional Neural Network

Writer Identification and Writer Retrieval Using the Fisher Vector on Visual Vocabularies

3D MURALE: a multimedia system for archaeology