We have witnessed great interest and a wealth of promise in content-based image retrieval as an emerging technology. While the last decade laid foundation to such promise, it also paved the way for a large number of new techniques and systems, got many new people involved, and triggered stronger association of weakly related fields. In this article, we survey almost 300 key theoretical and empirical contributions in the current decade related to image retrieval and automatic image annotation, and in the process discuss the spawning of related subfields. We also discuss significant challenges involved in the adaptation of existing image retrieval techniques to build systems that can be useful in the real world. In retrospect of what has been achieved so far, we also conjecture what the future may hold for image retrieval research.

/pdf/image-retrieval-ideas-influences-and-trends-of-the-new-age-51whw4w7su.pdf

Image retrieval: Ideas, influences, and trends of the new age

IEEE Transactions on Pattern Analysis and Machine Intelligence

Computer and Robot Vision Vol. 1, by R.M. Haralick and Linda G. Shapiro, Addison-Wesley, 1992, ISBN 0-201-10887-1.

Computer and Robot Vision

While research on articulated human motion and pose estimation has progressed rapidly in the last few years, there has been no systematic quantitative evaluation of competing methods to establish the current state of the art. We present data obtained using a hardware system that is able to capture synchronized video and ground-truth 3D motion. The resulting HumanEva datasets contain multiple subjects performing a set of predefined actions with a number of repetitions. On the order of 40,000 frames of synchronized motion capture and multi-view video (resulting in over one quarter million image frames in total) were collected at 60 Hz with an additional 37,000 time instants of pure motion capture data. A standard set of error measures is defined for evaluating both 2D and 3D pose estimation and tracking algorithms. We also describe a baseline algorithm for 3D articulated tracking that uses a relatively standard Bayesian framework with optimization in the form of Sequential Importance Resampling and Annealed Particle Filtering. In the context of this baseline algorithm we explore a variety of likelihood functions, prior models of human motion and the effects of algorithm parameters. Our experiments suggest that image observation models and motion priors play important roles in performance, and that in a multi-view laboratory environment, where initialization is available, Bayesian filtering tends to perform well. The datasets and the software are made available to the research community. This infrastructure will support the development of new articulated motion and pose estimation algorithms, will provide a baseline for the evaluation and comparison of new methods, and will help establish the current state of the art in human pose estimation and tracking.

/pdf/humaneva-synchronized-video-and-motion-capture-dataset-and-4dx3ly40om.pdf

HumanEva: Synchronized Video and Motion Capture Dataset and Baseline Algorithm for Evaluation of Articulated Human Motion

In this paper the main problems and the available solutions are addressed for the generation of 3D models from terrestrial images. Close range photogrammetry has dealt for many years with manual or automatic image measurements for precise 3D modelling. Nowadays 3D scanners are also becoming a standard source for input data in many application areas, but image-based modelling still remains the most complete, economical, portable, flexible and widely used approach. In this paper the full pipeline is presented for 3D modelling from terrestrial image data, considering the different approaches and analysing all the steps involved.

/pdf/image-based-3d-modelling-a-review-5msqumyd5o.pdf

Image-based 3d modelling: a review

In diesem Beitrag wird ein System zur automatischen Eichung von Wasserzahlern beschrieben. Das Hauptaugenmerk wird dabei auf die automatische Ablesung des analog angezeigten Meswertes gelegt, wobei eine korrekte Ablesung nur dann erfolgen kann, wenn die Position der Skalen, die Winkel der einzelnen Zeiger sowie die globale Orientierung des Mesgerates bekannt sind. Daher beinhaltet eine automatische Ablesung die Detektion der im Bild vorhandenen Elemente eines Wasserzahlers wie Skalen, Zeiger und Beschrifiungselemente durch Mustererkennungsalgorithmen und die anschliesende Bestimmung der geometrischen Relationen zwischen den Elementen. Die Entwicklung des Systems zur automatischen Ablesung beginnt bei der Definition der Primitiva eines Instrumentes und deren geometrische Relationen, fohrt uber die Erprobung von Detektionsalgorithmen zum Entwurf einer Auswerteabfolge, die anschliesend durch industrielle Anforderung zu einem Zielsystem adaptiert wird. Erfolgreiche Labor- und Feldtests des Systems beenden den Systementwurf.

Automatische Ablesung von Wasserzählern zur Qualitätssicherung bei der Eichung

In most implementations of 3D reconstruction, depth information is provided by RGB-D sensors recording RGB and depth for each pixel. However, these sensors are still considerably expensive, characterized by high consumption of resources on the hardware itself, and have failed to reach mass markets of everyday mobile devices. With this paper, we aim to demonstrate the possibility of leveraging devices lacking 3D sensory data, in combination with external processing, to enrich the AR experience. We propose a client-server architecture that, in combination with a mobile client, allows for scanning of indoor environments as 3D semantically labeled sceneries while providing the user with instant feedback about the scanning results. At its core, the system is composed of the following expendable and exchangeable modules: (1) Depth Prediction from 2D images (2) Semantic Instance Segmentation of 2D images and (3) 3D Projection and Reconstruction. The result is a continuously updated mesh of the scenery with instance level segmentations in 3D. In comparison to the state-of-the-art, we are not only independent of the RGB-D input, but offer an architecture that complements and enhances the current AR client frameworks while demanding little extra computation from the mobile device.

In-Time 3D Reconstruction and Instance Segmentation from Monocular Sensor Data

Manuscript analysis has long been solely the domain of scientists in the humanities who had to cope with their complex tasks without the aid of specialized tools and result management facilities. In a conjoint philological and computational approach, we have created a toolbox for semi-automated manuscript analysis. Using high resolution digital images, it combines tools for the codicological (and in the case of documents, diplomatic) and paleographical investigation of manuscripts with sophisticated state of the art image processing methods. The results of the individual tools interact with one another and can be stored as images, metadata, and database entries for statistical processing. This interaction produces valuable results for the field of graphemics, the psycholinguistic comparison of human versus machine analysis, computer aided script recognition, and the reconstruction of damaged source material.

/pdf/a-toolbox-for-manuscript-analysis-4xto7ohvut.pdf

A Toolbox for Manuscript Analysis

Georeferencing of multitemporal aerial imagery is a time-consuming and challenging task that typically requires a high degree of human intervention, and which appears in application domains of critical importance, like unexploded ordnance detection. In order to make a semi-automatic scenario possible, we introduce a Visual Analytics approach for multitemporal aerial image georeferencing designed in close collaboration with real-world analysts that face the problem on a daily basis, and implemented by combining computer vision and interactive visual exploration methods. We report on informal validation findings resulting from the integration of our solution into our users’ GIS platform of choice, which positively illustrate its effectiveness and time-saving potential.

Visual Analytics for Multitemporal Aerial Image Georeferencing

Parts of collections of libraries, archives, and e.g. museums can still be uncatalogued. Even if metadata is provided, only standardized information of the described resources (dependent on the collection) is available, i.e., creator names, titles, and subject terms, limiting the search options for experts and typical users. In the case of image-based collections, the information of the image itself can be used as an additional feature to extend the search capabilities of the user. This paper analyzes the use of standard computer vision methods to explore the Vienna City Library poster collection using additional image-based properties. The proposed exploration tool allows a search based on the provided metadata and features based on face detection and retrieval, image retrieval, object detection, text recognition, and main color similarity. The OpenSearch engine is used to index the metadata and visual features, allowing for a real-time search of extensive collections. The qualitative and quantitative analysis shows the potential of visual features within a search tool.

Robert Sablatnig

Papers

Automatische Ablesung von Wasserzählern zur Qualitätssicherung bei der Eichung

In-Time 3D Reconstruction and Instance Segmentation from Monocular Sensor Data

A Toolbox for Manuscript Analysis

Visual Analytics for Multitemporal Aerial Image Georeferencing

Exploration of the Vienna City Library Poster Collection using Computer Vision Approaches