In this article, we present the Yale-Carnegie Mellon University (CMU)-Berkeley (YCB) object and model set, intended to be used to facilitate benchmarking in robotic manipulation research. The objects in the set are designed to cover a wide range of aspects of the manipulation problem. The set includes objects of daily life with different shapes, sizes, textures, weights, and rigidities as well as some widely used manipulation tests. The associated database provides high-resolution red, green, blue, plus depth (RGB-D) scans, physical properties, and geometric models of the objects for easy incorporation into manipulation and planning software platforms. In addition to describing the objects and models in the set along with how they were chosen and derived, we provide a framework and a number of example task protocols, laying out how the set can be used to quantitatively evaluate a range of manipulation approaches, including planning, learning, mechanical design, control, and many others. A comprehensive literature survey on the existing benchmarks and object data sets is also presented, and their scope and limitations are discussed. The YCB set will be freely distributed to research groups worldwide at a series of tutorials at robotics conferences. Subsequent sets will be, otherwise, available to purchase at a reasonable cost. It is our hope that the ready availability of this set along with the ground laid in terms of protocol templates will enable the community of manipulation researchers to more easily compare approaches as well as continually evolve standardized benchmarking tests and metrics as the field matures.

Benchmarking in Manipulation Research: Using the Yale-CMU-Berkeley Object and Model Set

The recent introduction of 3D shape analysis frameworks able to quantify the deformation of a shape into another in terms of the variation of real functions yields a new interpretation of the 3D shape similarity assessment and opens new perspectives. Indeed, while the classical approaches to similarity mainly quantify it as a numerical score, map-based methods also define dense shape correspondences. After presenting in detail the theoretical foundations underlying these approaches, we classify them by looking at their most salient features, including the kind of structure and invariance properties they capture, as well as the distances and the output modalities according to which the similarity between shapes is assessed and returned. We also review the usage of these methods in a number of 3D shape application domains, ranging from matching and retrieval to annotation and segmentation. Finally, the most promising directions for future research developments are discussed.

Recent Trends, Applications, and Perspectives in 3D Shape Similarity Assessment

https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=917883

A comparison of 3D shape retrieval methods based on a large-scale benchmark supporting multimodal queries

Advances in image sensors and evolution of digital computation is a strong stimulus for development and implementation of sophisticated methods for capturing, processing and analysis of 3D data from dynamic scenes. Research on perspective time-varying 3D scene capture technologies is important for the upcoming 3DTV displays. Methods such as shape-from-texture, shape-from-shading, shape-from-focus, and shape-from-motion extraction can restore 3D shape information from a single camera data. The existing techniques for 3D extraction from single-camera video sequences are especially useful for conversion of the already available vast mono-view content to the 3DTV systems. Scene-oriented single-camera methods such as human face reconstruction and facial motion analysis, body modeling and body motion tracking, and motion recognition solve efficiently a variety of tasks. 3D multicamera dynamic acquisition and reconstruction, their hardware specifics including calibration and synchronization and software demands form another area of intensive research. Different classes of multiview stereo algorithms such as those based on cost function computing and optimization, fusing of multiple views, and feature-point reconstruction are possible candidates for dynamic 3D reconstruction. High-resolution digital holography and pattern projection techniques such as coded light or fringe projection for real-time extraction of 3D object positions and color information could manifest themselves as an alternative to traditional camera-based methods. Apart from all of these approaches, there also are some active imaging devices capable of 3D extraction such as the 3D time-of-flight camera, which provides 3D image data of its environment by means of a modulated infrared light source.

3-D Time-Varying Scene Capture Technologies—A Survey

Unsupervised word translation from non-parallel inter-lingual corpora has attracted much research interest. Very recently, neural network methods trained with adversarial loss functions achieved high accuracy on this task. Despite the impressive success of the recent techniques, they suffer from the typical drawbacks of generative adversarial models: sensitivity to hyper-parameters, long training time and lack of interpretability. In this paper, we make the observation that two sufficiently similar distributions can be aligned correctly with iterative matching methods. We present a novel method that first aligns the second moment of the word distributions of the two languages and then iteratively refines the alignment. Extensive experiments on word translation of European and Non-European languages show that our method achieves better performance than recent state-of-the-art deep adversarial approaches and is competitive with the supervised baseline. It is also efficient, easy to parallelize on CPU and interpretable.

/pdf/non-adversarial-unsupervised-word-translation-4j95kjrd4k.pdf

Non-Adversarial Unsupervised Word Translation

The availability of semantically annotated image and video assets constitutes a critical prerequisite for the realisation of intelligent knowledge management services pertaining to realistic user needs. Given the extend of the challenges involved in the automatic extraction of such descriptions, manually created metadata play a significant role, further strengthened by their deployment in training and evaluation tasks related to the automatic extraction of content descriptions. The different views taken by the two main approaches towards semantic content description, namely the Semantic Web and MPEG-7, as well as the traits particular to multimedia content due to the multiplicity of information levels involved, have resulted in a variety of image and video annotation tools, adopting varying description aspects. Aiming to provide a common framework of reference and furthermore to highlight open issues, especially with respect to the coverage and the interoperability of the produced metadata, in this chapter we present an overview of the state of the art in image and video annotation tools.

/pdf/a-survey-of-semantic-image-and-video-annotation-tools-10gkrpgxvq.pdf

A survey of semantic image and video annotation tools

In this paper, a software-based system for the real-time synchronization of images captured by a low-cost camera framework is presented. It is most well suited for cases where special hardware cannot be utilized (e.g. remote or wireless applications) and when cost efficiency is critical. The proposed method utilizes messages to establish a consensus on the time of image acquisition and NTP synchronization of computer clocks. It also provides with an error signal, in case of failure of the synchronization. The evaluation of the proposed algorithm using a precise LED array system (1ms accuracy) proves the effectiveness of this method.

/pdf/synchronous-image-acquisition-based-on-network-28uaa4w3g8.pdf

Synchronous Image Acquisition based on Network Synchronization

This paper proposes a novel framework for 3-D object retrieval, taking into account most of the factors that may affect the retrieval performance. Initially, a novel 3-D model alignment method is introduced, which achieves accurate rotation estimation through the combination of two intuitive criteria, plane reflection symmetry and rectilinearity. After the pose normalization stage, a low-level descriptor extraction procedure follows, using three different types of descriptors, which have been proven to be effective. Then, a novel combination procedure of the above descriptors takes place, which achieves higher retrieval performance than each descriptor does separately. The paper provides also an in-depth study of the factors that can further improve the 3-D object retrieval accuracy. These include selection of the appropriate dissimilarity metric, feature selection/dimensionality reduction on the initial low-level descriptors, as well as manifold learning for re-ranking of the search results. Experiments performed on two 3-D model benchmark datasets confirm our assumption that future research in 3-D object retrieval should focus more on the efficient combination of low-level descriptors as well as on the selection of the best features and matching metrics, than on the investigation of the optimal 3-D object descriptor.

Investigating the Effects of Multiple Factors Towards More Accurate 3-D Object Retrieval

In this paper we present the results of the 3D Shape Retrieval Contest 2011 (SHREC'11) track on generic shape retrieval. The aim of this track is to evaluate the performance of 3D shape retrieval algorithms that can operate on arbitrary 3D models. The benchmark dataset consists of 1000 3D objects classified in 50 categories. The 3D models are mainly classified based on visual shape similarity and each class has equal number of models to reduce the possible bias in evaluation results. Two groups have participated in the track with six methods in total.

SHREC'11 track: generic shape retrieval

In this paper, a novel framework for 3D object retrieval is presented. The paper focuses on the investigation of an accurate 3D model alignment method, which is achieved by combining two intuitive criteria, the plane reflection symmetry and rectilinearity. After proper positioning in a coordinate system, a set of 2D images (multi-views) are automatically generated from the 3D object, by taking views from uniformly distributed viewpoints. For each image, a set of flip-invariant shape descriptors is extracted. Taking advantage of both the pose estimation of the 3D objects and the flip-invariance property of the extracted descriptors, a new matching scheme for fast computation of 3D object dissimilarity is introduced. Experiments conducted in SHREC 2009 benchmark show the superiority of the pose estimation method over similar approaches, as well as the efficiency of the new matching scheme.

Georgios Litos

Papers

A survey of semantic image and video annotation tools

Synchronous Image Acquisition based on Network Synchronization

Investigating the Effects of Multiple Factors Towards More Accurate 3-D Object Retrieval

SHREC'11 track: generic shape retrieval

3D model retrieval using accurate pose estimation and view-based similarity