“Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告

A novel multi-modality image fusion method based on image decomposition and sparse representation

A deep learning architecture is proposed to predict graspable locations for robotic manipulation. It considers situations where no, one, or multiple object(s) are seen. By defining the learning problem to be classified with null hypothesis competition instead of regression, the deep neural network with red, green, blue and depth (RGB-D) image input predicts multiple grasp candidates for a single object or multiple objects, in a single shot. The method outperforms state-of-the-art approaches on the Cornell dataset with 96.0% and 96.1% accuracy on imagewise and object-wise splits, respectively. Evaluation on a multiobject dataset illustrates the generalization capability of the architecture. Grasping experiments achieve 96.0% grasp localization and 89.0% grasping success rates on a test set of household objects. The real-time process takes less than 0.25 s from image to plan.

Real-World Multiobject, Multigrasp Detection

Robotic tactile perception of object properties: A review

Unsupervised contrastive learning has achieved outstanding success, while the mechanism of contrastive loss has been less studied. In this paper, we concentrate on the understanding of the behaviours of unsupervised contrastive loss. We will show that the contrastive loss is a hardness-aware loss function, and the temperature {\tau} controls the strength of penalties on hard negative samples. The previous study has shown that uniformity is a key property of contrastive learning. We build relations between the uniformity and the temperature {\tau} . We will show that uniformity helps the contrastive learning to learn separable features, however excessive pursuit to the uniformity makes the contrastive loss not tolerant to semantically similar samples, which may break the underlying semantic structure and be harmful to the formation of features useful for downstream tasks. This is caused by the inherent defect of the instance discrimination objective. Specifically, instance discrimination objective tries to push all different instances apart, ignoring the underlying relations between samples. Pushing semantically consistent samples apart has no positive effect for acquiring a prior informative to general downstream tasks. A well-designed contrastive loss should have some extents of tolerance to the closeness of semantically similar samples. Therefore, we find that the contrastive loss meets a uniformity-tolerance dilemma, and a good choice of temperature can compromise these two properties properly to both learn separable features and tolerant to semantically similar samples, improving the feature qualities and the downstream performances.

Understanding the Behaviour of Contrastive Loss

Dexterous robots have emerged in the last decade in response to the need for fine-motor-control assistance in applications as diverse as surgery, undersea welding, and mechanical manipulation in space. Crucial to the fine operation and contact environmental perception are tactile sensors that are fixed on the robotic fingertips. These can be used to distinguish material texture, roughness, spatial features, compliance, and friction. In this paper, we regard the investigated tactile data as time sequences, of which dissimilarity can be evaluated by the popular dynamic time warping method. A kernel sparse coding method is therefore developed to address the tactile data representation and classification problem. However, the naive use of sparse coding neglects the intrinsic relation between individual fingers, which simultaneously contact the object. To tackle this problem, we develop a joint kernel sparse coding model to solve the multifinger tactile sequence classification problem. In this model, the intrinsic relations between fingers are explicitly taken into account using the joint sparse coding, which encourages all of the coding vectors to share the same sparsity support pattern. The experimental results show that the joint sparse coding achieves better performance than conventional sparse coding.

Object Recognition Using Tactile Measurements: Kernel Sparse Coding Methods

The robotic grasp detection is a great challenge in the area of robotics. Previous work mainly employs the visual approaches to solve this problem. In this paper, a hybrid deep architecture combining the visual and tactile sensing for robotic grasp detection is proposed. We have demonstrated that the visual sensing and tactile sensing are complementary to each other and important for the robotic grasping. A new THU grasp dataset has also been collected which contains the visual, tactile and grasp configuration information. The experiments conducted on a public grasp dataset and our collected dataset show that the performance of the proposed model is superior to state of the art methods. The results also indicate that the tactile data could help to enable the network to learn better visual features for the robotic grasp detection task.

A hybrid deep architecture for robotic grasp detection

Tactile sensors play very important role for robot perception in the dynamic or unknown environment. However, the tactile object recognition exhibits great challenges in practical scenarios. In this paper, we address this problem by developing an extreme kernel sparse learning methodology. This method combines the advantages of extreme learning machine and kernel sparse learning by simultaneously addressing the dictionary learning and the classifier design problems. Furthermore, to tackle the intrinsic difficulties which are introduced by the representer theorem , we develop a reduced kernel dictionary learning method by introducing row-sparsity constraint. A globally convergent algorithm is developed to solve the optimization problem and the theoretical proof is provided. Finally, we perform extensive experimental validations on some public available tactile sequence datasets and show the advantages of the proposed method.

Extreme Kernel Sparse Learning for Tactile Object Recognition

With the development of robotics, the application of robots has gradually evolved from industrial scenes to more intelligent service scenarios. For multitasking operations of robots in complex and uncertain environments, the traditional manual coding method is not only cumbersome but also unable to adapt to sudden changes in the environment. Imitation learning that avoids learning skills from scratch by using the expert demonstration has become the most effective way for robotic manipulation. The paper is intended to provide the survey of imitation learning of robotic manipulation and explore the future research trend. The review of the art of imitation learning for robotic manipulation involves three aspects that are demonstration, representation and learning algorithms. Towards the end of the paper, we highlight areas of future research potential.

Survey of imitation learning for robotic manipulation

The ever-growing development of sensor technology has led to the use of multimodal sensors to develop robotics and automation systems. It is therefore highly expected to develop methodologies capable of integrating information from multimodal sensors with the goal of improving the performance of surveillance, diagnosis, prediction, and so on. However, real multimodal data often suffer from significant weak-pairing characteristics, i.e., the full pairing between data samples may not be known, while pairing of a group of samples from one modality to a group of samples in another modality is known. In this paper, we establish a novel projective dictionary learning framework for weakly paired multimodal data fusion. By introducing a latent pairing matrix, we realize the simultaneous dictionary learning and the pairing matrix estimation, and therefore improve the fusion effect. In addition, the kernelized version and the optimization algorithms are also addressed. Extensive experimental validations on some existing data sets are performed to show the advantages of the proposed method. Note to Practitioners —In many industrial environments, we usually use multiple heterogeneous sensors, which provide multimodal information. Such multimodal data usually lead to two technical challenges. First, different sensors may provide different patterns of data. Second, the full-pairing information between modalities may not be known. In this paper, we develop a unified model to tackle such problems. This model is based on a projective dictionary learning method, which efficiently produces the representation vector for the original data by an explicit form. In addition, the latent pairing relation between samples can be learned automatically and be used to improve the classification performance. Such a method can be flexibly used for multimodal fusion with full-pairing, partial-pairing and weak-pairing cases.

Di Guo

Papers

Object Recognition Using Tactile Measurements: Kernel Sparse Coding Methods

A hybrid deep architecture for robotic grasp detection

Extreme Kernel Sparse Learning for Tactile Object Recognition

Survey of imitation learning for robotic manipulation

Weakly Paired Multimodal Fusion for Object Recognition