Cognition-Enabled Robot Manipulation in Human Environments: Requirements, Recent Work, and Open Problems
TL;DR: Problem in different research areas related to mobile manipulation from the cognitive perspective are outlined, recently published works and the state-of-the-art approaches to address these problems are reviewed, and open problems to be solved are discussed.
Abstract: Service robots are expected to play an important role in our daily lives as our companions in home and work environments in the near future. An important requirement for fulfilling this expectation is to equip robots with skills to perform everyday manipulation tasks, the success of which is crucial for most home chores, such as cooking, cleaning, and shopping. Robots have been used successfully for manipulation tasks in wellstructured and controlled factory environments for decades. Designing skills for robots working in uncontrolled human environments raises many potential challenges in various subdisciplines, such as computer vision, automated planning, and human-robot interaction. In spite of the recent progress in these fields, there are still challenges to tackle. This article outlines problems in different research areas related to mobile manipulation from the cognitive perspective, reviews recently published works and the state-of-the-art approaches to address these problems, and discusses open problems to be solved to realize robot assistants that can be used in manipulation tasks in unstructured human environments.
Citations
More filters
Journal Article•
TL;DR: In this article, the authors propose that the brain produces an internal representation of the world, and the activation of this internal representation is assumed to give rise to the experience of seeing, but it leaves unexplained how the existence of such a detailed internal representation might produce visual consciousness.
Abstract: Many current neurophysiological, psychophysical, and psychological approaches to vision rest on the idea that when we see, the brain produces an internal representation of the world. The activation of this internal representation is assumed to give rise to the experience of seeing. The problem with this kind of approach is that it leaves unexplained how the existence of such a detailed internal representation might produce visual consciousness. An alternative proposal is made here. We propose that seeing is a way of acting. It is a particular way of exploring the environment. Activity in internal representations does not generate the experience of seeing. The outside world serves as its own, external, representation. The experience of seeing occurs when the organism masters what we call the governing laws of sensorimotor contingency. The advantage of this approach is that it provides a natural and principled way of accounting for visual consciousness, and for the differences in the perceived quality of sensory experience in the different sensory modalities. Several lines of empirical evidence are brought forward in support of the theory, in particular: evidence from experiments in sensorimotor adaptation, visual \"filling in,\" visual stability despite eye movements, change blindness, sensory substitution, and color perception.
2,271 citations
TL;DR: A comprehensive review of EMG-based motor intention prediction of continuous human upper limb motion, which will cover the models and approaches used in continuous motion estimation, the kinematic motion parameters estimated from EMG signal, and the performance metrics utilized for system validation.
Abstract: Electromyography (EMG) signal is one of the widely used biological signals for human motor intention prediction, which is an essential element in human-robot collaboration systems. Studies on motor intention prediction from EMG signal have been concentrated on classification and regression models, and there are numerous review and survey papers on classification models. However, to the best of our knowledge, there is no review paper on regression models or continuous motion prediction from EMG signal. Therefore, in this paper, we provide a comprehensive review of EMG-based motor intention prediction of continuous human upper limb motion. This review will cover the models and approaches used in continuous motion estimation, the kinematic motion parameters estimated from EMG signal, and the performance metrics utilized for system validation. From the review, we will provide some insights into future research directions on these subjects. We first review the overall structure and components of EMG-based human-robot collaboration systems. We then discuss the state of arts in continuous motion prediction of the human upper limb. Finally, we conclude the paper with a discussion of the current challenges and future research directions.
216 citations
TL;DR: The application of the DBNLP algorithm model to collaborative robots can significantly improve its accuracy and safety, providing an experimental basis for the performance improvement of later collaborative robots.
Abstract: Objective: This paper is to analyze the performance of the control system of collaborative robots based on cognitive computing technology. Methods: This study combines cognitive computing and deep belief network algorithms with collaborative robots to construct a cognitive computing system model based on deep belief networks, which is applied to the control system of collaborative robots. Further, the simulation is used to compare and analyze the algorithm performance of deep belief network (DBN), multilayer perceptron (MLP) and the cognitive computing system model of deep belief network and linear perceptron (DBNLP) proposed in this study. Results: The results show that compared with the DBN and MLP algorithms, the DBNLP algorithm model has a significantly lower error rate in the number of repetitions of the training set, the number of hidden neurons, and the number of network layers. And the number of task backlog, the number of resources to be allocated and the time consumption are less, as well as the accuracy is high. After comparing and analyzing the changes in the estimated value of Ex (expected value), En (entropy value) and He (hyper entropy value), it is found that the estimated value of the DBNLP algorithm model is closer to the true value than that of the DBN and MLP algorithms. Conclusion: The application of the DBNLP algorithm model to collaborative robots can significantly improve its accuracy and safety, providing an experimental basis for the performance improvement of later collaborative robots.
155 citations
TL;DR: A layered architecture of SH that combines ontology and MA technologies is designed to automatically acquire semantic knowledge, and support heterogeneity and interoperability services, and results are provided to show the feasibility, effectiveness, and robustness of this proposal.
Abstract: Smart home (SH) as an emerging paradigm for alleviating the overstretched healthcare resources, and enhancing the quality of life has received increasing attention. While the remarkable progress has been made for the development of SH, it still suffers from a number of issues (e.g., device heterogeneity, composite activities recognition, and providing appropriate services). To address these issues, this paper proposes a knowledge-based approach for multiagent (MA) collaboration. Specifically, a layered architecture of SH that combines ontology and MA technologies is designed to automatically acquire semantic knowledge, and support heterogeneity and interoperability services. In such architecture, a generic inference algorithm is presented based on unordered actions and temporal property of activity for inferring both continuous composite activity and personalized service in real time. Then a novel idea is introduced for agent to learn the knowledge of human activity (HA) autonomously and translate into itself knowledge, the purpose of which is to guide agent for performing services in a way that is compatible with HA. The proposed schemes have been implemented in an SH, and evaluated through extensive experiments. The results are provided to show the feasibility, effectiveness, and robustness of our proposal.
56 citations
TL;DR: An extensive review of research directions and topics of different approaches such as sensing, learning and gripping, which have been implemented within the current five years are presented.
Abstract: Interaction between a robot and its environment requires perception about the environment, which helps the robot in making a clear decision about the object type and its location. After that, the end effector will be brought to the object’s location for grasping. There are many research studies on the reaching and grasping of objects using different techniques and mechanisms for increasing accuracy and robustness during grasping and reaching tasks. Thus, this paper presents an extensive review of research directions and topics of different approaches such as sensing, learning and gripping, which have been implemented within the current five years.
37 citations
References
More filters
TL;DR: New results are derived on the minimum number of landmarks needed to obtain a solution, and algorithms are presented for computing these minimum-landmark solutions in closed form that provide the basis for an automatic system that can solve the Location Determination Problem under difficult viewing.
Abstract: A new paradigm, Random Sample Consensus (RANSAC), for fitting a model to experimental data is introduced. RANSAC is capable of interpreting/smoothing data containing a significant percentage of gross errors, and is thus ideally suited for applications in automated image analysis where interpretation is based on the data provided by error-prone feature detectors. A major portion of this paper describes the application of RANSAC to the Location Determination Problem (LDP): Given an image depicting a set of landmarks with known locations, determine that point in space from which the image was obtained. In response to a RANSAC requirement, new results are derived on the minimum number of landmarks needed to obtain a solution, and algorithms are presented for computing these minimum-landmark solutions in closed form. These results provide the basis for an automatic system that can solve the LDP under difficult viewing
23,396 citations
Book•
01 Jan 1979TL;DR: The relationship between Stimulation and Stimulus Information for visual perception is discussed in detail in this article, where the authors also present experimental evidence for direct perception of motion in the world and movement of the self.
Abstract: Contents: Preface. Introduction. Part I: The Environment To Be Perceived.The Animal And The Environment. Medium, Substances, Surfaces. The Meaningful Environment. Part II: The Information For Visual Perception.The Relationship Between Stimulation And Stimulus Information. The Ambient Optic Array. Events And The Information For Perceiving Events. The Optical Information For Self-Perception. The Theory Of Affordances. Part III: Visual Perception.Experimental Evidence For Direct Perception: Persisting Layout. Experiments On The Perception Of Motion In The World And Movement Of The Self. The Discovery Of The Occluding Edge And Its Implications For Perception. Looking With The Head And Eyes. Locomotion And Manipulation. The Theory Of Information Pickup And Its Consequences. Part IV: Depiction.Pictures And Visual Awareness. Motion Pictures And Visual Awareness. Conclusion. Appendixes: The Principal Terms Used in Ecological Optics. The Concept of Invariants in Ecological Optics.
21,493 citations
"Cognition-Enabled Robot Manipulatio..." refers methods in this paper
...The affor dance concept of Gibson [128] is used in a variety of studies (e....
[...]
Proceedings Article•
01 Jan 2009TL;DR: This paper discusses how ROS relates to existing robot software frameworks, and briefly overview some of the available application software which uses ROS.
Abstract: This paper gives an overview of ROS, an opensource robot operating system. ROS is not an operating system in the traditional sense of process management and scheduling; rather, it provides a structured communications layer above the host operating systems of a heterogenous compute cluster. In this paper, we discuss how ROS relates to existing robot software frameworks, and briefly overview some of the available application software which uses ROS.
8,387 citations
TL;DR: In this article, a new representation learning approach for domain adaptation is proposed, in which data at training and test time come from similar but different distributions, and features that cannot discriminate between the training (source) and test (target) domains are used to promote the emergence of features that are discriminative for the main learning task on the source domain.
Abstract: We introduce a new representation learning approach for domain adaptation, in which data at training and test time come from similar but different distributions. Our approach is directly inspired by the theory on domain adaptation suggesting that, for effective domain transfer to be achieved, predictions must be made based on features that cannot discriminate between the training (source) and test (target) domains.
The approach implements this idea in the context of neural network architectures that are trained on labeled data from the source domain and unlabeled data from the target domain (no labeled target-domain data is necessary). As the training progresses, the approach promotes the emergence of features that are (i) discriminative for the main learning task on the source domain and (ii) indiscriminate with respect to the shift between the domains. We show that this adaptation behaviour can be achieved in almost any feed-forward model by augmenting it with few standard layers and a new gradient reversal layer. The resulting augmented architecture can be trained using standard backpropagation and stochastic gradient descent, and can thus be implemented with little effort using any of the deep learning packages.
We demonstrate the success of our approach for two distinct classification problems (document sentiment analysis and image classification), where state-of-the-art domain adaptation performance on standard benchmarks is achieved. We also validate the approach for descriptor learning task in the context of person re-identification application.
4,862 citations
01 Sep 2004
TL;DR: Gazebo is designed to fill this niche by creating a 3D dynamic multi-robot environment capable of recreating the complex worlds that would be encountered by the next generation of mobile robots.
Abstract: Simulators have played a critical role in robotics research as tools for quick and efficient testing of new concepts, strategies, and algorithms. To date, most simulators have been restricted to 2D worlds, and few have matured to the point where they are both highly capable and easily adaptable. Gazebo is designed to fill this niche by creating a 3D dynamic multi-robot environment capable of recreating the complex worlds that would be encountered by the next generation of mobile robots. Its open source status, fine grained control, and high fidelity place Gazebo in a unique position to become more than just a stepping stone between the drawing board and real hardware: data visualization, simulation of remote environments, and even reverse engineering of blackbox systems are all possible applications. Gazebo is developed in cooperation with the Player and Stage projects (Gerkey, B. P., et al., July 2003), (Gerkey, B. P., et al., May 2001), (Vaughan, R. T., et al., Oct. 2003), and is available from http://playerstage.sourceforge.net/gazebo/ gazebo.html.
2,824 citations