In this work, we study 3D object detection from RGBD data in both indoor and outdoor scenes. While previous methods focus on images or 3D voxels, often obscuring natural 3D patterns and invariances of 3D data, we directly operate on raw point clouds by popping up RGB-D scans. However, a key challenge of this approach is how to efficiently localize objects in point clouds of large-scale scenes (region proposal). Instead of solely relying on 3D proposals, our method leverages both mature 2D object detectors and advanced 3D deep learning for object localization, achieving efficiency as well as high recall for even small objects. Benefited from learning directly in raw point clouds, our method is also able to precisely estimate 3D bounding boxes even under strong occlusion or with very sparse points. Evaluated on KITTI and SUN RGB-D 3D detection benchmarks, our method outperforms the state of the art by remarkable margins while having real-time capability.

/pdf/frustum-pointnets-for-3d-object-detection-from-rgb-d-data-49j9mx9mu6.pdf

Frustum PointNets for 3D Object Detection from RGB-D Data

Human-Robot Interaction (HRI) has recently received considerable attention in the academic community, in labs, in technology companies, and through the media. Because of this attention, it is desirable to present a survey of HRI to serve as a tutorial to people outside the field and to promote discussion of a unified vision of HRI within the field. The goal of this review is to present a unified treatment of HRI-related problems, to identify key themes, and discuss challenge problems that are likely to shape the field in the near future. Although the review follows a survey structure, the goal of presenting a coherent "story" of HRI means that there are necessarily some well-written, intriguing, and influential papers that are not referenced. Instead of trying to survey every paper, we describe the HRI story from multiple perspectives with an eye toward identifying themes that cross applications. The survey attempts to include papers that represent a fair cross section of the universities, government efforts, industry labs, and countries that contribute to HRI, and a cross section of the disciplines that contribute to the field, such as human, factors, robotics, cognitive psychology, and design.

/pdf/human-robot-interaction-a-survey-4q8yi606a9.pdf

Human-Robot Interaction: A Survey

Traditional robots have rigid underlying structures that limit their ability to interact with their environment. For example, conventional robot manipulators have rigid links and can manipulate objects using only their specialised end effectors. These robots often encounter difficulties operating in unstructured and highly congested environments. A variety of animals and plants exhibit complex movement with soft structures devoid of rigid components. Muscular hydrostats e.g. octopus arms and elephant trunks are almost entirely composed of muscle and connective tissue and plant cells can change shape when pressurised by osmosis. Researchers have been inspired by biology to design and build soft robots. With a soft structure and redundant degrees of freedom, these robots can be used for delicate tasks in cluttered and/or unstructured environments. This paper discusses the novel capabilities of soft robots, describes examples from nature that provide biological inspiration, surveys the state of the art and outlines existing challenges in soft robot design, modelling, fabrication and control.

/pdf/soft-robotics-biological-inspiration-state-of-the-art-and-1esuws0p9g.pdf

Soft robotics: Biological inspiration, state of the art, and future research

Advances in soft robotics, materials science, and stretchable electronics have enabled rapid progress in soft grippers. Here, a critical overview of soft robotic grippers is presented, covering different material sets, physical principles, and device architectures. Soft gripping can be categorized into three technologies, enabling grasping by: a) actuation, b) controlled stiffness, and c) controlled adhesion. A comprehensive review of each type is presented. Compared to rigid grippers, end-effectors fabricated from flexible and soft components can often grasp or manipulate a larger variety of objects. Such grippers are an example of morphological computation, where control complexity is greatly reduced by material softness and mechanical compliance. Advanced materials and soft components, in particular silicone elastomers, shape memory materials, and active polymers and gels, are increasingly investigated for the design of lighter, simpler, and more universal grippers, using the inherent functionality of the materials. Embedding stretchable distributed sensors in or on soft grippers greatly enhances the ways in which the grippers interact with objects. Challenges for soft grippers include miniaturization, robustness, speed, integration of sensing, and control. Improved materials, processing methods, and sensing play an important role in future research.

https://onlinelibrary.wiley.com/doi/pdf/10.1002/adma.201707035

Soft Robotic Grippers.

An intelligent trial-and-error learning algorithm is presented that allows robots to adapt in minutes to compensate for a wide variety of types of damage. Autonomous mobile robots would be extremely useful in remote or hostile environments such as space, deep oceans or disaster areas. An outstanding challenge is to make such robots able to recover after damage. Jean-Baptiste Mouret and colleagues have developed a machine learning algorithm that enables damaged robots to quickly regain their ability to perform tasks. When they sustain damage — such as broken or even missing legs — the robots adopt an intelligent trial-and-error approach, trying out possible behaviours that they calculate to be potentially high-performing. After a handful of such experiments they discover, in less than two minutes, a compensatory behaviour that works in spite of the damage. Robots have transformed many industries, most notably manufacturing1, and have the power to deliver tremendous benefits to society, such as in search and rescue2, disaster response3, health care4 and transportation5. They are also invaluable tools for scientific exploration in environments inaccessible to humans, from distant planets6 to deep oceans7. A major obstacle to their widespread adoption in more complex environments outside factories is their fragility6,8. Whereas animals can quickly adapt to injuries, current robots cannot ‘think outside the box’ to find a compensatory behaviour when they are damaged: they are limited to their pre-specified self-sensing abilities, can diagnose only anticipated failure modes9, and require a pre-programmed contingency plan for every type of potential damage, an impracticality for complex robots6,8. A promising approach to reducing robot fragility involves having robots learn appropriate behaviours in response to damage10,11, but current techniques are slow even with small, constrained search spaces12. Here we introduce an intelligent trial-and-error algorithm that allows robots to adapt to damage in less than two minutes in large search spaces without requiring self-diagnosis or pre-specified contingency plans. Before the robot is deployed, it uses a novel technique to create a detailed map of the space of high-performing behaviours. This map represents the robot’s prior knowledge about what behaviours it can perform and their value. When the robot is damaged, it uses this prior knowledge to guide a trial-and-error learning algorithm that conducts intelligent experiments to rapidly discover a behaviour that compensates for the damage. Experiments reveal successful adaptations for a legged robot injured in five different ways, including damaged, broken, and missing legs, and for a robotic arm with joints broken in 14 different ways. This new algorithm will enable more robust, effective, autonomous robots, and may shed light on the principles that animals use to adapt to injury.

/pdf/robots-that-can-adapt-like-animals-21owdbvqr7.pdf

Robots that can adapt like animals

On March 11, 2011, a massive earthquake (magnitude 9.0) and accompanying tsunami hit the Tohoku region of eastern Japan. Since then, the Fukushima Daiichi Nuclear Power Plants have been facing a crisis due to the loss of all power that resulted from the meltdown accidents. Three buildings housing nuclear reactors were seriously damaged from hydrogen explosions, and, in one building, the nuclear reactions became out of control. It was too dangerous for humans to enter the buildings to inspect the damage because radioactive materials were also being released. In response to this crisis, it was decided that mobile rescue robots would be used to carry out surveillance missions. The mobile rescue robots needed could not be delivered to the Tokyo Electric Power Company (TEPCO) until various technical issues were resolved. Those issues involved hardware reliability, communication functions, and the ability of the robots' electronic components to withstand radiation. Additional sensors and functionality that would enable the robots to respond effectively to the crisis were also needed. Available robots were therefore retrofitted for the disaster reponse missions. First, the radiation tolerance of the electronic componenets was checked by means of gamma ray irradiation tests, which were conducted using the facilities of the Japan Atomic Energy Agency (JAEA). The commercial electronic devices used in the original robot systems operated long enough (more than 100 h at a 10% safety margin) in the assumed environment (100 mGy/h). Next, the usability of wireless communication in the target environment was assessed. Such tests were not possible in the target environment itself, so they were performed at the Hamaoka Daiichi Nuclear Power Plants, which are similar to the target environment. As previously predicted, the test results indicated that robust wireless communication would not be possible in the reactor buildings. It was therefore determined that a wired communication device would need to be installed. After TEPCO's official urgent mission proposal was received, the team mounted additional devices to facilitate the installation of a water gauge in the basement of the reactor buildings to determine flooding levels. While these preparations were taking place, prospective robot operators from TEPCO trained in a laboratory environment. Finally, one of the robots was delivered to the Fukushima Daiichi Nuclear Power Plants on June 20, 2011, where it performed a number of important missions inside the buildings. In this paper, the requirements for the exploration mission in the Fukushima Daiichi Nuclear Power Plants are presented, the implementation is discussed, and the results of the mission are reported. © 2013 Wiley Periodicals, Inc.

(Webpage: http://www.astro.mech.tohoku.ac.jp/)

http://www.astro.mech.tohoku.ac.jp/~keiji/papers/pdf/2013-JFR-Quince-online.pdf

Emergency response to the nuclear accident at the Fukushima Daiichi Nuclear Power Plants using mobile rescue robots

Disaster rescue is one of the most serious social issue which involves very large numbers of heterogeneous agents in the hostile environment. RoboCup-Rescue intends to promote research and development in this socially significant domain by creating a standard simulator and forum for researchers and practitioners. While the rescue domain intuitively appealing as large scale multi-agent domains, it has not yet given through analysis on its domain characteristics. In this paper, we present detailed analysis on the task domain and elucidate characteristics necessary for multi-agent systems for this domain.

RoboCup Rescue: search and rescue in large-scale disasters as a domain for autonomous agents research

We report recent results from field experiments conducted with a team of ground and aerial robots engaged in the collaborative mapping of an earthquake-damaged building. The goal of the experimental exercise is the generation of three-dimensional maps that capture the layout of a multifloor environment. The experiments took place in the top three floors of a structurally compromised building at Tohoku University in Sendai, Japan that was damaged during the 2011 Tohoku earthquake. We provide details of the approach to the collaborative mapping and report results from the experiments in the form of maps generated by the individual robots and as a team. We conclude by discussing observations from the experiments and future research topics. © 2012 Wiley Periodicals, Inc.

(This work builds upon the conference paper (Michael et al., 2012).)

/pdf/collaborative-mapping-of-an-earthquake-damaged-building-via-4ees2efish.pdf

Collaborative mapping of an earthquake-damaged building via ground and aerial robots

In order to summarize the status of rescue robotics, this chapter will cover the basic characteristics of disasters and their impact on robotic design, describe the robots actually used in disasters to date, promising robot designs (e.g., snakes, legged locomotion) and concepts (e.g., robot teams or swarms, sensor networks), methods of evaluation in benchmarks for rescue robotics, and conclude with a discussion of the fundamental problems and open issues facing rescue robotics, and their evolution from an interesting idea to widespread adoption. The Chapter will concentrate on the rescue phase, not recovery, with the understanding that capabilities for rescue can be applied to, and extended for, the recovery phase. The use of robots in the prevention and preparedness phases of disaster management are outside the scope of this chapter.

Satoshi Tadokoro

Papers

Emergency response to the nuclear accident at the Fukushima Daiichi Nuclear Power Plants using mobile rescue robots

RoboCup Rescue: search and rescue in large-scale disasters as a domain for autonomous agents research

Collaborative mapping of an earthquake-damaged building via ground and aerial robots

Search and Rescue Robotics

Collaborative mapping of an earthquake-damaged building via ground and aerial robots