Recognizing and manipulating objects is an important task for mobile robots performing useful services in everyday environments. In this paper, we develop a system that enables a robot to grasp an object and to move it in front of its depth camera so as to build a 3D surface model of the object. We derive an information gain based variant of the next best view algorithm in order to determine how the manipulator should move the object in front of the camera. By considering occlusions caused by the robot manipulator, our technique also determines when and how the robot should re-grasp the object in order to build a complete model.

/pdf/autonomous-generation-of-complete-3d-object-models-using-327j18w5f4.pdf

Autonomous generation of complete 3D object models using next best view manipulation planning

With robotics hardware becoming more and more safe and compliant, robots are not far from entering our homes. The robot, that will share the same environment with humans, will be expected to consider the geometry of the interaction and to perform intelligent space sharing.

Synthesizing Robot Motions Adapted to Human Presence A Planning and Control Framework for Safe and Socially Acceptable Robot Motions

Three-dimensional (3D) object reconstruction is the process of building a 3D model of a real object. This task is performed by taking several scans of an object from different locations (views). Due to the limited field of view of the sensor and the object's self-occlusions, it is a difficult problem to solve. In addition, sensor positioning by robots is not perfect, making the actual view different from the expected one. We propose a next best view (NBV) algorithm that determines each view to reconstruct an arbitrary object. Furthermore, we propose a method to deal with the uncertainty in sensor positioning. The algorithm fulfills all the constraints of a reconstruction process, such as new information, positioning constraints, sensing constraints and registration constraints. Moreover, it improves the scan's quality and reduces the navigation distance. The algorithm is based on a search-based paradigm where a set of candidate views is generated and then each candidate view is evaluated to determine whic...

https://journals.sagepub.com/doi/pdf/10.5772/58759

Volumetric Next-best-view Planning for 3D Object Reconstruction with Positioning Error

We study the problem of actively searching for an object in a three-dimensional (3-D) environment under the constraint of a maximum search time using a visually guided humanoid robot with 26 degrees of freedom. The inherent intractability of the problem is discussed, and a greedy strategy for selecting the best next viewpoint is employed. We describe a target probability updating scheme approximating the optimal solution to the problem, providing an efficient solution to the selection of the best next viewpoint. We employ a hierarchical recognition architecture, inspired by human vision, that uses contextual cues for attending to the view-tuned units at the proper intrinsic scales and for active control of the robotic platform sensor's coordinate frame, which also gives us control of the extrinsic image scale and achieves the proper sequence of pathognomonic views of the scene. The recognition model makes no particular assumptions on shape properties like texture and is trained by showing the object by hand to the robot. Our results demonstrate the feasibility of using state-of-the-art vision-based systems for efficient and reliable object localization in an indoor 3-D environment.

/pdf/active-3d-object-localization-using-a-humanoid-robot-1vtu5hgrkb.pdf

Active 3D Object Localization Using a Humanoid Robot

We propose a holistic approach for three-dimensional (3D) object reconstruction with a mobile manipulator robot with an eye-in-hand sensor; considering the plan to reach the desired view/state, and the uncertainty in both observations and controls. This is one of the first methods that determines the next best view/state in the state space, following a methodology in which a set of candidate views/states is directly generated in the state space, and later only a subset of these views is kept by filtering the original set. It also determines the controls that yield a collision free trajectory to reach a state using rapidly-exploring random trees. To decrease the processing time we propose an efficient evaluation strategy based on filters, and a 3D visibility calculation with hierarchical ray tracing. The next best view/state is selected based on the expected utility, generating samples in the control space based on an error distribution according to the dynamics of the robot. This makes the method robust to positioning error, significantly reducing the collision rate and increasing the coverage, as shown in the experiments. Several experiments in simulation and with a real mobile manipulator robot with 8 degrees of freedom show that the proposed method provides an effective and fast method for a mobile manipulator to build 3D models of unknown objects. To our knowledge, this is one of the first works that demonstrates the reconstruction of complex objects with a real mobile manipulator considering uncertainty in the controls.

View/state planning for three-dimensional object reconstruction under uncertainty

This paper proposes a method for planning the pick-and-place motion of an object by dual-arm manipulators. Our planner is composed of the offline and the online phases. The offline phase generates a set of regions on the object and the environment surfaces and calculates several parameters needed in the online phase. In the online phase, the planner selects a grasping pose of the robot and a putting posture of the object by searching for the regions calculated in the offline phase. By using the proposed method, we can also plan the trajectory of the robot, and the regrasping strategy of the dual-arm. Here, the putting posture of the object can be planned by considering stability of the object placed on the environment. The effectiveness of the proposed method is confirmed by simulation and experimental results by using the dual-arm robot NX-HIRO.

Pick and place planning for dual-arm manipulators

A novel approach is presented which aims at building autonomously visual models of unknown objects, using a humanoid robot. Previous methods have been proposed for the specific problem of the next-best-view during the modeling and the recognition process. However our approach differs as it takes advantage of humanoid specificities in terms of embedded vision sensor and redundant motion capabilities. In a previous work, another approach to this specific problem was presented which relies on a derivable formulation of the visual evaluation in order to integrate it with our posture generation method. However to get rid of some limitations we propose a new method, formulated using two steps: (i) an optimization algorithm without derivatives is used to find a camera pose which maximizes the amount of unknown data visible, and (ii) a whole robot posture is generated by using a different optimization method where the computed camera pose is set as a constraint on the robot head.

/pdf/a-two-steps-next-best-view-algorithm-for-autonomous-3d-va1ug49ykg.pdf

A two-steps next-best-view algorithm for autonomous 3D object modeling by a humanoid robot

Necessity to recognize the world like a home environment by a humanoid robot has recently been arisen for daily usages. As an observation sensor, stereo vision is the most common device for a humanoid robot to obtain the environmental data, but it is more erroneous than a laser sensor. To overcome the inaccuracy of stereo vision, we propose a particle-based SLAM technique so that the SLAM posterior is estimated by multiple hypotheses. The major difficulty of the particle-based SLAM with 3D grid maps is the high computational cost. To reduce the computational cost, we also propose a scheduling method for the time when to match and for particles that engage in the matching process. Through experiments with a humanoid robot, HRP-2, it is shown that the proposed approach can reduce the computational cost while preserving estimation accuracy.

3D grid and particle based SLAM for a humanoid robot

Dexterous manipulation is an important function for working robots. Manipulator tasks such as grasping, assembly and disassembly can generally be divided into several motion primitives. We call such motion primitives “skills” and explain how most manipulator tasks can be composed of sequences of these skills. We will address the issues involved with various types of robots such as maintenance robots and service robots. We have considered hierarchizing the manipulation tasks of these robots since their tasks have become more complex than ever before. Additionally, as errors are seen likely to increase in complex tasks, it is important to implement effective error recovery technology. This paper presents our proposal for a new type of error recovery that uses the concepts of task stratification and error classification which can be expressed specifically using flow charts.

Error recovery using task stratification and error classification for manipulation robots in various fields

We present our investigation to make humanoids build autonomously geometric models of unknown objects. Although good methods have been proposed for the specific problem of the next-best-view during the modeling and the recognition process; our approach is different and takes into account humanoid specificities in terms of embedded vision sensor and redundant motion capabilities. The problem to select the best next view of interest at each modeling step is formulated as an optimization problem where the whole robot posture needs to be defined jointly with the robot cameraspsila position and orientation. To achieve this, we propose a differentiable formula that expresses the amount of unknown data visible from a specific viewpoint, given only knowledge acquired in previous steps. In addition, a specific stability constraint is introduced to allow the robot to reach a configuration where its feet can be moved away from their initial position.

/pdf/a-next-best-view-algorithm-for-autonomous-3d-object-modeling-58gguhzdkx.pdf

Torea Foissotte

Papers

Pick and place planning for dual-arm manipulators

A two-steps next-best-view algorithm for autonomous 3D object modeling by a humanoid robot

3D grid and particle based SLAM for a humanoid robot

Error recovery using task stratification and error classification for manipulation robots in various fields

A next-best-view algorithm for autonomous 3D object modeling by a humanoid robot