Pose estimation of texture-less cylindrical objects in bin picking using sensor fusion
01 Oct 2016-pp 2279-2284
TL;DR: Disturbance of objects caused during pick up has been modelled, which allows pickup of multiple pellets based on information from a single range scan, which eliminates the necessity of repeated scanning and data conditioning.
Abstract: We propose an approach for emptying of bin using a combination of Image and Range sensor. Offering a complete solution: calibration, segmentation and pose estimation, along with approachability analysis for the estimated pose. The work is novel in the sense that the objects to be picked are featureless and uniformly black in colour, hence existing approaches are not directly applicable. A key point involves optimal utilization of range data acquired from the laser scanner for 3-D segmentation using localized geometric information. This information guides segmentation of the image for better object pose estimation, used for pick-and-drop. We analytically assure the approachability of the object to avoid collision of the manipulator with the bin. Disturbance of objects caused during pick up has been modelled, which allows pickup of multiple pellets based on information from a single range scan. This eliminates the necessity of repeated scanning and data conditioning. The proposed method offers high object detection rate and pose estimation accuracy. The innovative techniques aimed at reducing the average pickup time makes it suitable for robust industrial operation.
Citations
More filters
TL;DR: The results show that the proposed method could reduce the errors in pose in a consistent manner, even when different measurement instruments were used, or when there was a deterioration in observability due to the choice of poses during identification.
Abstract: Combination of geometric and parametric approaches of kinematic identification is proposed in this article. The experimental strategy is similar to that used in geometric approach wherein each axis of the robot is actuated one after the other. This adds clarity to experimental strategy, which becomes ambiguous while solely using a conventional parametric approach. Therefore it is easier to conduct experiments even if there are restrictions in workspace. The estimation was done using a parametric technique, but in a stage wise manner using a divide and conquer strategy. This allowed measurement of the robot accuracy after removing the errors arising due to the definition of base and end-effector frames. Additionally it is possible to visualize the reduction in errors during the estimation process. In addition to this, the Jacobian matrix that relates the pose errors to the correction in parameters is adapted during estimation using a damped least squares method depending upon the convergence of the parameters. Results were obtained after extensive experiments on industrial robots using three different measurement instruments namely laser tracker, monocular camera and multi-camera system. The proposed method performs better than the conventional approach which uses only geometric approach. Finally thanks to the new approach, it was possible to conduct experiments after dividing the workspace region into those with high and low levels of observability; which is not easy while using conventional approaches. It was also possible to perform identification in regions closer and farther away from the robot where there is deterioration of observability. The results show that the proposed method could reduce the errors in pose in a consistent manner, even when different measurement instruments were used, or when there was a deterioration in observability due to the choice of poses during identification.
24 citations
TL;DR: This work develops an automated synthetic data generation pipeline to produce large-scale photo-realistic data with precise annotations and formulates an instance segmentation network for object recognition and a 6D pose estimation network for localization.
Abstract: We present a generic and robust sim-to-real deep-learning-based framework, namely S2R-Pick, for fast and accurate object recognition and localization in industrial robotic bin picking. Unlike existing works designed for general everyday environments, objects for industrial bin picking are often texture-less, metallic, and also suffer from severe occlusion and cluttering. In this work, we first develop an automated synthetic data generation pipeline to produce large-scale photo-realistic data with precise annotations. We then formulate an instance segmentation network for object recognition and a 6D pose estimation network for localization. We cascade and train the two networks only on our generated synthetic data and can directly transfer them for processing real inputs without needing to retrain them on real samples. Extensive experiments show the effectiveness and superiority of our framework over state-of-the-art methods.
12 citations
TL;DR: In this article, the identification of elasto-static parameters of an industrial robot using measurements from a monocular camera was discussed, and the method of identification involved a parametric method for estimation and the experimental strategy involved joint wise actuation about a particular pose.
Abstract: This article discusses the identification of elasto-static parameters of an industrial robot using measurements from a monocular camera. The method of identification involved a parametric method for estimation and the experimental strategy involved joint wise actuation about a particular pose, inspired by the geometric method of parameter identification. The measurements were made using a monocular camera utilizing fiducial markers. The joint compliance values were obtained using a two-stage approach which made it easy to analyze the observability of parameters to be identified. Coupled with the proposed strategy of experiments, it was possible to check the observability of parameters throughout the whole workspace. It was found that most of the workspace regions of the robot where the experiments were feasible, had high values of observability which simplified the experiments. During experimental validation on the Fanuc 165F robot, the kinematic parameters were initially estimated and the results were found to be superior compared to a recently proposed approach while considering the errors in positioning. Later the end-effector of the robot was subjected to an external load. Deflection due to loading on the end-effector, calculated using pose measurements from an industrial camera was found comparable to that from a laser tracker. Measured values for position repeatability of the robot were close to the robot specifications as well. The end-effector errors could be reduced by 45% after compensating errors due to kinematic and elasto-static parameters while using the proposed method wherein measurements from a monocular camera were used.
9 citations
01 Apr 2022
TL;DR: In this article , the identification of elasto-static parameters of an industrial robot using measurements from a monocular camera was discussed, and the method of identification involved a parametric method for estimation and the experimental strategy involved joint wise actuation about a particular pose.
Abstract: This article discusses the identification of elasto-static parameters of an industrial robot using measurements from a monocular camera. The method of identification involved a parametric method for estimation and the experimental strategy involved joint wise actuation about a particular pose, inspired by the geometric method of parameter identification. The measurements were made using a monocular camera utilizing fiducial markers. The joint compliance values were obtained using a two-stage approach which made it easy to analyze the observability of parameters to be identified. Coupled with the proposed strategy of experiments, it was possible to check the observability of parameters throughout the whole workspace. It was found that most of the workspace regions of the robot where the experiments were feasible, had high values of observability which simplified the experiments. During experimental validation on the Fanuc 165F robot, the kinematic parameters were initially estimated and the results were found to be superior compared to a recently proposed approach while considering the errors in positioning. Later the end-effector of the robot was subjected to an external load. Deflection due to loading on the end-effector, calculated using pose measurements from an industrial camera was found comparable to that from a laser tracker. Measured values for position repeatability of the robot were close to the robot specifications as well. The end-effector errors could be reduced by 45% after compensating errors due to kinematic and elasto-static parameters while using the proposed method wherein measurements from a monocular camera were used.
7 citations
TL;DR: S2R-Pick as discussed by the authors is a sim-to-real deep learning-based framework for fast and accurate object recognition and localization in industrial robotic bin picking, which can directly transfer them for processing real inputs without needing to retrain them on real samples.
Abstract: We present a generic and robust sim-to-real deep-learning-based framework, namely S2R-Pick, for fast and accurate object recognition and localization in industrial robotic bin picking. Unlike existing works designed for general everyday environments, objects for industrial bin picking are often texture-less, metallic, and also suffer from severe occlusion and cluttering. In this work, we first develop an automated synthetic data generation pipeline to produce large-scale photo-realistic data with precise annotations. We then formulate an instance segmentation network for object recognition and a 6D pose estimation network for localization. We cascade and train the two networks only on our generated synthetic data and can directly transfer them for processing real inputs without needing to retrain them on real samples. Extensive experiments show the effectiveness and superiority of our framework over state-of-the-art methods.
6 citations
References
More filters
TL;DR: A flexible technique to easily calibrate a camera that only requires the camera to observe a planar pattern shown at a few (at least two) different orientations is proposed and advances 3D computer vision one more step from laboratory environments to real world use.
Abstract: We propose a flexible technique to easily calibrate a camera. It only requires the camera to observe a planar pattern shown at a few (at least two) different orientations. Either the camera or the planar pattern can be freely moved. The motion need not be known. Radial lens distortion is modeled. The proposed procedure consists of a closed-form solution, followed by a nonlinear refinement based on the maximum likelihood criterion. Both computer simulation and real data have been used to test the proposed technique and very good results have been obtained. Compared with classical techniques which use expensive equipment such as two or three orthogonal planes, the proposed technique is easy to use and flexible. It advances 3D computer vision one more step from laboratory environments to real world use.
13,200 citations
"Pose estimation of texture-less cyl..." refers methods in this paper
...An initial estimate of the transformation was done using Camera calibration toolbox [11]....
[...]
TL;DR: A non-iterative solution to the PnP problem—the estimation of the pose of a calibrated camera from n 3D-to-2D point correspondences—whose computational complexity grows linearly with n, which can be done in O(n) time by expressing these coordinates as weighted sum of the eigenvectors of a 12×12 matrix.
Abstract: We propose a non-iterative solution to the PnP problem--the estimation of the pose of a calibrated camera from n 3D-to-2D point correspondences--whose computational complexity grows linearly with n This is in contrast to state-of-the-art methods that are O(n 5) or even O(n 8), without being more accurate Our method is applicable for all n?4 and handles properly both planar and non-planar configurations Our central idea is to express the n 3D points as a weighted sum of four virtual control points The problem then reduces to estimating the coordinates of these control points in the camera referential, which can be done in O(n) time by expressing these coordinates as weighted sum of the eigenvectors of a 12×12 matrix and solving a small constant number of quadratic equations to pick the right weights Furthermore, if maximal precision is required, the output of the closed-form solution can be used to initialize a Gauss-Newton scheme, which improves accuracy with negligible amount of additional time The advantages of our method are demonstrated by thorough testing on both synthetic and real-data
2,598 citations
"Pose estimation of texture-less cyl..." refers methods in this paper
...The measured values of the crossectional points found using the camera and the laser sensor were compared against their estimate in the tool frame, to minimize the errors....
[...]
TL;DR: The novel techniques include statistical analysis, persistent histogram features estimation that allows for a consistent registration, resampling with additional robust fitting techniques, and segmentation of the environment into meaningful regions.
Abstract: This article investigates the problem of acquiring 3D object maps of indoor household environments, in particular kitchens. The objects modeled in these maps include cupboards, tables, drawers and shelves, which are of particular importance for a household robotic assistant. Our mapping approach is based on PCD (point cloud data) representations. Sophisticated interpretation methods operating on these representations eliminate noise and resample the data without deleting the important details, and interpret the improved point clouds in terms of rectangular planes and 3D geometric shapes. We detail the steps of our mapping approach and explain the key techniques that make it work. The novel techniques include statistical analysis, persistent histogram features estimation that allows for a consistent registration, resampling with additional robust fitting techniques, and segmentation of the environment into meaningful regions.
950 citations
"Pose estimation of texture-less cyl..." refers methods in this paper
...Given a general situation, segmentation methods based on multi sensory input (grey-scale/RGB and depth data) [8] tend to work better than using any one sensor....
[...]
TL;DR: The dissertation presented in this article proposes Semantic 3D Object Models as a novel representation of the robot’s operating environment that satisfies these requirements and shows how these models can be automatically acquired from dense 3D range data.
Abstract: Environment models serve as important resources for an autonomous robot by providing it with the necessary task-relevant information about its habitat. Their use enables robots to perform their tasks more reliably, flexibly, and efficiently. As autonomous robotic platforms get more sophisticated manipulation capabilities, they also need more expressive and comprehensive environment models: for manipulation purposes their models have to include the objects present in the world, together with their position, form, and other aspects, as well as an interpretation of these objects with respect to the robot tasks. The dissertation presented in this article (Rusu, PhD thesis, 2009) proposes Semantic 3D Object Models as a novel representation of the robot’s operating environment that satisfies these requirements and shows how these models can be automatically acquired from dense 3D range data.
908 citations
"Pose estimation of texture-less cyl..." refers methods in this paper
...Later, the results were processed by image based segmentation for missing information....
[...]
881 citations