Real-Time Robotic Grasping and Localization Using Deep Learning-Based Object Detection Technique

TLDR: This work achieved the state-of-the-art results at 100% precision of object detection, 100% accuracy for robotic positioning and 100% successful real-time robotic grasping within 0.38 seconds as detection time.

Abstract: This work aims to increase the impact of computer vision on robotic positioning and grasping in industrial assembly lines. Real-time object detection and localization problem is addressed for robotic grasp-and-place operation using Selective Compliant Assembly Robot Arm (SCARA). The movement of SCARA robot is guided by deep learning-based object detection for grasp task and edge detection-based position measurement for place task. Deep Convolutional Neural Network (CNN) model, called KSSnet, is developed for object detection based on CNN Alexnet using transfer learning approach. SCARA training dataset with 4000 images of two object categories associated with 20 different positions is created and labeled to train KSSnet model. The position of the detected object is included in prediction result at the output classification layer. This method achieved the state-of-the-art results at 100% precision of object detection, 100% accuracy for robotic positioning and 100% successful real-time robotic grasping within 0.38 seconds as detection time. A combination of Zerocross and Canny edge detectors is implemented on a circular object to simplify the place task. For accurate position measurement, the distortion of camera lens is removed using camera calibration technique where the measured position represents the desired location to place the grasped object. The result showed that the robot successfully moved to the measured position with positioning Root Mean Square Error (0.361, 0.184) mm and 100% for successful place detection.