Due to ever increasing precision and automation demands in robotic welding, the automatic and robust 3D seam extraction has become a research hot-spot of the welding robots. At present, most of the research work about seam extraction is aimed at butt joints. Nevertheless,too little work has been devoted to fillet joints and lap joints. Consequently,to achieve robust 3D seam extraction of different weld seams, a novel seam extraction system is proposed according to the 3D structures of welding work pieces. Firstly, a fringe projection system based on Digital Light Processing(DLP) projector is designed to measure the appearance of welding work pieces. Secondly, fusion of the shape information of welding work piece, a 3D seam extraction algorithm is proposed based on point cloud segmentation. Finally, according to the space structure of weld seams, the 3D seam path model and pose estimation are solved based on the established mathematical model of weld seams. Experiments show that the proposed algorithm could well solve different weld seams, such as fillet joints, butt joints and lap joints. Meanwhile, it could well overcome the influence of the materials of welding work pieces, scratch and rust.

A novel system for off-line 3D seam extraction and path planning based on point cloud segmentation for arc welding robot

One of the most important operations during the manufacturing process of a pressure vessel is welding. The result of this operation has a great impact on the vessel integrity; thus, welding inspection procedures must detect defects that could lead to an accident. This paper introduces a computer vision system based on structured light for welding inspection of liquefied petroleum gas (LPG) pressure vessels by using combined digital image processing and deep learning techniques. The inspection procedure applied prior to the welding operation was based on a convolutional neural network (CNN), and it correctly detected the misalignment of the parts to be welded in 97.7% of the cases during the method testing. The post-welding inspection procedure was based on a laser triangulation method, and it estimated the weld bead height and width, with average relative errors of 2.7% and 3.4%, respectively, during the method testing. This post-welding inspection procedure allows us to detect geometrical nonconformities that compromise the weld bead integrity. By using this system, the quality index of the process was improved from 95.0% to 99.5% during practical validation in an industrial environment, demonstrating its robustness.

/pdf/computer-vision-system-for-welding-inspection-of-liquefied-2dveipyhxu.pdf

Computer Vision System for Welding Inspection of Liquefied Petroleum Gas Pressure Vessels Based on Combined Digital Image Processing and Deep Learning Techniques

Application of sensing technology in intelligent robotic arc welding: A review

Welding is a widely used connection method because it is economical materials and strong adaptability to various geometric shapes. For welding assemblies that do not require high rigidity, fillet welding is commonly used due to its economy and efficiency. Robot-based intelligent welding has become popular because the arc light and fume generated during the welding process are harmful to humans. Weld recognition and positioning, as a core in robot-based intelligent welding, have been researched over the last few decades. In the study of fillet weld joint positioning based on structured light vision, it is a commonly used method to position weld by extracting the centerline and feature point of the structured-light stripe. However, the centerline is sometimes inaccurately positioned under strong interference such as highly reflective material and arc light and spatter, resulting in incorrect weld positioning. To solve this shortcoming, this study proposes a universal fillet weld joint recognition and positioning method using structured light. Firstly, the weld likelihood ("likelihood" comes from inferential statistics and is synonymous with "probability") is calculated using the designed convolution kernel. Secondly, the fillet weld joint candidates are preselected using an efficient non-maximum suppression algorithm. Finally, the candidates are reexamined based on local structural feature, and the true fillet weld joints are recognized. The main novelties of the proposed method include: (1) The idea using weld likelihood calculation, preselection and reexamination to position fillet weld joint is proposed for the first time, which can consider both the false positives and false negatives. (2) The proposed method uses convolution, non-maximum suppression and local feature to position fillet weld joint, bypassing the step of calculating the structured-light centerline, thereby reducing the influence of arc light and high reflection. The proposed method is robust, universal and accurate. Moreover, as demonstrated by the following performance indices: the false positives and false negatives are both 0 for normal steel, rusty, highly reflective, and arc light-and-spatter interference welding assemblies, and the false positives are 0 and false negatives are 14.6% for welding assemblies under a multi-line structured light. In addition, the average and maximum biases in the accuracy test are 0.1 mm and 0.52 mm.

Universal fillet weld joint recognition and positioning for robot welding using structured light

Welding is a widely used connection method because it is economical materials and strong adaptability to various geometric shapes. For welding assemblies that do not require high rigidity, fillet welding is commonly used due to its economy and efficiency. Robot-based intelligent welding has become popular because the arc light and fume generated during the welding process are harmful to humans. Weld recognition and positioning, as a core in robot-based intelligent welding, have been researched over the last few decades. In the study of fillet weld joint positioning based on structured light vision, it is a commonly used method to position weld by extracting the centerline and feature point of the structured-light stripe. However, the centerline is sometimes inaccurately positioned under strong interference such as highly reflective material and arc light and spatter, resulting in incorrect weld positioning. To solve this shortcoming, this study proposes a universal fillet weld joint recognition and positioning method using structured light. Firstly, the weld likelihood ("likelihood" comes from inferential statistics and is synonymous with "probability") is calculated using the designed convolution kernel. Secondly, the fillet weld joint candidates are preselected using an efficient non-maximum suppression algorithm. Finally, the candidates are reexamined based on local structural feature, and the true fillet weld joints are recognized. The main novelties of the proposed method include: (1) The idea using weld likelihood calculation, preselection and reexamination to position fillet weld joint is proposed for the first time, which can consider both the false positives and false negatives. (2) The proposed method uses convolution, non-maximum suppression and local feature to position fillet weld joint, bypassing the step of calculating the structured-light centerline, thereby reducing the influence of arc light and high reflection. The proposed method is robust, universal and accurate. Moreover, as demonstrated by the following performance indices: the false positives and false negatives are both 0 for normal steel, rusty, highly reflective, and arc light-and-spatter interference welding assemblies, and the false positives are 0 and false negatives are 14.6% for welding assemblies under a multi-line structured light. In addition, the average and maximum biases in the accuracy test are 0.1 mm and 0.52 mm. 

Structured-light vision systems are widely used in robotic welding. The key to improving the robotic visual servo performance and weld quality is the weld seam recognition accuracy. Common detection algorithms are likely to be disturbed by the noise of spatter and arc during the welding process. In this paper, a weld seam recognition algorithm is proposed based on structured light vision to overcome this challenge. The core of this method is fully utilizing information of previous frames to process the current frame, which can make weld seam extraction both more robust and effective. The algorithm can be divided into three steps: initial laser center line recognition, online laser center line detection, and weld feature extraction. A Laplacian of Gaussian filter is used for recognizing the laser center line in the first frame. Afterwards, an algorithm based on the NURBS-snake model detects the laser center line online in a dynamic region of interest (abbreviated ROI). The center line obtained from first step is set as the initial contour of the NURBS-snake model. Using the line obtained from the previous step, feature points are determined by segmentation and straight-line fitting, while the position of the weld seam can be calculated according to the feature points. The accuracy, efficiency and robustness of the recognition algorithm are verified by experiments.

A robust weld seam recognition method under heavy noise based on structured-light vision

In many manufacturing processes (e.g., welding, spraying, coating adhesive), the control for the velocity in the main direction heavily affects operation quality. In addition to traditional manual operations, the industrial robot with a tracking system is capable of accurate and stable velocity control. In this paper, an intelligent robot tracking system is designed for implementing an appropriate velocity control and improving the performance of an autonomous system with online structured light vision tracking. For this aim, an effective tracking algorithm is proposed based on position-based visual servoing (PBVS), and motion compensation is implemented according to both detected path and taught path. To improve the adaptability of the system, a Fuzzy-PI double-layer controller is developed, which adjusts the movement of the end effector in both cases of large and small deviation. Welding experiments demonstrate the effectiveness of the proposed vision tracking system.

Fuzzy-PI double-layer stability control of an online vision-based tracking system

In this paper, a method for welding track correction based on emulational laser and trajectory are introduced. The proposed method is divided into two parts: seam tracking and trajectory correction. In the seam tracking method, by using the prior information of emulational laser stripes which are generated by the simulation software and affine transformation of emulational laser stripes, the real seam point can be detected. And then the trajectory correction method mainly consists of three steps: pre-processing, coarse matching and presice matching by using Iterative Closest Point Matching (ICP). For various conditions and workpieces, the corresponding experiments are conducted in this paper. Experimental results demonstrate that the method can meet the requirement of the internal seams tracking of workpiece and accuracy of welding track correction.

A Method for Welding Track Correction Based on Emulational Laser and Trajectory.

Force Tracking Impedance Control Based on Contour Following Algorithm

Coded structured light based 3D reconstruction is one of the most reliable methods to recover a scene. The errors in the reconstruction are mainly caused by indirect illumination. So it is necessary to suppress the effect of indirect illumination to improve the reconstruction accuracy. In this paper, a simulation method of coded structured light based 3D reconstruction is presented. A simulated point cloud with high accuracy is generated for the noise reduction for the actual point cloud. A down-sampling method based on edge detection is also proposed to improve the efficiency of loading and transporting a point cloud. The experiment results verify the performance of simulation and down-sampling.

Kaifan Zhong

Papers

A robust weld seam recognition method under heavy noise based on structured-light vision

Fuzzy-PI double-layer stability control of an online vision-based tracking system

A Method for Welding Track Correction Based on Emulational Laser and Trajectory.

Force Tracking Impedance Control Based on Contour Following Algorithm

A Brief Simulation Method for Coded Structured Light Based 3D Reconstruction