This letter illustrates how a challenging problem be solved by assuring the adequacy of the raw information and using an appropriate deep learning network to extract the relevant information. The problem concerned is accurate monitoring of the penetration, in a fully penetrated weld pool, as quantified by the width of the weld on the back-side of the workpiece. This is challenging as the penetration occurs below the workpiece surface and is not visible. A popular method is to use a weld pool image to derive it. Analysis of the physical process suggests that a single weld pool does not contain adequate information but most recent serial weld pools may. As such, although a deep learning model may extract information that is already there, the raw information may not be sufficient. Hence, a model that is capable of extracting information from dynamic serial weld pool images is needed. To this end, a CNN-LSTM (convolutional neural network combined with long-short term memory one) model is proposed. Dynamic weld pools are experimentally generated by changing the welding current and welding speed randomly. The weld pools are imaged using an HDR camera during experiments. Images are also captured from the back-side surface of the workpiece to provide the ground truth for training, validation, and testing. It is found that the highly dynamically changing weld pool can be accurately predicted using serial weld pool images at 0.3 mm for its back-side bead width. Comparison has been made with results from comparative studies to verify the effectiveness of and the contribution from the information adequacy (by using serial images) and the feature extracting capability (by using deep learning).

How to Accurately Monitor the Weld Penetration From Dynamic Weld Pool Serial Images Using CNN-LSTM Deep Learning Model?

To efficiently integrate deep learning (DL) and Internet of Things (IoT) with tennis robot research, the concept and characteristics of mask region-convolutional neural network (Mask R-CNN) under DL are analyzed. Then, the IoT radio frequency identification (RFID) is introduced and the suitable RFID structure is established. Moreover, the real-time intelligent image recognition tennis robot is designed, and simulation experiment on the robot is performed. The results show that when the positioning label is eight, the convergence speed of the optimized algorithm is improved relative to the unoptimized one, and the error is reduced by 0.82. The positioning algorithm proposed in this research has high convergence speed and small system error. The positioning accuracy of the tennis robot is improved by at least 6%, the positioning targets are close to the target to be located, and the tennis robot has the best shortest path effect. In addition, the tennis recognition algorithm based on Mask R-CNN can accurately distinguish dense tennis balls with high accuracy. It shows that the tennis robot positioning algorithm and target recognition algorithm proposed in this research are of practical adoption value.

/pdf/tennis-robot-design-via-internet-of-things-and-deep-learning-j6q4e4kyyq.pdf

Tennis Robot Design via Internet of Things and Deep Learning

To solve the current problem of poor weld formation due to groove width variation in swing arc narrow gap welding, an infrared passive visual sensing detection approach was developed in this work to measure groove width under intense welding interferences. This approach, called global pattern recognition, includes self-adaptive positioning of the ROI window, equal division thresholding and in situ dynamic clustering algorithms. Accordingly, the self-adaptive positioning method filters several of the nearest values of the arc’s highest point of the vertical coordinate and groove’s same-side edge position to determine the origin coordinates of the ROI window; the equal division thresholding algorithm then divides and processes the ROI window image to extract the groove edge and forms a raw data distribution of groove width in the data window. The in situ dynamic clustering algorithm dynamically classifies the preprocessed data in situ and finally detects the value of the groove width from the remaining true data. Experimental results show that the equal division thresholding algorithm can effectively reduce the influences of arc light and welding fume on the extraction of the groove edge. The in situ dynamic clustering algorithm can avoid disturbances from simulated welding spatters with diameters less than 2.19 mm, thus realizing the high-precision detection of the actual groove width and demonstrating stronger environmental adaptability of the proposed global pattern recognition approach.

/pdf/infrared-visual-sensing-detection-of-groove-width-for-swing-36k1otpk.pdf

Infrared Visual Sensing Detection of Groove Width for Swing Arc Narrow Gap Welding

Vision-based welding seam tracking is an important and unique branch of welding automation. Active vision seam tracking systems achieve accurate feature extraction by using an auxiliary light source, but this will introduce extra costs and the real-time performance will be affected. In contrast, passive vision systems achieve better real-time performance and their structure is relatively simple. This paper proposes a passive vision welding seam tracking system in Plasma Arc Welding (PAW) based on semantic segmentation. The BiseNetV2 network is adopted in this paper and online hard example mining (OHEM) is used to improve the segmentation effect. This network structure is a lightweight structure allowing effective image feature extraction. According to the segmentation results, the offset between the welding seam and the welding torch can be calculated. The results of the experiments show that the proposed method can achieve 57 FPS and the average error of the offset calculation is within 0.07 mm, meaning it can be used for real-time seam tracking.

/pdf/a-seam-tracking-method-based-on-an-image-segmentation-deep-q57m76fb.pdf

A Seam Tracking Method Based on an Image Segmentation Deep Convolutional Neural Network

Joining processes play a critical role across several industries and manufacturing activities. One such activity is narrow gap TIG arc welding, present in a number of high value manufacturing domains the quality and outcomes are highly dependent on the operating parameters and the evolving state of the weld. Human supervision, and the application of vision systems to allow the operator to monitor the process as it is being undertaken is commonplace, however monitoring, characterisation and feedback in narrow gap use cases is less so. The work undertakes a comparative study into two different approaches to process monitoring of TIG arc welding. The first through more traditional machine vision (MV) methods, and the second through more modern deep neural network methods. Results show both methods developed are able to perform the role of process monitoring and defect detection with accuracy of 82% and 93% for the traditional and DNN methods respectively. 

Narrow gap TIG arc weld process monitoring

In the process of K-TIG deep penetration welding, the workpiece does not need to be bevelled; therefore, the welding method is butt welding, and the gap to be welded is very narrow (0.2~1 mm). Because of the large welding current, the welding arc light intensity is very strong. These factors cause difficulties in the K-TIG weld seam tracking process. To realize weld seam tracking in the K-TIG welding process, it is necessary to extract the keyhole entrance centre and weld centreline accurately. To reduce the interference of strong arc light in the process of K-TIG welding, the information of keyhole entrance and weld seam in the process of K-TIG welding is obtained by using a high-dynamic-range camera. An image processing algorithm based on Mask-RCNN is proposed to extract the centre of the keyhole entrance accurately. An image processing algorithm based on Hough line fitting is used to accurately identify the weld centreline in the welding image and extract the welding deviation. In welding experiments, it is verified that the welding deviation extracted by the method proposed in this paper fluctuates within ± 0.133 mm, which meets the requirements of actual K-TIG welding seam tracking.

Narrow gap deviation detection in Keyhole TIG welding using image processing method based on Mask-RCNN model

The high dynamic range existing in arc welding with high energy density challenges most of the industrial cameras, causing badly exposed pixels in the captured images and bringing difficulty to the feature detection from internal weld pool. This paper proposes a novel monitoring method called adaptive image fusion, which increases the amount of information contained in the welding images and can be realized on the common industrial camera with low cost. It combines original images captured rapidly by the camera into one fused image, and the setting of these images is based on the real-time analysis of realistic scene irradiance during the welding process. Experiments are carried out to find out the operating window for the adaptive image fusion method, providing the rules for getting a fused image with as much as information as possible. The comparison between the imaging with or without the proposed method proves that the fused image has a wider dynamic range and includes more useful features from the weld pool. The improvement is also verified by extracting both the internal and external features of weld pool within a same fused image with proposed method. The results show that the proposed method can adaptively expand the dynamic range of visual monitoring system with low cost, which benefits the feature extraction from the internal weld pool.

/pdf/a-high-dynamic-range-visual-sensing-method-for-feature-57btbefk5c.pdf

Xiyin Chen

Papers

Narrow gap deviation detection in Keyhole TIG welding using image processing method based on Mask-RCNN model

A high-dynamic-range visual sensing method for feature extraction of welding pool based on adaptive image fusion

Weld pool and keyhole geometric feature extraction in K-TIG welding with a gradual gap based on an improved HDR algorithm

Real-time K-TIG welding penetration prediction on embedded system using a segmentation-LSTM model