Machine vision intelligence for product defect inspection based on deep learning and Hough transform

The detection of product defects is essential in quality control in manufacturing. This study surveys stateoftheart deep-learning methods in defect detection. First, we classify the defects of products, such as electronic components, pipes, welded parts, and textile materials, into categories. Second, recent mainstream techniques and deep-learning methods for defects are reviewed with their characteristics, strengths, and shortcomings described. Third, we summarize and analyze the application of ultrasonic testing, filtering, deep learning, machine vision, and other technologies used for defect detection, by focusing on three aspects, namely method and experimental results. To further understand the difficulties in the field of defect detection, we investigate the functions and characteristics of existing equipment used for defect detection. The core ideas and codes of studies related to high precision, high positioning, rapid detection, small object, complex background, occluded object detection and object association, are summarized. Lastly, we outline the current achievements and limitations of the existing methods, along with the current research challenges, to assist the research community on defect detection in setting a further agenda for future studies.

Using Deep Learning to Detect Defects in Manufacturing: A Comprehensive Survey and Current Challenges.

Unsupervised representation learning has proved to be a critical component of anomaly detection/localization in images. The challenges to learn such a representation are two-fold. Firstly, the sample size is not often large enough to learn a rich generalizable representation through conventional techniques. Secondly, while only normal samples are available at training, the learned features should be discriminative of normal and anomalous samples. Here, we propose to use the "distillation" of features at various layers of an expert network, pre-trained on ImageNet, into a simpler cloner network to tackle both issues. We detect and localize anomalies using the discrepancy between the expert and cloner networks' intermediate activation values given the input data. We show that considering multiple intermediate hints in distillation leads to better exploiting the expert's knowledge and more distinctive discrepancy compared to solely utilizing the last layer activation values. Notably, previous methods either fail in precise anomaly localization or need expensive region-based training. In contrast, with no need for any special or intensive training procedure, we incorporate interpretability algorithms in our novel framework for the localization of anomalous regions. Despite the striking contrast between some test datasets and ImageNet, we achieve competitive or significantly superior results compared to the SOTA methods on MNIST, F-MNIST, CIFAR-10, MVTecAD, Retinal-OCT, and two Medical datasets on both anomaly detection and localization.

Multiresolution Knowledge Distillation for Anomaly Detection

Fabric defect detection is an intriguing but challenging topic. Many methods have been proposed for fabric defect detection, but these methods are still suboptimal due to the complex diversity of both fabric textures and defects. In this paper, we propose a generative adversarial network (GAN)-based framework for fabric defect detection. Considering existing challenges in real-world applications, the proposed fabric defect detection system is capable of learning existing fabric defect samples and automatically adapting to different fabric textures during different application periods. Specifically, we customize a deep semantic segmentation network for fabric defect detection that can detect different defect types. Furthermore, we attempted to train a multistage GAN to synthesize reasonable defects in new defect-free samples. First, a texture-conditioned GAN is trained to explore the conditional distribution of defects given different texture backgrounds. Given a novel fabric, we aim to generate reasonable defective patches. Then, a GAN-based fusion network fuses the generated defects to specific locations. Finally, the well-trained multistage GAN continuously updates the existing fabric defect datasets and contributes to the fine-tuning of the semantic segmentation network to better detect defects under different conditions. Comprehensive experiments on various representative fabric samples are conducted to verify the detection performance of our proposed method.

Multistage GAN for Fabric Defect Detection

Automated surface inspection is an important task in many manufacturing industries and often requires machine learning driven solutions. Supervised approaches, however, can be challenging, since it is often difficult to obtain large amounts of labeled training data. In this work, we instead perform one-class unsupervised learning on fault-free samples by training a deep convolutional neural network to complete images whose center regions are cut out. Since the network is trained exclusively on fault-free data, it completes the image patches with a fault-free version of the missing image region. The pixel-wise reconstruction error within the cut out region is an anomaly image which can be used for anomaly detection. Results on surface images of decorated plastic parts demonstrate that this approach is suitable for detection of visible anomalies and moreover surpasses all other tested methods.

Anomaly Detection Using Deep Learning Based Image Completion

Fabric defect detection is a necessary and essential step of quality control in the textile manufacturing industry. Traditional fabric inspections are usually performed by manual visual methods, which are low in efficiency and poor in precision for long-term industrial applications. In this paper, we propose an unsupervised learning-based automated approach to detect and localize fabric defects without any manual intervention. This approach is used to reconstruct image patches with a convolutional denoising autoencoder network at multiple Gaussian pyramid levels and to synthesize detection results from the corresponding resolution channels. The reconstruction residual of each image patch is used as the indicator for direct pixel-wise prediction. By segmenting and synthesizing the reconstruction residual map at each resolution level, the final inspection result can be generated. This newly developed method has several prominent advantages for fabric defect detection. First, it can be trained with only a small amount of defect-free samples. This is especially important for situations in which collecting large amounts of defective samples is difficult and impracticable. Second, owing to the multi-modal integration strategy, it is relatively more robust and accurate compared to general inspection methods (the results at each resolution level can be viewed as a modality). Third, according to our results, it can address multiple types of textile fabrics, from simple to more complex. Experimental results demonstrate that the proposed model is robust and yields good overall performance with high precision and acceptable recall rates.

Automatic Fabric Defect Detection with a Multi-Scale Convolutional Denoising Autoencoder Network Model

Semiconductor wafer is widely used in welding robot, spray robot, unmanned material delivery vehicle, and detection station sensor The defects of semiconductor wafer, such as stains, burrs, scratches, and holes generated in the manufacturing process, severely affect the quality of downstream products Therefore, the inspection of wafer defect could not be neglected Traditional semiconductor wafer defect inspection methods based on handcrafted features heavily rely on the expertise and are limited in some application scenario In this article, a novel method based on deep convolutional neural networks for semiconductor wafer surface defect inspection is proposed First, a new structure of feature pyramid networks with atrous convolution (FPNAC) is developed to extract the features and to generate feature maps Second, the feature maps are fed into region proposal network (RPN) to generate region proposals Finally, the region proposals are aligned to corresponding size as the inputs of deep multibranches neural network (DMBNN) consisting of three branches, to classify and segment the defects precisely Experimental results demonstrate that the proposed method yields good comprehensive performance with mean pixel accuracy (MPA) 9397% and mean intersection over union (MIoU) 8358%

A Novel Method Based on Deep Convolutional Neural Networks for Wafer Semiconductor Surface Defect Inspection

The drilling length is an important parameter in the process of horizontal directional drilling (HDD) exploration and recovery, but there has been a lack of accurate, automatically obtained statistics regarding this parameter. Herein, a technique for real-time HDD length detection and a management system based on the electromagnetic detection method with a microprocessor and two magnetoresistive sensors employing the software LabVIEW are proposed. The basic principle is to detect the change in the magnetic-field strength near a current coil while the drill stem and drill-stem joint successively pass through the current coil forward or backward. The detection system consists of a hardware subsystem and a software subsystem. The hardware subsystem employs a single-chip microprocessor as the main controller. A current coil is installed in front of the clamping unit, and two magneto resistive sensors are installed on the sides of the coil symmetrically and perpendicular to the direction of movement of the drill pipe. Their responses are used to judge whether the drill-stem joint is passing through the clamping unit; then, the order of their responses is used to judge the movement direction. The software subsystem is composed of a visual software running on the host computer and a software running in the slave microprocessor. The host-computer software processes, displays, and saves the drilling-length data, whereas the slave microprocessor software operates the hardware system. A combined test demonstrated the feasibility of the entire drilling-length detection system.

Horizontal Directional Drilling-Length Detection Technology While Drilling Based on Bi-Electro-Magnetic Sensing.

Increasing deployment of optical fiber networks and the need for reliable high bandwidth make the task of inspecting optical fiber connector end faces a crucial process that must not be neglected. Traditional end face inspections are usually performed by manual visual methods, which are low in efficiency and poor in precision for long-term industrial applications. More seriously, the inspection results cannot be quantified for subsequent analysis. Aiming at the characteristics of typical defects in the inspection process for optical fiber end faces, we propose a novel method, “difference of min-max ranking filtering” (DO2MR), for detection of region-based defects, e.g., dirt, oil, contamination, pits, and chips, and a special model, a “linear enhancement inspector” (LEI), for the detection of scratches. The DO2MR is a morphology method that intends to determine whether a pixel belongs to a defective region by comparing the difference of gray values of pixels in the neighborhood around the pixel. The LEI is also a morphology method that is designed to search for scratches at different orientations with a special linear detector. These two approaches can be easily integrated into optical inspection equipment for automatic quality verification. As far as we know, this is the first time that complete defect detection methods for optical fiber end faces are available in the literature. Experimental results demonstrate that the proposed DO2MR and LEI models yield good comprehensive performance with high precision and accepted recall rates, and the image-level detection accuracies reach 96.0 and 89.3%, respectively.

Automated Inspection of Defects in Optical Fiber Connector End Face Using Novel Morphology Approaches.

Numerous construction projects have caused the rapid development of the drilling industry in China. Various types of drilling equipment are widely used in the projects and have to be designed quick...

https://journals.sagepub.com/doi/pdf/10.1177/1687814016673497

Yudan Wang

Papers

Automatic Fabric Defect Detection with a Multi-Scale Convolutional Denoising Autoencoder Network Model

A Novel Method Based on Deep Convolutional Neural Networks for Wafer Semiconductor Surface Defect Inspection

Horizontal Directional Drilling-Length Detection Technology While Drilling Based on Bi-Electro-Magnetic Sensing.

Automated Inspection of Defects in Optical Fiber Connector End Face Using Novel Morphology Approaches.

Automated three-dimensional design of the chuck of core drilling rigs: