“Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告

Any computer vision application development starts off by acquiring images and data, then preprocessing and pattern recognition steps to perform a task. When the acquired images are highly imbalanced and not adequate, the desired task may not be achievable. Unfortunately, the occurrence of imbalance problems in acquired image datasets in certain complex real-world problems such as anomaly detection, emotion recognition, medical image analysis, fraud detection, metallic surface defect detection, disaster prediction, etc., are inevitable. The performance of computer vision algorithms can significantly deteriorate when the training dataset is imbalanced. In recent years, Generative Adversarial Neural Networks (GANs) have gained immense attention by researchers across a variety of application domains due to their capability to model complex real-world image data. It is particularly important that GANs can not only be used to generate synthetic images, but also its fascinating adversarial learning idea showed good potential in restoring balance in imbalanced datasets. In this paper, we examine the most recent developments of GANs based techniques for addressing imbalance problems in image data. The real-world challenges and implementations of synthetic image generation based on GANs are extensively covered in this survey. Our survey first introduces various imbalance problems in computer vision tasks and its existing solutions, and then examines key concepts such as deep generative image models and GANs. After that, we propose a taxonomy to summarize GANs based techniques for addressing imbalance problems in computer vision tasks into three major categories: 1. Image level imbalances in classification, 2. object level imbalances in object detection and 3. pixel level imbalances in segmentation tasks. We elaborate the imbalance problems of each group, and provide GANs based solutions in each group. Readers will understand how GANs based techniques can handle the problem of imbalances and boost performance of the computer vision algorithms.

/pdf/a-survey-on-generative-adversarial-networks-for-imbalance-3ec2b2ie2d.pdf

A survey on generative adversarial networks for imbalance problems in computer vision tasks

GAN-based anomaly detection: A review

Autoencoder-based anomaly detection for surface defect inspection

For a typical surface automated visual inspection (AVI) instrument of planar materials, defect classification is an indispensable part after defect detection, which acts as a crucial precondition for achieving the online quality inspection of end products. In the industrial environment of manufacturing flat steels, this task is awfully difficult due to diverse defect appearances, ambiguous intraclass, and interclass distances. This article attempts to present a focused but systematic review of the traditional and emerging automated computer-vision-based defect classification methods by investigating approximately 140 studies on three specific flat steel products of con-casting slabs, hot-rolled steel strips, and cold-rolled steel strips. According to the natural image processing procedure of defect recognition, the diverse approaches are grouped into five successive parts: image acquisition, image preprocessing, feature extraction, feature selection, and defect classifier. Recent literature has been reviewed from an industrial goal-oriented perspective to provide some guidelines for future studies and recommend suitable methods for boosting the surface quality inspection level of AVI instruments.

/pdf/automated-visual-defect-classification-for-flat-steel-507r5k015q.pdf

Automated Visual Defect Classification for Flat Steel Surface: A Survey

This article aims to improve deep-learning-based surface defect recognition. Owing to the insufficiency of the defect images in practical production lines and the high cost of labeling, it is difficult to obtain a sufficient defect data set in terms of diversity and quantity. A new generation method called surface defect-generation adversarial network (SDGAN), which employs generative adversarial networks (GANs), is proposed to generate defect images using a large number of defect-free images from industrial sites. Experiments show that the defect images generated by the SDGAN have better image quality and diversity than those generated by the state-of-the-art methods. The SDGAN is applied to expand the commutator cylinder surface defect image data sets with and without labels (referred to as the CCSD-L and CCSD-NL data sets, respectively). Regarding anomaly recognition, a 1.77% error rate and a 49.43% relative improvement (IMP) for the CCSD-NL defect data set are obtained. Regarding defect classification, a 0.74% error rate and a 57.47% IMP for the CCSD-L defect data set are achieved. Moreover, defect classification trained on the images augmented by the SDGAN is robust to uneven and poor lighting conditions. Note to Practitioners —This article proposes a method of defect image generation to address the lack of industrial defect images. Traditional defect recognition methods have two disadvantages: different types of defects require different algorithms and handcrafted features are deficient. Defect recognition using deep learning can solve the above problems. However, deep learning requires a plethora of images, and the number of industrial defect images cannot meet this requirement. We propose a new defect image-generation method called SDGAN to generate a defect image data set that balances diversity and authenticity. In practice, we employ a large number of defect-free images to generate a large number of defect images using our method to expand the industry defect-free image data set. Then, the augmented defect data set is used to build a deep-learning defect recognition model. Experiments show that the accuracy of defect recognition can be significantly improved by building a deep-learning defect recognition model using the augmented data set. Therefore, deep learning can achieve excellent performance in defect recognition with a limited number of defect images.

Defect Image Sample Generation With GAN for Improving Defect Recognition

Traditional methods for defect detection applied in industry are complex, time-consuming, not robust and demanding for professional experience due to hand-crafted features extraction and pipeline design. Besides, current deep learning based methods for general object segmentation demand for a large number of region-level human annotations.Instead, we present DefectGAN for defect detection in a weakly-supervised learning, which requires very a few human annotations. In practical application, images in training dataset are merely labeled with two categories: negative and positive. Despite being trained on image-level rather than region-level labels, DefectGAN has remarkable ability of localizing defect regions.DefectGAN can have comparable and visually even better performance than SegNet, a supervised learning method on dataset CCSD-NL and DAGM 2007. The detected regions are more similar to the original defect regions visually and it has the potential of detecting unseen defects.

DefectGAN: Weakly-Supervised Defect Detection using Generative Adversarial Network

The invention belongs to the field of image processing, and specifically discloses a chip defect recognition method based on a convolutional neural network (CNN). The method comprises the following steps of: acquiring respective types of chip images according to preset classification types, and performing size normalization to construct a data set; constructing a CNN initial model including an input layer, a convolution layer, a downsampling layer, a full connection layer and an output layer; randomly extracting images from the constructed data set to construct a training set, and inputting the images in the training set into the constructed CNN initial model for training to obtain a CNN final model; acquiring a chip image to be recognized as the input layer to be directly input into the final CNN final model, and completing the image feature extraction by successively passing the chip image through the convolution layer, the downsampling layer and the full connection layer, and enabling the output layer to output a recognition result so as to complete online chip defect recognition. The method can realize the rapid recognition of chip defects, has high recognition accuracy and high speed.

Shuanlong Niu

Papers

Defect Image Sample Generation With GAN for Improving Defect Recognition

DefectGAN: Weakly-Supervised Defect Detection using Generative Adversarial Network

Chip defect recognition method based on convolutional neural network

Defect Attention Template Generation CycleGAN for Weakly Supervised Surface Defect Segmentation

A novel deep learning motivated data augmentation system based on defect segmentation requirements