Multi-mode data augmentation and fault diagnosis of rotating machinery using modified ACGAN designed with new framework

The development of information technology and process technology have been enhanced the rapid changes in high-tech products and smart manufacturing, specifications become more sophisticated. Large amount of sensors are installed to record equipment condition during the manufacturing process. In particular, the characteristics of sensor data are temporal. Most the existing approaches for time series classification are not applicable to adaptively extract the effective feature from a large number of sensor data, accurately detect the fault, and provide the assignable cause for fault diagnosis. This study aims to propose a multiple time-series convolutional neural network (MTS-CNN) model for fault detection and diagnosis in semiconductor manufacturing. This study incorporates data augmentation with sliding window to generate amounts of subsequences and thus to enhance the diversity and avoid over-fitting. The key features of equipment sensor can be learned automatically through stacked convolution-pooling layers. The importance of each sensor is also identified through the diagnostic layer in the proposed MTS-CNN. An empirical study from a wafer fabrication was conducted to validate the proposed MTS-CNN and compare the performance among the other multivariate time series classification methods. The experimental results demonstrate that the MTS-CNN can accurately detect the fault wafers with high accuracy, recall and precision, and outperforms than other existing multivariate time series classification methods. Through the output value of the diagnostic layer in MTS-CNN, we can identify the relationship between each fault and different sensors and provider valuable information to associate the excursion for fault diagnosis.

Multiple time-series convolutional neural network for fault detection and diagnosis and empirical study in semiconductor manufacturing

Deep learning techniques, especially Convolutional Neural Networks (CNN), dominate the benchmarks for most computer vision tasks. These state-of-the-art results are typically obtained through supervised learning, for which large annotated datasets are required. However, acquiring such datasets for manufacturing applications remains a challenging proposition due to the time and costs involved in their collection. To overcome this disadvantage, a novel framework is proposed for data augmentation by creating synthetic images using Generative Adversarial Networks (GANs). The generator synthesizes new surface defect images from random noise which is trained over time to get realistic fakes. These synthetic images can be used further for training of classification algorithms. Three GAN architectures are trained, and the entire data augmentation pipeline is implemented for the Northeastern University (China) Classification (NEU-CLS) dataset for hot-rolled steel strips from NEU Surface Defect Database. The classification accuracy of a simple CNN architecture is measured on synthetic augmented data and further it is compared with similar state-of-the-arts. It is observed that the proposed GANs-based augmentation scheme significantly improves the performance of CNN for classification of surface defects. The classically augmented CNN yields sensitivity and specificity of 90.28% and 98.06% respectively. In contrast, the synthetically augmented CNN yields better results, with sensitivity and specificity of 95.33% and 99.16% respectively. Also, the use of GANs is demonstrated to disentangle the representation space and to add additional domain knowledge through synthetic augmentation that can be difficult to replicate through classic augmentation. The proposed framework demonstrates high generalization capability. It may be applied to other supervised surface inspection tasks, and thus facilitate the development of advanced vision-based inspection instruments for manufacturing applications.

Synthetic data augmentation for surface defect detection and classification using deep learning

The industrial advancement has promoted the development of deep learning (DL)-based intelligent fault diagnosis methods for condition-based maintenance (CBM). Though these methods rely on large dataset for training, the collection of large number of fault samples is not practically feasible. For this purpose, generative adversarial networks (GANs) are capable to generate high-quality synthetic samples. However, the problem still persists with the training of GAN using limited fault samples that are present in practical conditions. This article proposes a novel conditional auxiliary classifier GAN framework coupled with model agnostic meta learning (MAML) to resolve this problem. The objective is to initialize and update the network parameters using MAML instead of regular stochastic gradient learning. This modification enables GAN to learn the task of synthetic sample generation using the limited training dataset. The effectiveness of the proposed framework has been compared with several famous state-of-the-art intelligent fault diagnosis methods existing in the literature. The comparative performance has been validated on benchmarked datasets, i.e., air compressor and bearing datasets collected from a single-stage reciprocating air compressor. The proposed framework is able to achieve the classification accuracy of 99.26% and 98.55% for bearing and air compressor datasets, respectively, with only ten samples per class. Moreover, a real-time case study is performed to validate the proposed method in real time.

Intelligent Fault Diagnosis of Rotary Machines: Conditional Auxiliary Classifier GAN Coupled With Meta Learning Using Limited Data

Prognostics and Health Management (PHM), including monitoring, diagnosis, prognosis, and health management, occupies an increasingly important position in reducing costly breakdowns and avoiding catastrophic accidents in modern industry. With the development of artificial intelligence (AI), especially deep learning (DL) approaches, the application of AI-enabled methods to monitor, diagnose and predict potential equipment malfunctions has gone through tremendous progress with verified success in both academia and industry. However, there is still a gap to cover monitoring, diagnosis, and prognosis based on AI-enabled methods, simultaneously, and the importance of an open source community, including open source datasets and codes, has not been fully emphasized. To fill this gap, this paper provides a systematic overview of the current development, common technologies, open source datasets, codes, and challenges of AI-enabled PHM methods from three aspects of monitoring, diagnosis, and prognosis.

/pdf/challenges-and-opportunities-of-ai-enabled-monitoring-43p18hy8jm.pdf

Challenges and Opportunities of AI-Enabled Monitoring, Diagnosis & Prognosis: A Review

Due to the real working conditions, the collected mechanical fault datasets are actually limited and always highly imbalanced, which restricts the diagnosis accuracy and stability. To solve these problems, we present an imbalanced fault diagnosis method based on the generative model of conditional-deep convolutional generative adversarial network (C-DCGAN) and provide a study in detail. Deep convolutional generative adversarial network (DCGAN), based on traditional generative adversarial networks (GAN), introduces convolutional neural network into the training for unsupervised learning to improve the effect of generative networks. Conditional generative adversarial network (CGAN) is a conditional model obtained through introducing conditional extension into GAN. C-DCGAN is a combination of DCGAN and CGAN. In C-DCGAN, based on the feature extraction ability of convolutional networks, through the structural optimization, conditional auxiliary generative samples are used as augmented data and applied in machine fault diagnosis. Two datasets (Bearing dataset and Planetary gear box dataset) are carried out to validate. The simulation experiments showed that the improved performance is mainly due to the generated signals from C-DCGAN to balance the dataset. The proposed method can deal with imbalanced fault classification problem much more effectively. This model could improve the accuracy of fault diagnosis and the generalization ability of the classifier in the case of small samples and display better fault diagnosis performance.

A case study of conditional deep convolutional generative adversarial networks in machine fault diagnosis

In view of the limitations of existing rotating machine fault diagnosis methods in single-scale signal analysis, a fault diagnosis method based on multi-scale permutation entropy (MPE) and multi-channel fusion convolutional neural networks (MCFCNN) is proposed. First, MPE quantitatively analyzes the vibration signals of rotating machine at different scales, and obtains permutation entropy (PE) to construct feature vector sets. Then, considering the structure and spatial information between different sensor measurement points, MCFCNN constructs multiple channels in the input layer according to the number of sensors, and each channel corresponds to the MPE feature sets of different monitored points. MCFCNN uses convolutional kernels to learn the features of each channel in an unsupervised way, and fuses the features of each channel into a new feature map. At last, multi-layer perceptron is applied to fuse multi-channel features and identify faults. Through the health monitoring experiment of planetary gearbox and rolling bearing, and compared with single channel convolutional neural networks (CNN) and existing CNN based fusion methods, the proposed method based on MPE and MCFCNN model can diagnose faults with high accuracy, stability, and speed.

https://www.mdpi.com/1099-4300/22/8/851/pdf

Fault Diagnosis for Rotating Machinery Using Multiscale Permutation Entropy and Convolutional Neural Networks.

Generative models have been applied in many fields and can be evaluated with many methods. In the evaluation of generative models, the proper evaluation metric varies with the application field. Th...

An evaluation method of conditional deep convolutional generative adversarial networks for mechanical fault diagnosis

Deep neural network (DNN), with the capacity for feature inference and nonlinear mapping, has demonstrated its effectiveness in end-to-end fault diagnosis. However, the intermediate learning process of the DNN architecture is invisible, making it an uninterpretable black-box model. In this paper, a stacked residual multi-attention network (SRMANet) is proposed as a means of feature extraction of vibration signals, and visualizing the model training process, designing Squeeze-excitation residual (SE-Res) blocks to obtain additive features with minimal redundancy and sparsity. This study recommends the use of the attention fusion unit to ensure the interpretability of the model and ultimately to obtain representative features. By feeding the output gradient of the attention layer back to the original signal, the key feature components in the time domain signal can be effectively captured. Finally, the interpretability, identification accuracy and adaptability of the model under different operating conditions are verified on 12 different fault tasks in the planetary gearbox.

/pdf/srmanet-toward-an-interpretable-neural-network-with-multi-3p5e6o8l.pdf

SRMANet: Toward an Interpretable Neural Network with Multi-Attention Mechanism for Gearbox Fault Diagnosis

Ultrasound Diagnosis of Breast Masses Based on a Continuous Deep Learning Optimized Ensemble Approach: A Simulated Prospective Study and Multi-Institutional Data Validation

Jia Luo

Papers

A case study of conditional deep convolutional generative adversarial networks in machine fault diagnosis

Fault Diagnosis for Rotating Machinery Using Multiscale Permutation Entropy and Convolutional Neural Networks.

An evaluation method of conditional deep convolutional generative adversarial networks for mechanical fault diagnosis

SRMANet: Toward an Interpretable Neural Network with Multi-Attention Mechanism for Gearbox Fault Diagnosis

Ultrasound Diagnosis of Breast Masses Based on a Continuous Deep Learning Optimized Ensemble Approach: A Simulated Prospective Study and Multi-Institutional Data Validation