A multi-head attention network with adaptive meta-transfer learning for RUL prediction of rocket engines

Virtual sensor-based imputed graph attention network for anomaly detection of equipment with incomplete data

Time series anomaly detection has applications in a wide range of research fields and applications, including manufacturing and healthcare. The presence of anomalies can indicate novel or unexpected events, such as production faults, system defects, or heart fluttering, and is therefore of particular interest. The large size and complex patterns of time series have led researchers to develop specialised deep learning models for detecting anomalous patterns. This survey focuses on providing structured and comprehensive state-of-the-art time series anomaly detection models through the use of deep learning. It providing a taxonomy based on the factors that divide anomaly detection models into different categories. Aside from describing the basic anomaly detection technique for each category, the advantages and limitations are also discussed. Furthermore, this study includes examples of deep anomaly detection in time series across various application domains in recent years. It finally summarises open issues in research and challenges faced while adopting deep anomaly detection models.

Deep Learning for Time Series Anomaly Detection: A Survey

Intelligent fault diagnosis based on deep learning has yielded remarkable progress for its strong feature representation capability in recent years. Nevertheless, in engineering scenarios, machines often operate under normal condition and the probability of various faults is also different, which leads to the extreme class imbalance and long-tailed distribution between different health conditions. Long-tailed distribution usually degrades the performance of the model, since the model tends to focus on dominant classes and exhibits a poor performance on tail classes. To address this problem, numerous researches have been carried out and fruitful achievements have been achieved in recent years. However, a comprehensive summary of the latest achievements is still lacking. In addition, existing studies solely concentrate on the issue of class imbalance, and lack of analysis and future research directions of long-tailed problem. Hence, this paper specifically defines the long-tailed distribution and discusses the relations between long-tailed problem and class imbalance problem. Then, research achievements on class imbalance problems are reviewed. Specifically, we divide the existing research achievements into three categories: class rebalance, information augmentation and module design. Finally, this paper summarizes the challenges of the long-tailed problem in fault diagnosis and provides some future research directions. 

Imbalance fault diagnosis under long-tailed distribution: Challenges, solutions and prospects

A soft-target difference scaling network via relational knowledge distillation for fault detection of liquid rocket engine under multi-source trouble-free samples

Data-driven intelligent models have received more and more attention in liquid rocket engine (LRE) state recognition. However, due to practical limitations, the fault data are rare for model training, which results in poor testing results. To better recognize the state of LRE, we proposed a supervised recognition framework without fault samples. First, we proposed a negative sample generation method. Then, fusion recurrent convolutional neural network (FRCNN) is constructed for data fusion and feature extraction. Finally, we evaluated the recognition results with several quantization indicators. The proposed method is verified with the real monitoring data, which is obtained from the static firing tests of a certain type of LRE. Results show that the FRCNN could recognize the state of LRE with accuracy over 93% under the confidence of 96%, which shows the effectiveness of the proposed framework. And the comparison with the literature method shows the superiority of the proposed framework. In addition, it could also perform real-time monitoring for LRE, and recognize the abnormal at the initial stage of the steady state, which has practical potential in monitoring task. At last, we further discuss and analyze the generated negative sample and the proposed framework.

A Supervised Framework for Recognition of Liquid Rocket Engine Health State Under Steady-State Process Without Fault Samples

To ensure the safety and stability of the rocket, it is essential to implement accurate anomaly detection on key parts such as the liquid rocket engine (LRE). However, due to the indistinct features of signals under the interference of extreme conditions and the weak distinguishing ability to exist unsupervised methods, it is difficult to distinguish abnormal samples from normal samples, which leads to the failure of reliable anomaly detection. Aiming at this problem, this paper proposed an unsupervised learning algorithm named Memory-augmented skip-connected deep autoencoder (Mem-SkipAE) for anomaly detection of rocket engines with multi-source data fusion. Unlike traditional autoencoders, the input embedding for the decoder is not generated by an encoder but by a combination of memory items that record prototypical patterns of normal samples. Besides, each layer of the encoder and decoder has a skip connection to fully extract the multi-scale features of the normal sample in multi-dimensional space and suppress over-fitting caused by the memory-augmented network. Compared with existing methods and ablation control groups, experiments on four test sets prove the excellent generalization and satisfactory performances of the proposed Mem-SkipAE. Furthermore, the comparison of the single-source model and multi-source model verifies the effectiveness of multi-source fusion. 

Memory-augmented skip-connected autoencoder for unsupervised anomaly detection of rocket engines with multi-source fusion.

To achieve reliable and automatic anomaly detection (AD) for large equipment such as liquid rocket engine (LRE), multisource data are commonly manipulated in deep learning pipelines. However, current AD methods mainly aim at single source or single modality, whereas existing multimodal methods cannot effectively cope with a common issue, modality incompleteness. To this end, we propose an unsupervised multimodal method for AD with missing sources in LRE system. The proposed method handles intramodality fusion, intermodality fusion, and decision fusion in a unified framework composed of multiple deep autoencoders (AEs) and a skip-connected AE. Specifically, the first module restores missing sources to construct a complete modality, thus advancing the secondary reconstruction. Different from vanilla reconstruction-based methods, the proposed method minimizes reconstruction loss and meanwhile maximizes the dissimilarity of representations in two latent spaces. Utilizing reconstruction errors and latent representation discrepancy, the anomaly score is acquired. At decision level, the model performance can be further enhanced via anomaly score fusion. To demonstrate the effectiveness, extensive experiments are carried out on multivariate time-series data from static ignition of several LREs. The results indicate the superiority and potential of the proposed method for AD with missing sources for LRE.

Zijun Liu

Papers

A Supervised Framework for Recognition of Liquid Rocket Engine Health State Under Steady-State Process Without Fault Samples

Virtual sensor-based imputed graph attention network for anomaly detection of equipment with incomplete data

Memory-augmented skip-connected autoencoder for unsupervised anomaly detection of rocket engines with multi-source fusion.

Unsupervised Multimodal Anomaly Detection With Missing Sources for Liquid Rocket Engine.

A soft-target difference scaling network via relational knowledge distillation for fault detection of liquid rocket engine under multi-source trouble-free samples