scispace - formally typeset
Search or ask a question

Showing papers in "IEEE Transactions on Instrumentation and Measurement in 2022"


Journal ArticleDOI
TL;DR: Wang et al. as mentioned in this paper proposed a decoupling network-based IVIF method (DNFusion), which utilizes the decoupled maps to design additional constraints on the network and force the network to retain the saliency information of the source image effectively.
Abstract: In general, the goal of existing infrared and visible image fusion (IVIF) methods is to make the fused image contain both the high-contrast regions of the infrared image and the texture details of the visible image. However, this definition would lead the fusion image losing information from the visible image in high-contrast areas. For this problem, this paper proposed a decoupling network-based IVIF method (DNFusion), which utilizes the decoupled maps to design additional constraints on the network to force the network to retain the saliency information of the source image effectively. The current definition of image fusion is satisfied while effectively maintaining the saliency objective of the source images. Specifically, the feature interaction module inside effectively facilitates the information exchange within the encoder and improves the utilization of complementary information. Also, a hybrid loss function constructed with weight fidelity loss, gradient loss, and decoupling loss which ensures the fusion image to be generated to effectively preserves the source image’s texture details and luminance information. The qualitative and quantitative comparison of extensive experiments demonstrates that our model can generate a fused image containing saliency objects and clear details of the source images, and the method we proposed has a better performance than other state-of-the-art methods.

121 citations


Journal ArticleDOI
TL;DR: In this article, a novel enhanced multiscale feature fusion method is proposed, namely, the atrous spatial pyramid pooling-balanced-feature pyramid network (ABFPN), which uses atrous convolution operators with different dilation rates to make full use of context information.
Abstract: Object detection is a well-known task in the field of computer vision, especially the small target detection problem that has aroused great academic attention. In order to improve the detection performance of small objects, in this article, a novel enhanced multiscale feature fusion method is proposed, namely, the atrous spatial pyramid pooling-balanced-feature pyramid network (ABFPN). In particular, the atrous convolution operators with different dilation rates are employed to make full use of context information, where the skip connection is applied to achieve sufficient feature fusions. In addition, there is a balanced module to integrate and enhance features at different levels. The performance of the proposed ABFPN is evaluated on three public benchmark datasets, and experimental results demonstrate that it is a reliable and efficient feature fusion method. Furthermore, in order to validate the applicational potential in small objects, the developed ABFPN is utilized to detect surface tiny defects of the printed circuit board (PCB), which acts as the neck part of an improved PCB defect detection (IPDD) framework. While designing the IPDD, several powerful strategies are also employed to further improve the overall performance, which is evaluated via extensive ablation studies. Experiments on a public PCB defect detection database have demonstrated the superiority of the designed IPDD framework against the other seven state-of-the-art methods, which further validates the practicality of the proposed ABFPN.

114 citations


Journal ArticleDOI
Xue Wang, Zheng Guan, Shishuang Yu, Jinde Cao, Ya Li 
TL;DR: Wang et al. as mentioned in this paper proposed a decoupling network-based IVIF method (DNFusion), which utilizes the decoupled maps to design additional constraints on the network and force the network to retain the saliency information of the source image effectively.
Abstract: In general, the goal of the existing infrared and visible image fusion (IVIF) methods is to make the fused image contain both the high-contrast regions of the infrared image and the texture details of the visible image. However, this definition would lead the fusion image losing information from the visible image in high-contrast areas. For this problem, this article proposed a decoupling network-based IVIF method (DNFusion), which utilizes the decoupled maps to design additional constraints on the network to force the network to retain the saliency information of the source image effectively. The current definition of image fusion is satisfied while effectively maintaining the saliency objective of the source images. Specifically, the feature interaction module (FIM) inside effectively facilitates the information exchange within the encoder and improves the utilization of complementary information. Also, a hybrid loss function constructed with weight fidelity loss, gradient loss, and decoupling loss ensures the fusion image to be generated to effectively preserve the source image’s texture details and luminance information. The qualitative and quantitative comparison of extensive experiments demonstrates that our model can generate a fused image containing saliency objects and clear details of the source images, and the method we proposed has a better performance than other state-of-the-art (SOTA) methods.

86 citations


Journal ArticleDOI
TL;DR: In this paper , a command filter-based adaptive fuzzy finite-time output feedback control (FOFC) is investigated for the Electro-hydraulic servo system, where fuzzy logic systems are used to estimate the uncertain nonlinearities and fuzzy state observer is established to approximate the unmeasurable hydraulic cylinder stem speed and the internal cylinder force.
Abstract: In this paper, the command filter-based adaptive fuzzy finite-time output feedback control (FOFC) is investigated for the Electro-hydraulic servo system. For the uncertainties in the system, we utilize the fuzzy logic systems (FLSs) to estimate these uncertain nonlinearities and fuzzy state observer is established to approximate the unmeasurable hydraulic cylinder stem speed and the internal cylinder force. Then, a command filter-based finite time output feedback control is proposed to achieve high tracking precision, where the tracking errors can be regulated into a small neighborhood around the equilibrium proved by the Lyapunov finite-time stability theory. Moreover, a command filter is introduced to avoid the explosion of complexity in the backstepping procedure, where a compensation mechanism is developed to compensate for filter errors. Finally, simulation results are provided to demonstrate the effectiveness of the proposed control.

83 citations


Journal ArticleDOI
TL;DR: In this article , a separable synchronous (SS) interactive estimation method is proposed to eliminate the coupling parameters and perform the signal modeling algorithm in accordance with the hierarchical principle, which can be used for on-line identification.
Abstract: This article is aimed to study the modeling problems of combinational signals or periodic signals. To overcome the computation complexity of modeling the signals with plenty of characteristic parameters, a parameter separation scheme is developed based on the different characteristic of the signals to be modeled. For the purpose of achieving high-accuracy performance and reducing complexity, two multi-innovation gradient-based iterative (MIGI) subalgorithms are presented by means of gradient search. In terms of the phenomenon that the coupling parameters lead to the inability of algorithms, a separable synchronous (SS) interactive estimation method is proposed to eliminate the coupling parameters and perform the signal modeling algorithm in accordance with the hierarchical principle. By means of simulation experiments, the proposed SS iterative signal modeling algorithm based on the moving batch data is used for estimating a power signal with three sine waves and a periodic square wave signal. The results demonstrate the effectiveness of the proposed method for modeling the combinational signals with multiple frequencies and other periodic signals. Since the proposed method combines real-time data sampling and iterative estimation, it can be used for on-line identification.

72 citations


Journal ArticleDOI
TL;DR: Wang et al. as mentioned in this paper proposed a novel compound fault diagnosis method based on optimized maximum correlation kurtosis deconvolution (MCKD) and sparse representation, namely MDSRCFD, which has better global optimization performance and fast convergence speed.
Abstract: The effective separation of fault characteristic components is the core of compound fault diagnosis of rolling bearings. The intelligent optimization algorithm has better global optimization performance and fast convergence speed. Aiming at the problem of poor diagnosis effect caused by mutual interference between multiple fault responses, a novel compound fault diagnosis method based on optimized maximum correlation kurtosis deconvolution (MCKD) and sparse representation, namely MDSRCFD, is proposed in this article. For the MCKD, because it is very difficult to set reasonable parameter combination values, artificial fish school (AFS) with global search capability and strong robustness is fully utilized to optimize the key parameters of MCKD to achieve the best deconvolution and fault feature separation. Aiming at the problem that orthogonal matching pursuit (OMP) is difficult to be solved in sparse representation, an artificial bee colony (ABC) with global optimization ability and faster convergence speed is employed to solve OMP to obtain the approximate best atom and realize the reconstruction of signal transient components. The envelope demodulation analysis method is applied to realize feature extraction and fault diagnosis. The simulation and practical application results show that the proposed MDSRCFD can effectively separate and extract the compound fault characteristics of rolling bearings, which can realize the accurate compound fault diagnosis.

71 citations


Journal ArticleDOI
TL;DR: In this article , a tapered/etched multicore fiber (MCF) probes are spliced with multimode fiber (MMF) to fabricate the sensor structure, and the functionalization of the acetylcholinesterase enzyme over the NP-immobilized probe increases the specificity of the sensor later on.
Abstract: In this work, tapered/etched multicore fiber (MCF) probes are spliced with multimode fiber (MMF) to fabricate the sensor structure. To improve sensitivity, gold nanoparticles (AuNPs) and molybdenum disulfide nanoparticles (MoS2-NPs) are used to immobilize both probes. Synthesized AuNPs and molybdenum disulfide (MoS2)-nanoparticles (NPs) have peak absorption wavelengths of 519 and 330 nm, respectively. High-resolution transmission electron microscopy is used to examine the morphology of the NPs. The scanning electron microscope (SEM) is used to characterize the NP-immobilized optical fiber sensor structures, and SEM-EDX is used to verify the NP-coating over fiber structure. The functionalization of the acetylcholinesterase enzyme over the NP-immobilized probe increases the specificity of the sensor later on. Finally, the developed sensor probes are tested by detecting various acetylcholine concentrations. In addition, performance analyses, such as reusability, reproducibility, and selectivity (in the presence of ascorbic acid, glucose, dopamine, and uric acid), are carried out, and proposed biosensors are experimentally evaluated. The developed tapered fiber sensor with a sensitivity of 0.062 nm/ $\mu \text{M}$ can detect even very low concentrations, such as 14.28 $\mu \text{M}$ , over a wide detection range of 0–1000 $\mu \text{M}$ .

57 citations


Journal ArticleDOI
TL;DR: Wang et al. as discussed by the authors proposed a dual-scale encoder-decoder architecture with self-attention to enhance the semantic segmentation quality of varying medical images, which can effectively model the non-local dependencies and multi-scale contexts for enhancing the pixellevel intrinsic structural features inside each patch.
Abstract: Automatic medical image segmentation has made great progress owing to the powerful deep representation learning. Inspired by the success of self-attention mechanism in Transformer, considerable efforts are devoted to designing the robust variants of encoder-decoder architecture with Transformer. However, the patch division used in the existing Transformer-based models usually ignores the pixel-level intrinsic structural features inside each patch. In this paper, we propose a novel deep medical image segmentation framework called Dual Swin Transformer U-Net (DS-TransUNet), which aims to incorporate the hierarchical Swin Transformer into both encoder and decoder of the standard U-shaped architecture. Our DS-TransUNet benefits from the self-attention computation in Swin Transformer and the designed dual-scale encoding, which can effectively model the non-local dependencies and multi-scale contexts for enhancing the semantic segmentation quality of varying medical images. Unlike many prior Transformer-based solutions, the proposed DS-TransUNet adopts a well-established dual-scale encoding mechanism that utilizes dual-scale encoders based on Swin Transformer to extract the coarse and fine-grained feature representations of different semantic scales. Meanwhile, a well-designed Transformer Interactive Fusion (TIF) module is proposed to effectively perform the multi-scale information fusion through the self-attention mechanism. Furthermore, we introduce the Swin Transformer block into decoder to further explore the long-range contextual information during the up-sampling process. Extensive experiments across four typical tasks for medical image segmentation demonstrate the effectiveness of DS-TransUNet, and our approach significantly outperforms the state-of-the-art methods.

56 citations


Journal ArticleDOI
TL;DR: In this paper , a command filter-based adaptive fuzzy finite-time output feedback control (FOFC) is investigated for the electrohydraulic servo system, where a fuzzy state observer is established to approximate the unmeasurable hydraulic cylinder stem speed and the internal cylinder force.
Abstract: In this article, the command filter-based adaptive fuzzy finite-time output feedback control (FOFC) is investigated for the electrohydraulic servo system. For the uncertainties in the system, we utilize fuzzy logic systems (FLSs) to estimate these uncertain nonlinearities, and a fuzzy state observer is established to approximate the unmeasurable hydraulic cylinder stem speed and the internal cylinder force. Then, a command filter-based FOFC is proposed to achieve high tracking precision, where the tracking errors can be regulated into a small neighborhood around the equilibrium proved by the Lyapunov finite-time stability theory. Moreover, a command filter is introduced to avoid the explosion of complexity in the backstepping procedure, where a compensation mechanism is developed to compensate for filter errors. Finally, simulation results are provided to demonstrate the effectiveness of the proposed control.

56 citations


Journal ArticleDOI
TL;DR: This study proposes a novel multiclass wind turbine bearing fault diagnosis strategy based on the conditional variational generative adversarial network (CVAE-GAN) model combining multisource signals fusion and shows that the proposed strategy can increase wind turbines bearing fault diagnostic accuracy in complex scenarios.
Abstract: Low fault diagnosis accuracy in case of insufficient and imbalanced samples is a major problem in the wind turbine fault diagnosis. The imbalance of samples refers to the large difference in the number of samples of different categories or the lack of a certain fault sample, which requires good learning of the characteristics of a small number of samples. Sample generation in the deep learning generation model can effectively solve this problem. In this study, we proposed a novel multiclass wind turbine bearing fault diagnosis strategy based on the conditional variational generative adversarial network (CVAE-GAN) model combining multisource signals fusion. This strategy converts multisource 1-D vibration signals into 2-D signals, and the multisource 2-D signals were fused by using wavelet transform. The CVAE-GAN model was developed by merging the variational autoencoder (VAE) with the generative adversarial network (GAN). The VAE encoder was introduced as the front end of the GAN generator. The sample label was introduced as the model input to improve the model’s training efficiency. Finally, the sample set was used to train encoder, generator, and discriminator in the CVAE-GAN model to supplement the number of the fault samples. In the classifier, the sample set is used to do experimental analysis under various sample circumstances. The results show that the proposed strategy can increase wind turbine bearing fault diagnostic accuracy in complex scenarios.

51 citations


Journal ArticleDOI
TL;DR: In this paper , the authors present their latest research findings in the area of sensing technology, measurement methodology, and data analytics approaches in the fast-changing era of artificial intelligence (AI).
Abstract: Nowadays, sensing and measurement are going to play an integral role in artificial intelligence (AI). With the rapid development of intelligent manufacturing and frontier equipment, there is an urgent need for providing a dedicated and unusual forum for researchers, scientists, and engineers to present their latest research findings in the area of sensing technology, measurement methodology, and data analytics approaches in the fast-changing era of AI.

Journal ArticleDOI
TL;DR: A class-imbalance adversarial transfer learning (CIATL) network with input being imbalanced data to learn domain-invariant and knowledge and extends the application of the transfer learning method for real-industrial cross-domain diagnosis tasks.
Abstract: Recently, cross-domain fault diagnosis based on transfer learning methods has been extensively explored and well-addressed when class-balance data with supervision information are available. However, data under machine faulty states are mostly difficult to collect; there is a huge divide between current transfer learning methods based on implicit class-balance data and real industrial applications. In this article, we propose a class-imbalance adversarial transfer learning (CIATL) network with input being imbalanced data to learn domain-invariant and knowledge. Within this framework, class-imbalance learning is embedded into the adversarial training process to learn class-separate diagnostic knowledge with imbalanced data, double-level adversarial transfer learning including marginal and conditional distribution adaptations is conducted to learn domain-invariant knowledge. Extensive experiments on a planetary gearbox rig with imbalanced data verify the effectiveness and generalization of the proposed method and show its superior performance over contrastive transfer learning methods. Moreover, the proposed method relaxes the underlying assumption that the success of current transfer learning regimes is rooted in class-balance data and extends the application of the transfer learning method for real-industrial cross-domain diagnosis tasks.

Journal ArticleDOI
TL;DR: In the proposed deep learning framework, a consistency-based regularization term is added to the objective function to remove the negative effect of missing information in the incomplete target domain dataset.
Abstract: Due to the successful implementation of intelligent data-driven approaches, these methods are gaining remarkable attention in predicting the remaining useful life (RUL) problems. Within this scope, transfer learning approaches are exploited to transfer the obtained knowledge from the source domain data to the target domain data. Due to the different working regimes and operating conditions, there exists a discrepancy between the data distribution of source and target domain datasets. Domain adaptation techniques are deployed to tackle the data distribution discrepancy. In most prognostic problems, it is assumed that the complete life-cycle run-to-failure information for the target domain dataset is available. However, in real-practical scenarios, providing complete life-cycle data is not straightforward. To solve this issue, this article proposed a transfer learning approach for RUL prediction using a consistency-based regularization. In the proposed deep learning framework, a consistency-based regularization term is added to the objective function to remove the negative effect of missing information in the incomplete target domain dataset. In order to further validate the effectiveness of the proposed method, a comprehensive experimental analysis has been done on two different aerospace and bearing datasets.

Journal ArticleDOI
TL;DR: This article provides a comprehensive analysis of machine learning and deep learning algorithms on 70 recordings of the PhysioNet ECG Sleep Apnea v1.0 dataset and found that the best detection performance is achieved by hybrid deep models.
Abstract: Sleep apnea is a common sleep breathing disorder (SBD) in which patients suffer from stopping or decreasing airflow to the lungs for more than 10 sec. Accurate detection of sleep apnea episodes is an important step in devising appropriate therapies and management strategies. This article provides a comprehensive analysis of machine learning and deep learning algorithms on 70 recordings of the PhysioNet ECG Sleep Apnea v1.0.0 dataset. First, electrocardiogram signals were pre-processed and segmented and then machine learning and deep learning methods were applied for sleep apnea detection. Among conventional machine learning algorithms, linear and quadratic discriminate analyses, logistic regression, Gaussian naïve Bayes, Gaussian process, support-vector machines, $k$ -nearest neighbor, decision tree, extra tree, random forest, AdaBoost, gradient boosting, multi-layer perceptron, and majority voting were implemented. Among deep algorithms, convolutional networks (Alex-Net, VGG16, VGG19, ZF-Net), recurrent networks (LSTM, bidirectional LSTM, gated recurrent unit), and hybrid convolutional-recurrent networks were implemented. All networks were similarly modified to handle our biosignal processing task. The available data were divided into a training set to adjust the model parameters, a validation set to adjust hyperparameters, avoid overfitting, and improve the generalizability of the models, and a test set to evaluate the generalizability of the models on unseen data. This procedure was then repeated in a fivefold cross-validation scheme so that all the recordings were once located in the test set. It was found that the best detection performance is achieved by hybrid deep models where the best accuracy, sensitivity, and specificity were 88.13%, 84.26%, and 92.27%, respectively. This study provides valuable information on how different machine learning and deep learning algorithms perform in the detection of sleep apnea and can further be extended toward the detection of other sleep events. Our developed algorithms are publicly available on GitHub.

Journal ArticleDOI
TL;DR: An anomaly detection framework using causal network and feature-attention-based long short-term memory (CN-FA-LSTM) has a stronger interpretability than other commonly used prediction models and the universal applicability of the method is verified.
Abstract: Most of the data-driven satellite telemetry data anomaly detection methods suffer from high false positive rate (FPR) and poor interpretability. To solve the above problems, we propose an anomaly detection framework using causal network and feature-attention-based long short-term memory (CN-FA-LSTM). In our method, a causal network of telemetry parameters is constructed by calculating normalized modified conditional transfer entropy (NMCTE) and optimized by conditional independence tests based on the conditional mutual information (CMI). Then, a CN-FA-LSTM is established to predict telemetry data, and a nonparametric dynamic $k$ -sigma threshold updating method is proposed to set thresholds. A case study on a real satellite demonstrates that anomaly detection using the CN-FA-LSTM and nonparametric dynamic $k$ -sigma threshold updating has an average F1-score of 0.9462 and an FPR of 0.0021, which are better than the baseline methods. Furthermore, CN-FA-LSTM has a stronger interpretability than other commonly used prediction models. Supplementary experiment on two public datasets verifies the universal applicability of our method.

Journal ArticleDOI
TL;DR: An m-D signatures’ dataset covering army crawling, boxing, jumping while holding a gun, army jogging, army marching, and stone-pelting/grenade-throwing is introduced, and a lightweight DCNN model, “DIAT-RadHARNet,” designed for human suspicious activity classification is introduced.
Abstract: Recognizing suspicious human activities is one of the critical requirements for national security considerations. Nowadays, designing the deep convolution neural network (DCNN) models suitable for micro-Doppler (m-D) signature-based human activity classification is rapidly growing. However, high computation cost and a huge number of parameters limit their direct/effective usability in field applications. This article introduces an m-D signatures’ dataset “DIAT- $\mu $ RadHAR” covering army crawling, boxing, jumping while holding a gun, army jogging, army marching, and stone-pelting/grenade-throwing, generated using an $X$ -band continuous wave (CW) radar. This article also introduces a lightweight DCNN model, “DIAT-RadHARNet,” designed for those human suspicious activity classification. To reduce the computation cost and to improve the generalization ability, DIAT-RadHARNet is designed with four design principles: depthwise separable convolutions, channel weighting (CHW) based on the importance, different size filters in the depthwise part, and operating different size kernels on the same input tensor. The network has 213 793 parameters with a total of 55 layers. Our extensive experimental analysis demonstrates that the DIAT-RadHARNet model efficiently classifies the activities with 99.22% accuracy, giving minimal false positive and false negative outcomes. The time complexity of the proposed DCNN model observed during the testing phase is 0.35 s. The same accuracy and time complexity are obtained even at adverse weather conditions, low-lighting environments, and long-range operations.

Journal ArticleDOI
TL;DR: Wang et al. as mentioned in this paper proposed an end-to-end convolutional neural network (CNN) to extract complementary information from multilevel features and detect power lines with different pixel widths and orientations.
Abstract: The existing unmanned aerial vehicle (UAV)-based electric transmission line inspection systems generally adopt manual control and follow the predefined path, which reduces the efficiency and makes a high inspection cost. In this article, a UAV system with advanced embedded processors and binocular visual sensors is developed to generate guidance information from power lines in real-time and achieve automatic transmission line inspection. To realize 3-D autonomous perception of power lines, we first propose an end-to-end convolutional neural network (CNN) to extract complementary information from multilevel features and detect power lines with different pixel widths and orientations. Specifically, multilevel feature aggregation module fuses multilevel features within the same stage by learning the weight vector related to the content. The joint attention (JA) module is proposed to extract rich semantic information and suppress the background noises. Meanwhile, multistage detection results are fused to enhance the robustness of the proposed network. Subsequently, power lines are grouped according to the morphological characteristics of thinning detection results, and 3-D point sets of power lines are constructed based on the epipolar constraint of binocular images. Finally, the target point of current stereo images is generated by projecting 3-D power line points to the horizontal and vertical planes. The few-waypoint trajectory is generated based on continuous target points, and automatic inspection is finished with the proposed real-time motion planning strategy. Experimental results on four datasets show that the proposed power line detection method outperforms other state-of-the-art methods. The developed UAV platform and the proposed autonomous inspection strategy are evaluated in practical environments to validate the robustness and effectiveness.

Journal ArticleDOI
Jin Li, Jianming Zhu, Chang Li, Xun Chen, Bin Yang 
TL;DR: Wang et al. as discussed by the authors proposed a convolution-guided transformer framework for infrared and visible image fusion, which combines the local features of convolutional network and the long-range dependence features of transformer to produce satisfactory fused image.
Abstract: Deep learning has been successfully applied to infrared and visible image fusion due to its powerful ability of feature representation. Existing most deep learning-based infrared and visible image fusion methods mainly utilize pure convolution model or pure transformer model, which leads to that the fused image cannot preserve long-range dependences (global context) and local features simultaneously. To this end, we propose a convolution-guided transformer framework for infrared and visible image fusion (CGTF), which aims to combine the local features of convolutional network and the long-range dependence features of transformer to produce satisfactory fused image. In CGTF, the local features are calculated by convolution feature extraction module (CFEM), and then, the local features are used to guide the transformer feature extraction module (TFEM) to capture the long-range dependences of the image, which can overcome not only the lack of long-range dependences that exist in convolutional fusion methods but also the deficiency of local feature that exists in transformer models. Moreover, the convolution-guided transformer fusion framework can consider the inherent relationship of local feature and long-range dependences due to the alternate use of CFEM and transformer module. In addition, to strengthen local feature propagation, we employ dense connections among CFEMs. Ablation experiments demonstrate the effectiveness of convolution-guided transformer fusion framework and loss function. We employ two datasets to compare our method with other nine methods, which include three traditional methods, five deep learning-based methods, and one transformer-based method. Qualitative and quantitative experiments demonstrate the advantages of our method.

Journal ArticleDOI
TL;DR: In this article , a new deep learning method based on the target gray-level distribution constraint mechanism model was proposed to solve the infrared dim small target detection problem in the complex environment.
Abstract: A new deep learning method based on the target gray-level distribution constraint mechanism model is proposed to solve the infrared dim small target detection problem in the complex environment. First, to solve the uneven distribution of positive and negative samples, the designed smoothness operator is used to suppress the background and enhancement target by measuring the difference in their features in 1D and 2D gradient. Second, an infrared dim small target detection network based on dense feature fusion, namely the DFFIR-net network, is proposed. The DFFIR-net enhances the feature expression of dim small targets by integrating the original features and the smoothness features of gray-level gradient. Also, the DFFIR-net alleviates the problem of sparse feature extraction. Finally, a multiscale 2D Gaussian label generation strategy is proposed. This strategy is critical in supervising the training of DFFIR-net in multi-dimensional Gaussian space, improving the feature exploration ability of the network and detection performance under small training samples. The experimental results show that compared with the existing advanced detection methods, the proposed method has higher accuracy and lower false alarm rates in various complex scenes.

Journal ArticleDOI
TL;DR: A novel multisource domain feature adaptation network (MDFAN) is proposed for bearing fault diagnosis under time-varying working conditions and the comparison results show its robustness and superiority.
Abstract: Intelligent fault diagnosis methods based on domain adaptation (DA) have been extensively employed for tackling domain shift problems, and the basic diagnosis tasks under time-varying working conditions were well achieved. Nevertheless, the existing methods usually focus on learning diagnosis knowledge from single-source domain while ignoring abundant underlying information in multisource domain. In practical scenarios, multiple source domains are available, and there are few studies in fault diagnosis based on multisource DA. To this end, a novel multisource domain feature adaptation network (MDFAN) is proposed for bearing fault diagnosis under time-varying working conditions. The proposed network first uses a feature extractor to learn transferable features from different pairs of source and target domains, and then, a domain-specific distribution alignment module is constructed, which adopts the intra-domain alignment strategy and the inter-domain alignment strategy to reduce the shift between all domain pairs. Meanwhile, considering the classifier prediction disagreement, a classifier alignment module is further designed to relieve the classifier prediction discrepancy and enhance the prediction consistency. Case studies on two real datasets with multiple sources demonstrate the effectiveness of the proposed network, and the comparison results show its robustness and superiority.

Journal ArticleDOI
TL;DR: A series of new handcrafted feature flows (HFFs) are proposed, which can suppress the raw signal noise and thus improve the encoded sequential information for the RUL prediction, and a novel bidirectional LSTM (Bi-LSTM)-based two-stream network is proposed.
Abstract: In industry, prognostics and health management (PHM) is used to improve the system reliability and efficiency. In PHM, remaining useful life (RUL) prediction plays a key role in preventing machine failure and reducing operation cost. Recently, with the development of deep learning technology, long short-term memory (LSTM) and convolutional neural networks (CNNs) are adopted into many RUL prediction approaches, which shows impressive performances. However, existing deep learning-based methods directly utilize raw signals. Since noise widely exists in raw signals, the quality of these approaches’ feature representation is degraded, which degenerates their RUL prediction accuracy. To address this issue, we first propose a series of new handcrafted feature flows (HFFs), which can suppress the raw signal noise and thus improve the encoded sequential information for the RUL prediction. In addition, to effectively integrate our proposed HFFs with the raw input signals, a novel bidirectional LSTM (Bi-LSTM)-based two-stream network is proposed. In this novel two-stream network, three different fusion methods are designed to investigate how to combine both streams’ feature representations in a reasonable way. To verify our proposed Bi-LSTM-based two-stream network, extensive experiments are carried out on the commercial modular aero propulsion system simulation (C-MAPSS) dataset, showing superior performances over state-of-the-art approaches.

Journal ArticleDOI
TL;DR: A supervised bidirectional long short-term memory (SBiLSTM) is proposed for data-driven dynamic soft sensor modeling and outperforms state-of-the-art and traditional deep learning-based soft sensor models.
Abstract: Data-driven soft sensors have been widely adopted in industrial processes to learn hidden knowledge automatically from process data, then to monitor difficult-to-measure quality variables. However, to extract and utilize useful dynamic latent features accurately for efficient quality estimations remains one of the most important research issues in soft sensor modeling. In this article, a supervised bidirectional long short-term memory (SBiLSTM) is proposed for data-driven dynamic soft sensor modeling. The SBiLSTM incorporates extended quality information with a moving window up to $k$ time steps and enhances learning efficiency by bidirectional architecture. With this novel structure, the SBiLSTM can extract and utilize nonlinear dynamic latent information from both process variables and quality variables, then further improve the prediction performance significantly. The effectiveness of the proposed SBiLSTM network-based soft sensor model is demonstrated through two case studies on a debutanizer column process and an industrial wastewater treatment process. Results show that the SBiLSTM outperforms state-of-the-art and traditional deep learning-based soft sensor models.

Journal ArticleDOI
TL;DR: Wang et al. as mentioned in this paper used a lightweight convolutional model in the backbone network and employed a triplet loss function to train the model, which not only improves the matching accuracy, but also satisfies the real-time matching requirements.
Abstract: Even though the deep neural networks have strong feature representation capability and high recognition accuracy in finger vein recognition, the deep models are computationally intensive and poor in timeliness. To address these issues, this article proposes a lightweight algorithm for finger vein image recognition and matching. The proposed algorithm uses a lightweight convolutional model in the backbone network and employs a triplet loss function to train the model, which not only improves the matching accuracy, but also satisfies the real-time matching requirements. In addition, the Mini-region of interest (RoI) and finger vein pattern feature extraction also effectively solve the problems of large amounts of calculation and background noise. Moreover, the present model recognizes new categories based on the feature vector space constructed by the finger vein recognition system, so that new categories can be recognized without retraining the model. The results show that the finger vein recognition and matching algorithm proposed in this article achieves 99.3% and 99.6% in recognition accuracy and 14.2 and 16.5 ms in matching time for the dataset Shandong University Machine Learning and Applications Laboratory-Homologous Multimodal Biometric Traits (SDUMLA-HMT) and Peking University Finger Vein Dataset (PKU-FVD), respectively. These metrics show that our approach is time-saving and more effective than previous algorithms. Compared with the state-of-the-art finger vein recognition algorithm, the proposed algorithm improves 1.45% in recognition accuracy while saving 45.7% in recognition time.

Journal ArticleDOI
TL;DR: In this paper , a hypersensitive multispectral partial discharge (PD) optical sensor array was developed, by which the optical pulses in seven independent bands can be acquired simultaneously.
Abstract: In this article, a hypersensitive multispectral partial discharge (PD) optical sensor array was developed, by which the optical pulses in seven independent bands can be acquired simultaneously. By using this sensor array, the multispectral pulses for three typical PDs in gas insulated system were obtained experimentally and analyzed with phase-based (phase-resolved) and nonphase-based (spectral-ratio-based) multispectral characteristics, respectively. It indicates that the multispectral characteristics produced by a specific discharge defect provide unique spectral signatures in discharge mode as well as stage evolution. Based on the intrinsic relationship between the discharge feature and optical emission spectrum, we adopted the classification algorithms and spectral-ratio-reserved multispectral characteristics to implement pattern recognition as well as assessment on the three typical PDs, which obtained the hit ratios exceeding 91%. In principle, such detection approach also supports the phase-independent PD diagnosis especially for dc power equipment.

Journal ArticleDOI
TL;DR: The experimental results show that the proposed heterogeneous knowledge distillation network with multilayer attention embedding preserves the image information of both visible and infrared modalities, achieves sound visual effects, and displays accurate and natural texture details.
Abstract: Recently, infrared–visible image fusion has attracted more and more attention, and numerous excellent methods in this field have emerged. However, when the low-resolution images are being fused, most fusion results are of low resolution, limiting the practical application of the fusion results. Although some methods can simultaneously realize the fusion and super-resolution of low-resolution images, the improvement of fusion performance is limited due to the lack of guidance of high-resolution fusion results. To address this issue, we propose a heterogeneous knowledge distillation network (HKDnet) with multilayer attention embedding to jointly implement the fusion and super-resolution of infrared and visible images. Precisely, the proposed method consists of a high-resolution image fusion network (teacher network) and a low-resolution image fusion and super-resolution network (student network). The teacher network mainly fuses the high-resolution input images and guides the student network to obtain the ability of joint implementation of fusion and super-resolution. In order to make the student network pay more attention to the texture details of the visible input image, we designed a corner embedding attention mechanism. The mechanism integrates channel attention, position attention, and corner attention to highlight the visible image’s edge, texture, and structure. For the input infrared image, the dual-frequency attention (DFA) is constructed by mining the relationship of interlayer features to highlight the role of salient targets of the infrared image in the fusion result. The experimental results show that compared with the existing methods, the proposed method preserves the image information of both visible and infrared modalities, achieves sound visual effects, and displays accurate and natural texture details. The code of the proposed method can be available at https://github.com/firewaterfire/HKDnet.

Journal ArticleDOI
TL;DR: A comparative analysis with state-of-the-art (SOTA) DCNN models evidences the betterness, more efficiency, and more accuracy of the novel “DIAT-RadSATNet” architecture.
Abstract: Due to the smaller size, low cost, and easy operational features, small unmanned aerial vehicles (SUAVs) become more popular for various defense as well as civil applications. They can also give threat to national security if intentionally operated by any hostile actor(s). Since all the SUAV targets have a high degree of resemblances in their micro-Doppler (m-D) space, their accurate detection/classification can be highly guaranteed by the appropriate deep convolutional neural network (DCNN) architecture. In this work, a lightweight novel DCNN model (named “DIAT-RadSATNet”) is designed for the accurate SUAV targets: RC plane, three-short-blade rotor, three-long-blade rotor, quadcopter, bionic bird, and mini-helicopter + bionic bird; and detection/classification based on their m-D signatures. A diversified, $X$ -band (10 GHz) continuous-wave (CW) radar-based, open-field-collected m-D signatures dataset (named “DIAT- $\mu $ SAT”) is used for the design/testing of “DIAT-RadSATNet.” A set of new design principles is proposed through multifactors: layers, #parameters, floating-point operations (FLOPs), number of blocks, filter dimension, memory size, number of parallel paths, and accuracy; optimization is applied via a series of in-depth ablation studies. The novel “DIAT-RadSATNet” module consists of 0.45 M trainable parameters, 40 layers, 2.21-Mb memory size, 0.59G FLOPs, and 0.21-s computation-time complexity. The detection/classification accuracy of “DIAT-RadSATNet,” based on the open-field unknown dataset experiments, falls within 97.1% and 97.3%. A comparative analysis with state-of-the-art (SOTA) DCNN models evidences the betterness, more efficiency, and more accuracy of our novel “DIAT-RadSATNet” architecture.

Journal ArticleDOI
TL;DR: The efficient stair pyramid (ESP) is a further improvement of feature pyramid network (FPN)-based network, which can adaptively select the best detection receptive field according to different scales of targets, thereby improving the detection performance of tiny targets.
Abstract: Defect detection is to locate and classify the possible defects in an image, which plays a key role in the quality inspection link in the manufacturing process of industrial products. Defects in industrial products are generally very small and extremely uneven in scale, resulting in poor detection results. Therefore, we propose an efficient scale-aware network (ES-Net) to improve the effect of defect detection. By addressing the information loss of tiny targets and the mismatch between the receptive field of detection head and the scale of targets, ES-Net improves the overall defect detection effect, especially for tiny defects. Considering that existing works directly use an integrated feature to enhance features at all levels, it may cause confusion in the direction of network optimization. Therefore, we propose the aggregated feature guidance module (AFGM), which first performs efficient cascading fusion of multi-level features to filter cross-layer conflicts. Then the split and aggregation enhancement (SAE) module is designed to further optimize the integrated feature map, and the result is used to guide the shallow features. Moreover, we also introduce the multi-receptive field fusion (MFF) module to generate multi-receptive field information to supplement the shallow features after dimensionality reduction. The efficient stair pyramid (ESP) is a further improvement of feature pyramid network (FPN)-based network. In particular, we propose the dynamic scale-aware head (DSH) in shallow detection layer, which can adaptively select the best detection receptive field according to different scales of targets, thereby improving the detection performance of tiny targets. Extensive experimental results on Aliyun Tianchi fabric dataset (76.2% mAP), NEU-DET (79.1% mAP), and printed circuit board (PCB) defect dataset of Peking University (97.5% mAP) demonstrate the proposed ES-Net achieves competitive results compared to the state-of-the-art (SOTA) methods. Moreover, the high efficiency of ES-Net makes it more applicable in scenarios with high real-time requirements.

Journal ArticleDOI
TL;DR: An end-to-end dense attention-guided cascaded network (DACNet) to detect salient objects (i.e., defects) on the strip steel surface is proposed, where the proposed DACNet is a U-shape network including an encoder and a decoder.
Abstract: Recently, more and more researchers have paid attention to the surface defect detection of strip steel. However, the performance of existing methods usually fails to detect the defect regions from some complex scenes, especially with the noise disturbance and diverse defect types. Therefore, this article proposes an end-to-end dense attention-guided cascaded network (DACNet) to detect salient objects (i.e., defects) on the strip steel surface, where the proposed DACNet is a U-shape network including an encoder and a decoder. The encoder first deploys multiresolution convolutional branches (i.e., high/medium/low) in a cascaded way. Concretely, the cascaded feature integration (CFI) unit fuses the deep features from the last convolutional blocks of multiresolution branches, yielding the enhanced high-level deep semantic feature. Subsequently, coupled with the multilevel deep features from high-resolution branch, the new multiscale deep features are capable of characterizing various defects. Then, driven by the dense attention mechanism which enables the deeper attention cues flow into decoding stages, the decoder progressively integrates the multiscale deep features into the final saliency map, where the dense attention is designed to steer deep features pay more concerns to the defect regions. Comprehensive experiments are conducted on the public strip steel datasets, and the experimental results demonstrate that our model consistently outperforms the state-of-the-art models in all evaluation metrics.

Journal ArticleDOI
TL;DR: A deep learning model (DLM) using smoothed Gabor spectrogram and SGS of ECG signals as input to the pretrained Squeeze-Net, Res-Net50, and developed DLM called obstructive sleep apnea convolutional neural network (OSACN-Net).
Abstract: Obstructive sleep apnea (OSA) is a severe sleep-associated respiratory disorder, caused due to periodic disruption of breath during sleep. It may cause a number of serious cardiovascular complications, including stroke. Generally, OSA is detected by polysomnography (PSG), a costly procedure, and may cause discomfort to the patient. Nowadays, electrocardiogram (ECG) signal-based detection techniques have been explored as an alternative to PSG for OSA detection. Usual linear and nonlinear machine learning techniques are mainly focused on handcrafted feature extraction and classification that are time-consuming and may not be suitable for huge data. Therefore, in this work, a deep learning model (DLM) using smoothed Gabor spectrogram (SGS) of ECG signals is proposed for automated OSA detection to obtain high performance. The proposed framework fed Gabor spectrogram and SGS of ECG signals as input to the pretrained Squeeze-Net, Res-Net50, and developed DLM called obstructive sleep apnea convolutional neural network (OSACN-Net). The proposed OSACN-Net achieved an average classification accuracy of 94.81% with SGS using a tenfold cross-validation strategy. Compared to Squeeze-Net and Res-Net50, developed OSACN-Net is more accurate and lightweight as it requires few learnable parameters, which makes it computationally fast and efficient. The comparison results showed that the proposed framework outperformed all existing state-of-the-art methodologies.

Journal ArticleDOI
TL;DR: This study proposes a new feature extraction method to diminish the influence of limb position on sEMG-based pattern recognition (PR) and the CVA and the generalization of the proposed features improved substantially, aiming to facilitate the practical implementation of myoelectric interfaces.
Abstract: Gesture recognition via surface electromyography (sEMG) has drawn significant attention in the field of human–computer interaction. An important factor limiting the performance of sEMG-based pattern recognition (PR) is the generalization ability which sEMG changes for the identical movements when conducted at various positions or by different persons. Thus, this study aims to explore the generalization of classifier to develop a stable classification model that does not require relearning, even if it is used by other people. We propose a new feature extraction method to diminish the influence of limb position on sEMG-based PR. Specifically, the sEMG features are extracted directly from time domain. This condition is accomplished by using Fourier transform properties, difference, and the sum of squares differences. The best offline cross-validation accuracy (CVA) results are 88.775% training data from the tenth subject and testing data from the fifth subject in the NinaPro dataset. The best online CVA is 99%, and the movement selection time is 47.036 ± 1.028 ms. In comparison with the well-known sEMG feature, the CVA and the generalization of the proposed features improved substantially. These improvements aim to facilitate the practical implementation of myoelectric interfaces.