Showing papers on "Deep belief network published in 2021"
TL;DR: The structural principle, the characteristics, and some kinds of classic models of deep learning, such as stacked auto encoder, deep belief network, deep Boltzmann machine, and convolutional neural network are described.
408 citations
TL;DR: A secure intrusion, detection with blockchain based data transmission with classification model for CPS in healthcare sector, which achieves privacy and security and uses a multiple share creation (MSC) model for the generation of multiple shares of the captured image.
130 citations
TL;DR: A novel image segmentation method is developed in this paper for quantitative analysis of GICS based on the deep reinforcement learning (DRL), which can accurately distinguish the test line and the control line in the GICS images.
119 citations
TL;DR: This paper summarizes the knowledge and interpretation of Smart Cities (SC), Cyber Security (CS), and Deep Learning (DL) concepts as well as discussed existing related work on IoT security in smart cities.
106 citations
TL;DR: In this article, a data-driven approach for condition monitoring of generator bearing using temporal temperature data is presented, where four algorithms, the support vector regression machine, neural network, extreme learning machine, and the deep belief network are applied to model the bearing behavior.
Abstract: Wind turbines are widely installed as the new source of cleaner energy production. Dynamic and random stress imposed on the generator bearing of a wind turbine may lead to overheating and failure. In this paper, a data-driven approach for condition monitoring of generator bearings using temporal temperature data is presented. Four algorithms, the support vector regression machine, neural network, extreme learning machine, and the deep belief network are applied to model the bearing behavior. Comparative analysis of the models has demonstrated that the deep belief network is most accurate. It has been observed that the bearing failure is preceded by a change in the prediction error of bearing temperature. An exponentially-weighted moving average (EWMA) control chart is deployed to trend the error. Then a binary vector containing the abnormal errors and the normal residuals are generated for classifying failures. LS-SVM based classification models are developed to classify the fault bearings and the normal ones. The proposed approach has been validated with the data collected from 11 wind turbines.
106 citations
TL;DR: When deep learning SDAE is applied to IoT convergence-based intrusion security detection, the Detection load can be reduced, the detection effect can be improved, and the operation is more secure and stable.
Abstract: In order to explore the application value of deep learning denoising autoencoder (DAE) in Internet-of-Things (IoT) fusion security, in this study, a hierarchical intrusion security detection model stacked DAE supporting vector machine (SDAE-SVM) is constructed based on the three-layer neural network of self-encoder. The sample data after dimension reduction are obtained by layer by layer pretraining and fine-tuning. The traditional deep learning algorithms [stacked noise autoencoder (SNAE), stacked autoencoder (SAE), stacked contractive autoencoder (SCAE), stacked sparse autoencoder (SSAE), deep belief network (DBN)] are introduced to carry out the comparative simulation with the model in this study. The results show that when the encoder in the model is a 4-layer network structure, the accuracy rate (Ac) of the model is the highest (97.83%), the false-negative rate (Fn) (1.27%) and the false-positive rate (Fp) (3.21%) are the lowest. When the number of nodes in the first hidden layer is about 110, the model accuracy is about 98%. When comparing the model designed in this study with the common feature dimension reduction methods, the Ac, Fn, and Fp of this model are the best, which are 98.12%, 3.21%, and 1.27%, respectively. When compared with other deep learning algorithms of the same type, the recognition rate, Ac, error rate, and rejection rate show good results. In multiple data sets, the recognition rate, Ac, error rate, and rejection rate of the model in this study are always better than the traditional deep learning algorithms. In conclusion, when deep learning SDAE is applied to IoT convergence-based intrusion security detection, the detection load can be reduced, the detection effect can be improved, and the operation is more secure and stable.
104 citations
TL;DR: An intelligent fault diagnosis method for electromechanical system based on a new semisupervised graph convolution deep belief network algorithm is proposed in this article, which can achieve 98.66% accuracy with only 10 errors.
Abstract: The labeled monitoring data collected from the electromechanical system is limited in the real industries; traditional intelligent fault diagnosis methods cannot achieve satisfactory accurate diagnosis results. To deal with this problem, an intelligent fault diagnosis method for electromechanical system based on a new semisupervised graph convolution deep belief network algorithm is proposed in this article. Specifically, the labeled and unlabeled samples are first employed to design a new adaptive local graph learning method for constructing the graph neighbor relationship. Meanwhile, the labeled samples are applied to describe the discriminative structure information of data via the latest circle loss. Finally, the local and discriminative objective functions are reconstructed under the semisupervised learning framework. The experimental results from the motor-bearing system demonstrate that the method can achieve 98.66 $\%$ accuracy with only 10 $\%$ of training labeled data, which indicates that it is a promising semisupervised intelligent fault diagnosis method.
98 citations
TL;DR: Preliminary guidelines for a detailed view of deep learning techniques that researchers and engineers can use to improve the solar photovoltaic plant’s modeling and planning are offered.
94 citations
TL;DR: The existing studies of deep learning applied in ECG diagnosis according to four typical algorithms: stacked auto-encoders, deep belief network, convolutional neural network and recurrent neural network are reviewed.
Abstract: Cardiovascular disease (CVD) is a general term for a series of heart or blood vessels abnormality that serves as a global leading reason for death. The earlier the abnormal heart rhythm is discovered, the less severe the sequela and the faster the recovery. Electrocardiogram (ECG), as a main way to detect the electrical activity of heart, is a very important harmless means of predicting and diagnosing CVDs. However, ECG signal has characteristics of complex and high chaos, making it time-consuming and exhausting to interpret ECG signal even for experts. Hence, computer-aided methods are required to relief human burden and reduce errors caused by tiredness, inter- and intra-difference. Deep learning shows outstanding performance on ECG classification studies recent few years. Its hierarchical architecture enables higher-level features obtained and its strong ability to feature extraction contributes to classification project. Latest studies can achieve higher accuracy and efficiency than manual classification by experts. In this paper, we review the existing studies of deep learning applied in ECG diagnosis according to four typical algorithms: stacked auto-encoders, deep belief network, convolutional neural network and recurrent neural network. We first introduced the mechanism, development and application of the algorithms. Then we review their applications in ECG diagnosis systematically, discussing their highlights and limitations. Our view about future potential development of deep learning in ECG diagnosis is stated in the final part of this paper.
85 citations
TL;DR: The opportunities and challenges of deep learning for intelligent machining and tool monitoring, including the challenges associated with the data size, data nature, model selection, and process uncertainty, were discussed, and the research gaps were outlined.
Abstract: Data-driven methods provided smart manufacturing with unprecedented opportunities to facilitate the transition toward Industry 4.0–based production. Machine learning and deep learning play a critical role in developing intelligent systems for descriptive, diagnostic, and predictive analytics for machine tools and process health monitoring. This paper reviews the opportunities and challenges of deep learning (DL) for intelligent machining and tool monitoring. The components of an intelligent monitoring framework are introduced. The main advantages and disadvantages of machine learning (ML) models are presented and compared with those of deep models. The main DL models, including autoencoders, deep belief networks, convolutional neural networks (CNNs), and recurrent neural networks (RNNs), were discussed, and their applications in intelligent machining and tool condition monitoring were reviewed. The opportunities of data-driven smart manufacturing approach applied to intelligent machining were discussed to be (1) automated feature engineering, (2) handling big data, (3) handling high-dimensional data, (4) avoiding sensor redundancy, (5) optimal sensor fusion, and (6) offering hybrid intelligent models. Finally, the data-driven challenges in smart manufacturing, including the challenges associated with the data size, data nature, model selection, and process uncertainty, were discussed, and the research gaps were outlined.
83 citations
TL;DR: From the experimental analysis, it is clear that the deep learning model improved the accuracy, scalability, reliability, and performance of the cybersecurity applications when applied in realtime.
TL;DR: Deep learning algorithms do not seem to be appropriate models for credit scoring based on this comparison and XGBoost should be preferred over the other credit scoring methods considered here when classification performance is the main objective of credit scoring activities.
TL;DR: A novel hybrid multimodal deep learning system for identifying CO VID-19 virus in chest X-ray (CX-R) images is developed and termed as the COVID-DeepNet system to aid expert radiologists in rapid and accurate image interpretation.
Abstract: Coronavirus (COVID-19) epidemic outbreak has devastating effects on daily lives and healthcare systems worldwide This newly recognized virus is highly transmissible, and no clinically approved vaccine or antiviral medicine is currently available Early diagnosis of infected patients through effective screening is needed to control the rapid spread of this virus Chest radiography imaging is an effective diagnosis tool for COVID-19 virus and followup Here, a novel hybrid multimodal deep learning system for identifying COVID-19 virus in chest X-ray (CX-R) images is developed and termed as the COVID-DeepNet system to aid expert radiologists in rapid and accurate image interpretation First, Contrast-Limited Adaptive Histogram Equalization (CLAHE) and Butterworth bandpass filter were applied to enhance the contrast and eliminate the noise in CX-R images, respectively Results from two different deep learning approaches based on the incorporation of a deep belief network and a convolutional deep belief network trained from scratch using a large-scale dataset were then fused Parallel architecture, which provides radiologists a high degree of confidence to distinguish healthy and COVID-19 infected people, was considered The proposed COVID-DeepNet system can correctly and accurately diagnose patients with COVID-19 with a detection accuracy rate of 99 93%, sensitivity of 99 90%, specificity of 100%, precision of 100%, F1-score of 99 93%, MSE of 0 021%, and RMSE of 0 016% in a large-scale dataset This system shows efficiency and accuracy and can be used in a real clinical center for the early diagnosis of COVID-19 virus and treatment follow-up with less than 3 s per image to make the final decision
TL;DR: It is concluded that the DBPGA model is an excellent alternative tool for predicting flash flood susceptibility for other regions prone to flash floods.
Abstract: Flash floods are responsible for loss of life and considerable property damage in many countries. Flood susceptibility maps contribute to flood risk reduction in areas that are prone to this hazard if appropriately used by land-use planners and emergency managers. The main objective of this study is to prepare an accurate flood susceptibility map for the Haraz watershed in Iran using a novel modeling approach (DBPGA) based on Deep Belief Network (DBN) with Back Propagation (BP) algorithm optimized by the Genetic Algorithm (GA). For this task, a database comprising ten conditioning factors and 194 flood locations was created using the One-R Attribute Evaluation (ORAE) technique. Various well-known machine learning and optimization algorithms were used as benchmarks to compare the prediction accuracy of the proposed model. Statistical metrics include sensitivity, specificity accuracy, root mean square error (RMSE), and area under the receiver operatic characteristic curve (AUC) were used to assess the validity of the proposed model. The result shows that the proposed model has the highest goodness-of-fit (AUC = 0.989) and prediction accuracy (AUC = 0.985), and based on the validation dataset it outperforms benchmark models including LR (0.885), LMT (0.934), BLR (0.936), ADT (0.976), NBT (0.974), REPTree (0.811), ANFIS-BAT (0.944), ANFIS-CA (0.921), ANFIS-IWO (0.939), ANFIS-ICA (0.947), and ANFIS-FA (0.917). We conclude that the DBPGA model is an excellent alternative tool for predicting flash flood susceptibility for other regions prone to flash floods.
TL;DR: A novel graph-based classification model using the deep belief network (DBN) and the ABIDE database, which is a worldwide multisite functional and structural brain imaging data aggregation, enables the identification of the most remarkable autistic neural correlation patterns from the data-driven outcomes.
Abstract: With the increasing prevalence of autism spectrum disorder (ASD), it is important to identify ASD patients for effective treatment and intervention, especially in early childhood. Neuroimaging techniques have been used to characterize the complex biomarkers based on the functional connectivity anomalies in the ASD. However, the diagnosis of ASD still adopts the symptom-based criteria by clinical observation. The existing computational models tend to achieve unreliable diagnostic classification on the large-scale aggregated data sets. In this work, we propose a novel graph-based classification model using the deep belief network (DBN) and the Autism Brain Imaging Data Exchange (ABIDE) database, which is a worldwide multisite functional and structural brain imaging data aggregation. The remarkable connectivity features are selected through a graph extension of ${K}$ -nearest neighbors and then refined by a restricted path-based depth-first search algorithm. Thanks to the feature reduction, lower computational complexity could contribute to the shortening of the training time. The automatic hyperparameter-tuning technique is introduced to optimize the hyperparameters of the DBN by exploring the potential parameter space. The simulation experiments demonstrate the superior performance of our model, which is 6.4% higher than the best result reported on the ABIDE database. We also propose to use the data augmentation and the oversampling technique to identify further the possible subtypes within the ASD. The interpretability of our model enables the identification of the most remarkable autistic neural correlation patterns from the data-driven outcomes.
TL;DR: In this study, a new metaheuristic‐based system is presented for the early detection of brain tumors, which implements three main steps, namely tumor segmentation, feature extraction, and classification based on a deep belief network.
Abstract: The high mortality rate associated with brain tumors requires early detection in the early stages to treat and reduce mortality. Due to the complexity of brain tissue, manual diagnosis of the brain and tumor tissues is very time‐consuming and operator dependent. Furthermore, there is a need for experts who can review the images to detect these effects, rendering traditional methods inefficient in their presence. Therefore, the use of automated procedures for the careful examination of tumors can prove useful. In this study, a new metaheuristic‐based system is presented for the early detection of brain tumors. The proposed method implements three main steps, namely tumor segmentation, feature extraction, and classification based on a deep belief network. An improved version of the seagull optimization algorithm is adopted for optimal selection of the features and classification of the images. The simulation results of the proposed method are compared with a few existing methods. The final results demonstrate that the proposed method exhibits superior performance in terms of the CDR, FAR, and FRR indices compared with the other methods.
TL;DR: Results verify that DIDBN is able to learn distribution-invariant features and achieve higher diagnosis accuracies than recently proposed methods.
Abstract: As a deep learning model, a deep belief network (DBN) consists of multiple restricted Boltzmann machines (RBMs). Based on DBN, many intelligent fault diagnosis methods are proposed. However, these methods seldom considered the appearance of new working conditions during the operation of real machines. Varying working conditions lead to a change of feature distributions and finally result in low diagnosis accuracies. Therefore, we propose a distribution-invariant DBN (DIDBN) to learn distribution-invariant features directly from raw vibration data. DIDBN consists of a locally connected RBM (LCRBM) layer, a fully connected RBM layer, and an RBM layer with a mean discrepancy maximum (MDM-RBM). The LCRBM layer is designed with a locally connected structure. By proposing MDM, the MDM-RBM layer is able to obtain features that have close distributions under varying working conditions. Followed by a softmax classifier, DIDBN is able to recognize faults. The proposed method is applied to two diagnosis cases. Results verify that DIDBN is able to learn distribution-invariant features and achieve higher diagnosis accuracies than recently proposed methods. Moreover, the reason why DIDBN is able to learn distribution-invariant features is explained by visualizing the feature learning process.
TL;DR: This work develops deep belief network (DBN) models to predict the production performance of unconventional wells effectively and accurately and shows excellent reusability, making it a powerful tool in optimizing fracturing designs.
TL;DR: In this article, a patch-based deep learning method called Pa-DBN-BC is proposed to detect and classify breast cancer on histopathology images using the Deep Belief Network (DBN) Features are extracted through an unsupervised pre-training and supervised fine-tuning phase The network automatically extracts features from image patches.
Abstract: Accurate detection and classification of breast cancer is a critical task in medical imaging due to the complexity of breast tissues Due to automatic feature extraction ability, deep learning methods have been successfully applied in different areas, especially in the field of medical imaging In this study, a novel patch-based deep learning method called Pa-DBN-BC is proposed to detect and classify breast cancer on histopathology images using the Deep Belief Network (DBN) Features are extracted through an unsupervised pre-training and supervised fine-tuning phase The network automatically extracts features from image patches Logistic regression is used to classify the patches from histopathology images The features extracted from the patches are fed to the model as input and the model presents the result as a probability matrix as either a positive sample (cancer) or a negative sample (background) The proposed model is trained and tested on the whole slide histopathology image dataset having images from four different data cohorts and achieved an accuracy of 86% Consequently, the proposed method is better than the traditional ones, as it automatically learns the best possible features and experimental results show that the model outperformed the previously proposed deep learning methods
TL;DR: A deep learning-based axial capacity prediction for cold-formed steel channel sections is developed using Deep Belief Network and it was found that the DBN was conservative by 9%, 6% and 8% for stub columns, intermediate columns, and slender columns, respectively.
TL;DR: Comparisons with other state-of-the-art models confirm that the proposed interval forecasting model cannot only improve the forecasting efficiency and accuracy, but also simplify the forecasting process of deep learning approaches, which can provide great referential value for future work.
TL;DR: Novel deep learning algorithms such as the one used in this study can improve the accuracy of flood susceptibility maps that are required by planners, decision makers, and government agencies to manage of areas vulnerable to flood-induced damage.
TL;DR: In this paper, the authors proposed a hybrid learning algorithm for 5G network slicing, which involves three main phases: data collection, optimal weighted feature extraction (OWFE), and slicing classification.
TL;DR: Basic concepts on data set, performance metric, and restricted Boltzmann machines, to help understand the basic DBN based intrusion detection model are provided and a complete review and analysis on the previously published works on DBNbased IDS models is provided.
Abstract: With the recent growth in the number of IoT devices, the amount of personal, sensitive, and important data flowing through the global network have grown rapidly. Additionally, the malicious attempt to access important information or damage the network have also become more complex and advanced. Thus, cybersecurity has become an important issue for the evolution toward future networks that can react and counter such threats. Intrusion detection is an important part of the cybersecurity technology with the goal of monitoring and analyzing network traffic from various resources and detect malicious activities. In recent years, deep learning base deep neural network (DNN) techniques have been utilized as the key solution to detect malicious attacks and among many DNNs, deep belief network (DBN) has been the most influential technique. There have been many attempts to survey wide range of machine learning and deep learning technique based intrusion detection research works, including DBN, but failed to provide a complete review of all the aspects related to the DBN based intrusion detection models. Unlike previous survey papers, we first provide basic concepts on data set, performance metric, and restricted Boltzmann machines, to help understand the basic DBN based intrusion detection model. Finally, a complete review and analysis on the previously published works on DBN based IDS models is provided.
01 May 2021
TL;DR: In this paper, a comprehensive overview from the perspective of these neural networks and deep learning techniques according to today's diverse needs is presented, and the applicability of these techniques in various cybersecurity tasks such as intrusion detection, identification of malware or botnets, phishing, predicting cyberattacks, e.g. denial of service, fraud detection or cyberanomalies, etc.
Abstract: Deep learning, which is originated from an artificial neural network (ANN), is one of the major technologies of today’s smart cybersecurity systems or policies to function in an intelligent manner. Popular deep learning techniques, such as multi-layer perceptron, convolutional neural network, recurrent neural network or long short-term memory, self-organizing map, auto-encoder, restricted Boltzmann machine, deep belief networks, generative adversarial network, deep transfer learning, as well as deep reinforcement learning, or their ensembles and hybrid approaches can be used to intelligently tackle the diverse cybersecurity issues. In this paper, we aim to present a comprehensive overview from the perspective of these neural networks and deep learning techniques according to today’s diverse needs. We also discuss the applicability of these techniques in various cybersecurity tasks such as intrusion detection, identification of malware or botnets, phishing, predicting cyberattacks, e.g. denial of service, fraud detection or cyberanomalies, etc. Finally, we highlight several research issues and future directions within the scope of our study in the field. Overall, the ultimate goal of this paper is to serve as a reference point and guidelines for the academia and professionals in the cyber industries, especially from the deep learning point of view.
01 Jun 2021
TL;DR: In this paper, the investigation of embedding the Deep learning methodology is discussed, the DBN enhancement to the security network is compared with standard DGAs and IDS algorithms, and the results are analyzed.
Abstract: Internet of Things (IoT) is a new age technology, developed with the vision to connect and interconnect all the objects everywhere. This technology enables an overwhelming smartness, which helps the humankind in many ways. Connecting the objects around us, make them communicate with each other towards a mission of intelligent healthcare, safety, Industrial processing applications. As the Internet of Things involved in many various entities and diverse applications, that the vulnerability to unauthorized access is much higher. Today, cyber-attacks faced by the communication networks are very strong and critically alarming. This research represents an intelligent technique or methodology to defend the security breach , developed with the enhancement of Deep Learning algorithms (Deep Belief Network), i.e., Deep Belief Network . This intelligent intrusion detection methodology scrutinizes the malicious activity that is active inside the network, and one tries to get its entry. In this paper, the investigation of embedding the Deep learning methodology is discussed. The DBN enhancement to the security network is compared with standard DGAs and IDS algorithms, and the results are analyzed.
TL;DR: Experimental results demonstrate that the DBN-based MTL algorithm developed in this study is an effective, superior and practical method of AD diagnosis.
Abstract: Accurate classification of Alzheimer’s disease (AD) and mild cognitive impairment (MCI), especially distinguishing the progressive MCI (pMCI) from stable MCI (sMCI), will be helpful in both reducing the risk of converting into AD and also releasing the burden on the family and even the society. In this study, a novel deep belief network (DBN) based multi-task learning algorithm is developed for the classification issue. In particular, the dropout technology and zero-masking strategy are exploited for getting over the overfitting problem and also enhancing the generalization ability and robustness of the model. Then, a new framework based on the DBN-based multi-task learning is established for accurate diagnosis of AD. After MRI preprocessing, not only the principal component analysis is utilized to reduce the feature dimension, but also multi-task feature selection approach is introduced to select the feature set related to all tasks as a result of taking the internal relevancy among multiple related tasks into consideration. Using data from the ADNI dataset, our method achieves satisfactory results in six tasks of health control (HC) vs. AD, HC vs. pMCI, HC vs. sMCI, pMCI vs. AD, sMCI vs. AD and sMCI vs. pMCI with the accuracies are 98.62%, 96.67%, 92.31%, 91.89%, 99.62% and 87.78%, respectively. Experimental results demonstrate that the DBN-based MTL algorithm developed in this study is an effective, superior and practical method of AD diagnosis.
TL;DR: This paper introduces the development and application of PPIR technology, followed by its classification and analysis, and presents the theory of four types of deep learning methods and their applications in PPIR.
Abstract: Plant phenotypic image recognition (PPIR) is an important branch of smart agriculture. In recent years, deep learning has achieved significant breakthroughs in image recognition. Consequently, PPIR technology that is based on deep learning is becoming increasingly popular. First, this paper introduces the development and application of PPIR technology, followed by its classification and analysis. Second, it presents the theory of four types of deep learning methods and their applications in PPIR. These methods include the convolutional neural network, deep belief network, recurrent neural network, and stacked autoencoder, and they are applied to identify plant species, diagnose plant diseases, etc. Finally, the difficulties and challenges of deep learning in PPIR are discussed.
TL;DR: A novel method to classify stochastic remote sensing events and to perform adaptive posture estimation and a unified pseudo-2D stick model are proposed, which are superior compared to existing state-of-the-art methods.
Abstract: Advances in video capturing devices enable adaptive posture estimation (APE) and event classification of multiple human-based videos for smart systems. Accurate event classification and adaptive posture estimation are still challenging domains, although researchers work hard to find solutions. In this research article, we propose a novel method to classify stochastic remote sensing events and to perform adaptive posture estimation. We performed human silhouette extraction using the Gaussian Mixture Model (GMM) and saliency map. After that, we performed human body part detection and used a unified pseudo-2D stick model for adaptive posture estimation. Multifused data that include energy, 3D Cartesian view, angular geometric, skeleton zigzag and moveable body parts were applied. Using a charged system search, we optimized our feature vector and deep belief network. We classified complex events, which were performed over sports videos in the wild (SVW), Olympic sports, UCF aerial action dataset and UT-interaction datasets. The mean accuracy of human body part detection was 83.57% over the UT-interaction, 83.00% for the Olympic sports and 83.78% for the SVW dataset. The mean event classification accuracy was 91.67% over the UT-interaction, 92.50% for Olympic sports and 89.47% for SVW dataset. These results are superior compared to existing state-of-the-art methods.
TL;DR: A rolling element bearing fault diagnosis approach based on principal component analysis and adaptive deep belief network with Parametric Rectified Linear Unit activation layers is proposed, which results in an optimal DBN structure with high accuracy and convergence rate.