Showing papers in "Engineering Applications of Artificial Intelligence in 2022"
TL;DR: Deep Ensemble Learning (DEL) as discussed by the authors combines several individual models to obtain better generalization performance by combining the advantages of both the deep learning models as well as the ensemble learning.
Abstract: Ensemble learning combines several individual models to obtain better generalization performance. Currently, deep learning architectures are showing better performance compared to the shallow or traditional models. Deep ensemble learning models combine the advantages of both the deep learning models as well as the ensemble learning such that the final model has better generalization performance. This paper reviews the state-of-art deep ensemble models and hence serves as an extensive summary for the researchers. The ensemble models are broadly categorized into bagging, boosting, stacking, negative correlation based deep ensemble models, explicit/implicit ensembles, homogeneous/heterogeneous ensemble, decision fusion strategies based deep ensemble models. Applications of deep ensemble models in different domains are also briefly discussed. Finally, we conclude this paper with some potential future research directions. • This paper reviews the state-of-art deep ensemble models and hence serves as an extensive summary for the researchers. • The categorizes of deep ensemble models discussed are bagging, boosting and stacking, negative correlation based deep ensemble models, explicit/implicit ensembles, homogeneous/heterogeneous ensemble, decision fusion strategies, unsupervised, semi supervised, reinforcement learning and online/incremental, multilabel based deep ensemble models. • Application of deep ensemble models in different domains are discussed. • Finally, this paper provides an outlook towards future research directions.
140 citations
TL;DR: Clustering is an essential tool in data mining research and applications as discussed by the authors and it is the subject of active research in many fields of study, such as computer science, data science, statistics, pattern recognition, artificial intelligence, and machine learning.
Abstract: Clustering is an essential tool in data mining research and applications. It is the subject of active research in many fields of study, such as computer science, data science, statistics, pattern recognition, artificial intelligence, and machine learning. Several clustering techniques have been proposed and implemented, and most of them successfully find excellent quality or optimal clustering results in the domains mentioned earlier. However, there has been a gradual shift in the choice of clustering methods among domain experts and practitioners alike, which is precipitated by the fact that most traditional clustering algorithms still depend on the number of clusters provided a priori. These conventional clustering algorithms cannot effectively handle real-world data clustering analysis problems where the number of clusters in data objects cannot be easily identified. Also, they cannot effectively manage problems where the optimal number of clusters for a high-dimensional dataset cannot be easily determined. Therefore, there is a need for improved, flexible, and efficient clustering techniques. Recently, a variety of efficient clustering algorithms have been proposed in the literature, and these algorithms produced good results when evaluated on real-world clustering problems. This study presents an up-to-date systematic and comprehensive review of traditional and state-of-the-art clustering techniques for different domains. This survey considers clustering from a more practical perspective. It shows the outstanding role of clustering in various disciplines, such as education, marketing, medicine, biology, and bioinformatics. It also discusses the application of clustering to different fields attracting intensive efforts among the scientific community, such as big data, artificial intelligence, and robotics. This survey paper will be beneficial for both practitioners and researchers. It will serve as a good reference point for researchers and practitioners to design improved and efficient state-of-the-art clustering algorithms. • Provide an up-to-date comprehensive review of the different clustering techniques . • Highlight novel and most recent practical applications areas of clustering. • Provide a convenient research path for new researchers. • Help experts develop new algorithms for emerging challenges in the research area.
133 citations
TL;DR: In this article , the authors make a beneficial effort to explore the role of AI, including machine learning algorithms and deep learning architectures, in the foundation and development of the metaverse, and convey a comprehensive investigation of AI-based methods concerning several technical aspects (e.g., natural language processing, machine vision, blockchain, networking, digital twin, and neural interface).
Abstract: Along with the massive growth of the Internet from the 1990s until now, various innovative technologies have been created to bring users breathtaking experiences with more virtual interactions in cyberspace. Many virtual environments have been developed with immersive experience and digital transformation, but most are incoherent instead of being integrated into a platform. In this context, metaverse has been introduced as a shared virtual world that is fueled by many emerging technologies. Among such technologies, artificial intelligence (AI) has shown the great importance of enhancing immersive experience and enabling human-like intelligence of virtual agents. In this survey, we make a beneficial effort to explore the role of AI, including machine learning algorithms and deep learning architectures, in the foundation and development of the metaverse. As the main contributions, we convey a comprehensive investigation of AI-based methods concerning several technical aspects (e.g., natural language processing, machine vision, blockchain, networking, digital twin, and neural interface) that have potentials to build virtual worlds in the metaverse. Furthermore, several primary AI-aided applications, including healthcare, manufacturing, smart cities, and gaming, are studied to be promisingly deployed in the virtual worlds. Finally, we conclude the key contribution and open some future research directions of AI for the metaverse. Serving as a foundational survey, this work will help researchers, including experts and non-experts in related fields, in applying, developing, and optimizing AI techniques to polish the appearance of virtual worlds and improve the quality of applications built in the metaverse.
114 citations
TL;DR: In this paper , a parameter adaptation-based ant colony optimization algorithm based on particle swarm optimization (PSO) algorithm with the global optimization ability, fuzzy system with the fuzzy reasoning ability and 3-Opt algorithm with local search ability, namely PF3SACO is proposed to improve the optimization ability and convergence, avoid to fall into local optimum.
Abstract: In this paper, a parameter adaptation-based ant colony optimization (ACO) algorithm based on particle swarm optimization (PSO) algorithm with the global optimization ability, fuzzy system with the fuzzy reasoning ability and 3-Opt algorithm with local search ability, namely PF3SACO is proposed to improve the optimization ability and convergence, avoid to fall into local optimum. In the PF3SACO, a new dynamic parameter adjustment mechanism by the PSO and the fuzzy system is designed to adaptively adjust the pheromone importance factor α, pheromone volatilization coefficient ρ and the heuristic function importance factor β to accelerate the convergence, improve the search ability, enhance the local search ability and avoid premature. This is achievable by parameter adaptation to reflect the dynamic search characteristic by exploring and exploiting in the search process for the parameter values to be close to the optimal values. In addition, 3-Opt algorithm is applied to optimize the generated path to eliminate the cross path, obtain the optimal path and avoid to fall into local optimum. The optimization performance of the PF3SACO is investigated on fifteen travelling salesman problems (TSPs) with the scales from 42 to 783 cities. The experiment results show that the PF3SACO has better optimization performance by comparing with ABC, NACO, HYBRID, ACO-3Opt, PACO-3Opt, PSO-ACO-3Opt and some other well-known algorithms in most TSP in term of the solution quality, robustness and space distribution. It provides a reference to solve the large-scale TSP for obtaining better path length.
103 citations
TL;DR: Zhang et al. as mentioned in this paper proposed a new bio-inspired meta-heuristic algorithm, named artificial rabbits optimization (ARO), which is applied to the fault diagnosis of a rolling bearing, in which the back-propagation (BP) network optimized by ARO is developed.
Abstract: In this paper, a new bio-inspired meta-heuristic algorithm, named artificial rabbits optimization (ARO), is proposed and tested comprehensively. The inspiration of the ARO algorithm is the survival strategies of rabbits in nature, including detour foraging and random hiding. The detour foraging strategy enforces a rabbit to eat the grass near other rabbits’ nests, which can prevent its nest from being discovered by predators. The random hiding strategy enables a rabbit to randomly choose one burrow from its own burrows for hiding, which can decrease the possibility of being captured by its enemies. Besides, the energy shrink of rabbits will result in the transition from the detour foraging strategy to the random hiding strategy. This study mathematically models such survival strategies to develop a new optimizer. The effectiveness of ARO is tested by comparison with other well-known optimizers by solving a suite of 31 benchmark functions and five engineering problems. The results show that ARO generally outperforms the tested competitors for solving the benchmark functions and engineering problems. ARO is applied to the fault diagnosis of a rolling bearing, in which the back-propagation (BP) network optimized by ARO is developed. The case study results demonstrate the practicability of the ARO optimizer in solving challenging real-world problems. The source code of ARO is publicly available at https://seyedalimirjalili.com/aro and https://ww2.mathworks.cn/matlabcentral/fileexchange/110250-artificial-rabbits-optimization-aro.
89 citations
TL;DR: In this paper , a measure to quantify the information quality of intuitionistic fuzzy information based on a pseudo probability transformation is proposed, and its induced order is derived to rank intuitionistic values.
Abstract: Information quality has attracted increasing attention in recent years. In this paper, we propose a measure to quantify the information quality of intuitionistic fuzzy information based on a pseudo probability transformation, and derive its induced order to rank intuitionistic fuzzy values. The proposed measure and the induced order are proved to possess some well-defined properties that ensure rationality. Further, in order to rank intuitionistic fuzzy alternatives in decision making, we turn the proposed information quality into a ranking-oriented information quality, and present its induced order to generate the ranking. The decision capacity of our method is founded with the goal of rationalizing, simulating and further facilitating the interpretability and transparency in human decision making. A comparative study has been conducted, demonstrating the effectiveness of the proposed approach to ranking alternatives. The proposed approach is shown to outperform existing ranking methods via case-by-case comparisons. Finally, an application involving decision making of drug selection and an application in multi-criteria decision analysis of supplier selection are provided.
66 citations
TL;DR: Wang et al. as mentioned in this paper proposed a swarm intelligence bioinspired optimization algorithm, called the Dandelion Optimizer (DO), for solving continuous optimization problems, which simulates the process of dandelion seed long distance flight relying on wind, which is divided into three stages.
Abstract: This paper proposes a novel swarm intelligence bioinspired optimization algorithm, called the Dandelion Optimizer (DO), for solving continuous optimization problems. DO simulates the process of dandelion seed long-distance flight relying on wind, which is divided into three stages. In the rising stage, seeds raise in a spiral manner due to the eddies from above or drift locally in communities according to different weather conditions. In the descending stage, flying seeds steadily descend by constantly adjusting their direction in global space. In the landing stage, seeds land in randomly selected positions so that they grow. The moving trajectory of a seed in the descending stage and landing stage are described by Brownian motion and a Levy random walk. CEC2017 benchmark functions are utilized to evaluate the performance of DO, including the optimization accuracy, stability, convergence, and scalability, through a comparison with 9 well-known nature-inspired metaheuristic algorithms. Finally, the applicability of DO is verified by solving 4 real-world optimization problems. The experimental results indicate that the proposed DO method is a higher performing optimizer with outstanding iterative optimization and strong robustness compared with well-established algorithms. Source codes of DO are publicly available at https://ww2.mathworks.cn/matlabcentral/fileexchange/114680-dandelion-optimizer.
65 citations
TL;DR: In this paper , a multi-stage grey wolf optimizer (MGWO) was proposed to improve the performance of the basic GWO by dividing the search process into three stages and using different population updating strategies.
Abstract: Grey wolf optimizer (GWO) is a widespread metaphor-based algorithm based on the enhanced variants of velocity-free particle swarm optimizer with proven defects and shortcomings in performance. Regardless of the proven defect and lack of novelty in this algorithm, the GWO has a simple algorithm and it may face considerable unbalanced exploration and exploitation trends. However, GWO is easy to be utilized, and it has a low capacity to deal with multi-modal functions, and it quickly falls into the optima trap or fails to find the global optimal solution. To improve the shortcomings of the basic GWO, this paper proposes an improved GWO called multi-stage grey wolf optimizer (MGWO). By dividing the search process into three stages and using different population updating strategies at each stage, the MGWO’s optimization ability is improved while maintaining a certain convergence speed. The MGWO cannot easily fall into premature convergence and has a better ability to get rid of the local optima trap than GWO. Meanwhile, the MGWO achieves a better balance of exploration and exploitation and has a rough balance curve. Hence, the proposed MGWO can obtain a higher-quality solution. Based on verification on the thirty benchmark functions of IEEE CEC2017 as the objective functions, the simulation experiments in which MGWO compared with some swarm-based optimization algorithms and the balance and diversity analysis were conducted. The results verify the effectiveness and superiority of MGWO. Finally, the MGWO was applied to the multi-threshold image segmentation of Leaf Spot Diseases on Maize at four different threshold levels. The segmentation results were analysed by comparing each comparative algorithm’s PSNR, SSIM, and FSIM. The results proved that the MGWO has noticeable competitiveness, and it can be used as an effective optimizer for multi-threshold image segmentation.
65 citations
TL;DR: In this article , a spherical fuzzy GRA based on cumulative prospect theory (SF-CPT-GRA) method is proposed for MAGDM issues, which takes full account of the influence of DMs' risk attitude on decision result.
Abstract: Emergency supplies supplier selection can be regarded as a classic multiple attribute group decision making (MAGDM) problem. MAGDM is an interesting everyday problem full of uncertainty and ambiguity. As a new extension of fuzzy sets (FSs), spherical fuzzy sets (SFSs) can express vague and complex information in MAGDM more comprehensively. The gray relational analysis (GRA) is a practical method to process MAGDM problems. Furthermore, the cumulative prospect theory (CPT) can well capture the psychological behaviors of decision makers (DMs) in the assessment process. Therefore, in this paper, a spherical fuzzy GRA based on CPT (SF-CPT-GRA) method is proposed for MAGDM issues. In the meantime, the CRiteria Importance Through Intercriteria Correlation (CRITIC) method is used under the spherical fuzzy environment to obtain unknown attribute weights, which enhances the rationality of weight information. Finally, an example of emergency supplies supplier selection is given to illustrate the practicality of the proposed method. Sensitivity analysis and further comparative analysis attest the stability and validity of SF-CPT-GRA method. The proposed method takes full account of the influence of DMs’ risk attitude on decision result, which integrates CPT with GRA (CPT-GRA) and uses SFSs to express DMs’ preference information, making the decision results more scientific. Moreover, the SF-CPT-GRA method provides some references for dealing with other complex uncertain problems and further extension of CPT-GRA method in other decision environments.
59 citations
TL;DR: Wang et al. as mentioned in this paper proposed a convolutional block attention module (CBAM) to select the information critical to the vehicle detection task and suppress uncritical information, thus improving the detection accuracy of the algorithm.
Abstract: Vehicle detection technology is of great significance for realizing automatic monitoring and AI-assisted driving systems. The state-of-the-art object detection method, namely, a class of YOLOv5, has often been used to detect vehicles. However, it suffers some challenges, such as a high computational load and undesirable detection rate. To address these issues, an improved lightweight YOLOv5 method is proposed for vehicle detection in this paper. In the presented method, C3Ghost and Ghost modules are introduced into the YOLOv5 neck network to reduce the floating-point operations (FLOPs) in the feature channel fusion process and enhance the feature expression performance. A convolutional block attention module (CBAM) is introduced to the YOLOv5 backbone network to select the information critical to the vehicle detection task and suppress uncritical information, thus improving the detection accuracy of the algorithm. Furthermore, CIoU_Loss is considered the bounding box regression loss function to accelerate the bounding box regression rate and improve the localization accuracy of the algorithm. To verify the performance of the proposed approach, we tested our model via two case studies, i.e., the PASCAL VOC dataset and MS COCO dataset. The results show that the detection precision of the proposed model increased 3.2%, the FLOPs decreased 15.24%, and the number of model parameters decreased 19.37% compared with those of the existing YOLOv5. Through case studies and comparisons, the effectiveness and superiority of the presented approach are demonstrated.
55 citations
TL;DR: In this article , an efficient transfer learning (TL)-based multi-scale feature fused CNN (MSFFCNN) was proposed to capture the distinguishable features of various non-overlapping canonical frequency bands of EEG signals from different convolutional scales for multi-class MI classification.
Abstract: Deep learning (DL)-based brain–computer interface (BCI) in motor imagery (MI) has emerged as a powerful method for establishing direct communication between the brain and external electronic devices. However, due to inter-subject variability, inherent complex properties, and low signal-to-noise ratio (SNR) in electroencephalogram (EEG) signals are major challenges that significantly hinder the accuracy of the MI classifier. To overcome this, the present work proposes an efficient transfer learning (TL)-based multi-scale feature fused CNN (MSFFCNN) which can capture the distinguishable features of various non-overlapping canonical frequency bands of EEG signals from different convolutional scales for multi-class MI classification. In order to account for inter-subject variability from different subjects, the current work presents 4 different model variants including subject-independent and subject-adaptive classification models considering different adaptation configurations to exploit the full learning capacity of the classifier. Each adaptation configuration has been fine-tuned in an extensively trained pre-trained model and the performance of the classifier has been studied for a vast range of learning rates and degrees of adaptation which illustrates the advantages of using an adaptive transfer learning-based model. The model achieves an average classification accuracy of 94.06% (±0.70%) and the kappa value of 0.88 outperforming several baseline and current state-of-the-art EEG-based MI classification models with fewer training samples. The present research provides an effective and efficient transfer learning-based end-to-end MI classification framework for designing a high-performance robust MI-BCI system.
TL;DR: In this article , a weak fault diagnosis method for train axle box bearing based on parameter optimization Variational Mode Decomposition (VMD) and improved Deep Belief Network (DBN) is proposed.
Abstract: The vibration signal of the axle box bearing of the train is affected by the track excitation and the random noise of the environment. The vibration signal is nonlinear and non-stationary, and the signal characteristics of the early fault are weak and easy to be submerged, which leads to the low accuracy of the weak fault diagnosis of the bearing. To solve this problem, a weak fault diagnosis method for train axle box bearing based on parameter optimization Variational Mode Decomposition (VMD) and improved Deep Belief Network (DBN) is proposed. Firstly, the nonlinear convergence factor, Levy flight theory and greedy algorithm optimization theory are introduced into the Grey Wolf optimization algorithm (GWO), and an improved GWO algorithm based on hybrid strategy is proposed to improve the performance of the algorithm and solve the local optimal problem of the algorithm. Secondly, the improved GWO is applied to optimize the VMD parameters, which is used for signal decomposition. And the fault feature information of modal components with maximum correlation coefficient is extracted by multi-scale scatter entropy. Finally, the improved GWO algorithm is applied to optimize the parameters of the DBN to solve the parameter setting problem, and the optimized DBN is used as a pattern recognition algorithm for weak fault diagnosis of bearings. Through experimental comparison and analysis, the proposed method can effectively solve the problem of weak fault diagnosis of axle box bearings, and has high diagnostic accuracy.
TL;DR: Wang et al. as mentioned in this paper developed a restoration method based on backscatter pixel prior and color cast removal from the physical point of view of underwater image degradation to solve blurriness and color degradation issues.
Abstract: The use of underwater cameras instead of divers to enter complex underwater areas for real-time monitoring of fish, shrimp, and algae is significant to the aquaculture industry. However, underwater images are severely degraded due to light absorption and scattering, limiting the development of underwater computer vision and robot vision perception. To solve blurriness and color degradation issues, this paper developed a restoration method based on backscatter pixel prior and color cast removal from the physical point of view of underwater image degradation. The proposed method used only a single underwater image as an input to estimate various parameters accurately, such as depth map, backscatter map, and illuminant map. Specifically, a backscatter estimation algorithm based on a depth map was proposed to improve the contrast of underwater images. Then, an algorithm was designed to remove color deviation based on the illuminant map. In particular, a color compensation strategy was created that could completely eliminate red artifacts in underwater images that were generated by the strong attenuation of the red channel. We designed comparative experiments from multiple angles on different real underwater image datasets. Experiments showed that the proposed method improved the contrast and removed the color deviation of light absorption compared to several reported methods. Even on underwater images with severe attenuation, the proposed method showed a significant positive effectiveness and stability on color cast removal.
TL;DR: In this article , a thorough review of the AI techniques adopted for predictive and health management of engineering systems is conducted, and given that the future of inspection and maintenance will be predominantly AI-driven, the soft issues relating to manpower, cyber-security, standards and regulations under such a regime.
Abstract: Prognostics and health management (PHM) has become a crucial aspect of the management of engineering systems and structures, where sensor hardware and decision support tools are deployed to detect anomalies, diagnose faults and predict remaining useful lifetime (RUL). Methodologies for PHM are either model-driven, data-driven or a fusion of both approaches. Data-driven approaches make extensive use of large-scale datasets collected from physical assets to identify underlying failure mechanisms and root causes. In recent years, many data-driven PHM models have been developed to evaluate system’s health conditions using artificial intelligence (AI) and machine learning (ML) algorithms applied to condition monitoring data. The field of AI is fast gaining acceptance in various areas of applications such as robotics, autonomous vehicles and smart devices. With advancements in the use of AI technologies in Industry 4.0, where systems consist of multiple interconnected components in a cyber–physical space, there is increasing pressure on industries to move towards more predictive and proactive maintenance practices. In this paper, a thorough state-of-the-art review of the AI techniques adopted for PHM of engineering systems is conducted. Furthermore, given that the future of inspection and maintenance will be predominantly AI-driven, the paper discusses the soft issues relating to manpower, cyber-security, standards and regulations under such a regime. The review concludes that the current systems and methodologies for maintenance will inevitably become incompatible with future designs and systems; as such, continued research into AI-driven prognostics systems is expedient as it offers the best promise of bridging the potential gap.
TL;DR: Wang et al. as mentioned in this paper proposed a novel adaptive decoding biased random key genetic algorithm for cloud workflow scheduling, where the improved real number coding based on random key with limited value range is employed, and some novel schemes such as the population initialization based on level and heuristics including dynamic heterogeneous earliest finish time, the dynamic adaptive decoding, the load balance with communication avoidance and iterative forward-backward scheduling are designed for population initialization, chromosome decoding and improvement.
Abstract: With the ever-growing data and computing requirements, more and more scientific and business applications represented by workflows have been moved or are in active transition to cloud platforms. Therefore, the cloud workflow scheduling has become a hot topic. As a well-known NP-hard problem, many heuristic or metaheuristic algorithms/methods have been proposed. However, the heuristic method is problem-dependent which fits only a particular of problems, while the metaheuristic method has the problems of incomplete search space or low search efficiency in the complete space. To fill these gaps, a novel adaptive decoding biased random key genetic algorithm for cloud workflow scheduling is proposed. In this algorithm, the improved real number coding based on random key with limited value range is employed, and some novel schemes such as the population initialization based on level and heuristics including dynamic heterogeneous earliest finish time, the dynamic adaptive decoding, the load balance with communication avoidance and iterative forward–backward scheduling are designed for population initialization, chromosome decoding and improvement. To evaluate the performance, extensive experiments have been conducted on various real and random workflow applications, which demonstrates that the proposed algorithm outperforms the conventional approaches. • Propose a novel genetic algorithm for cloud workflow scheduling. • Use real number coding and dynamic adaptive decoding to improve efficiency. • Use level and heuristics including the DHEFT for a good initial population. • Use load balance and forward and backward scheduling to improve individual. • Verify the effectiveness of our algorithm by extensive experiments.
TL;DR: Wang et al. as discussed by the authors proposed a decision making framework based on Fermatean fuzzy integrated weighted distance measure and Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS) approach for the green low-carbon port (GLCP) evaluation.
Abstract: Green low-carbon port (GLCP) development and evaluation have gained increase attention in recent years. However, the uncertain assessment procedure and complex index in the GLCP assessment has brought great challenges to achieve a consensus for decision. To solve above problems, this study aims to present a novel decision making framework based on Fermatean fuzzy integrated weighted distance measure and Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS) approach for the GLCP evaluation. The Fermatean fuzzy integrated weighted average distance (FFIWAD) measure is first proposed, which takes into account the subjective significance of the variables and the attitudinal characteristics of decision makers. A FFIWAD-TOPSIS decision making framework is then developed, wherein the weights of index are determined by entropy method, and the FFIWAD is applied to calculate the distance between each alternative and the positive (or negative) ideal solution. Moreover, on the basis of the constructed index system of GLCP evaluation, the presented FFIWAD-TOPSIS method is applied to assess the development status GLCP of five major ports in China. The case analysis shows that Guangzhou Port behaves the best comprehensive performance among the five cities, while Shenzhen Port is the worst. Finally, the sensitivity analysis, as well as comparative analysis of the presented model are conducted.
TL;DR: In this paper , a non-intrusive surrogate modeling scheme based on deep learning for predictive modeling of complex systems, described by parametrized time-dependent partial differential equations, is presented.
Abstract: This paper presents a novel non-intrusive surrogate modeling scheme based on deep learning for predictive modeling of complex systems, described by parametrized time-dependent partial differential equations. Specifically, the proposed method utilizes a convolutional autoencoder in conjunction with a feed forward neural network to establish a mapping from the problem’s parametric space to its solution space. For this purpose, training data are collected by solving the high-fidelity model via finite elements for a reduced set of parameter values. Then, by applying the convolutional autoencoder, a low-dimensional vector representation of the high dimensional solution matrices is provided by the encoder, while the reconstruction map is obtained by the decoder. Using the latent vectors given by the encoder, a feed forward neural network is efficiently trained to map points from the parametric space to the compressed version of the respective solution matrices. This way, the proposed surrogate model is capable of predicting the entire time history response simultaneously with remarkable computational gains and very high accuracy. The elaborated methodology is demonstrated on the stochastic analysis of time-dependent partial differential equations solved with the Monte Carlo method. • A novel surrogate method is proposed for parametric prediction of dynamic systems. • Convolutional autoencoders are used to obtain low dimensional nonlinear manifolds. • The framework utilizes two levels of neural networks to build the surrogate. • The surrogate exhibits high accuracy and achieves drastic cost reduction. • It is highly applicable to problems that require multiple model evaluations.
TL;DR: Wang et al. as discussed by the authors proposed a novel fault diagnosis method of rolling bearing (RB) based on wavelet transform (WT) and an improved residual neural network (IResNet).
Abstract: The fault diagnosis (FD) of rolling bearing (RB) has a great significance in safe operation of engineering equipment. Many intelligent diagnosis methods have been successfully developed. However, the performances of traditional fault diagnosis methods are affected by noisy labels and environment which widely exist in realistic industrial applications. This article proposed a novel FD method of RB based on wavelet transform (WT) and an improved residual neural network (IResNet), named WT-IResNet. The proposed WT-IResNet approach uses a new pooling layer for dimension reduction and a global singular value decomposition (SVD) adaptive strategy for feature extraction. Furthermore, the original softmax layer and the logistic loss for training are replaced by a new loss function containing two adjustable parameters to address fault diagnosis with label noises. Two typical bearing failure datasets are used to evaluate the feasibility and effectiveness of WT-IResNet under noisy labels and noisy environment respectively. The experimental results indicate that WT-IResNet has better robustness against noise in comparison with other methods. Whatever under noisy labels or noisy environment, the performance of WT-IResNet outperforms other methods.
TL;DR: In this article , a systematic framework combining k-fold cross-validation (CV), metaheuristics (MHs), support vector regression (SVR), and Friedman and Nemenyi tests was proposed to improve the reliability and performance of geohazard modeling.
Abstract: Machine learning (ML) has been extensively applied to model geohazards, yielding tremendous success. However, researchers and practitioners still face challenges in enhancing the reliability of ML models. In the present study, a systematic framework combining k-fold cross-validation (CV), metaheuristics (MHs), support vector regression (SVR), and Friedman and Nemenyi tests was proposed to improve the reliability and performance of geohazard modeling. The average normalized mean square error (NMSE) from k-fold CV sets was adopted as the fitness metric. Twenty of the most well-established MHs and the most recent MHs were adopted to tune the hyperparameters of SVR and were evaluated through nonparametric Friedman and post hoc Nemenyi tests to identify significant differences. Observations from a typical reservoir landslide were selected as a benchmark dataset, and the accuracy, robustness, computational time, and convergence speed of the MHs were compared. Significant performance differences among the twenty MHs were identified by Friedman and post hoc Nemenyi tests of the mean absolute error (MAE), root mean squared error (RMSE), Kling–Gupta efficiency (KGE), and computational time, with p values lower than 0.05. The comparison of results demonstrated that the multiverse optimizer (MVO) is among the highest-performing, most stable, and computationally efficient algorithms, providing superior performance to other methods, with nearly optimum values of the correlation coefficient (R), a low MAE (23.5086 versus 23.9360), a low mean RMSE (48.6946 versus 50.1882), and a high mean KGE (0.9803 versus 0.9893) in predicting the displacement of the Shuping landslide. This paper considerably enriches the literature regarding hyperparameter optimization algorithms and the enhancement of their reliability. In addition, Friedman and post hoc Nemenyi tests have the potential for evaluating and comparing various ML-based geohazard models. • Introducing a systematic process for building a machine learning based prediction model in geohazard modeling. • Meta-heuristics are adopted for hyperparameter tuning of support vector regression, thus enhancing predictive accuracy. • A comprehensive comparison of twenty meta-heuristics for prediction of landslide displacement by Friedman and post hoc Nemenyi tests. • The multi-verse optimization (MVO) is very competitive because of the best tradeoff between accuracy, stability, and efficiency.
TL;DR: In this paper , a new method that uses one-dimensional CNNs based on Video Pixel Networks (VPNs) for short-term load forecasting, in which the gating mechanism of Multiplicative Units of the VPNs is modified in some sense, for short term load forecasting.
Abstract: The rising popularity of deep learning can largely be attributed to the big data phenomenon, the surge in the development of new and novel deep neural network architectures, and the advent of powerful computational innovations. However, the application of deep neural networks is rare for time series problems when compared to other application areas. Short-term load forecasting, a typical and difficult time series problem, is considered as the application domain in this study. One-dimensional Convolutional Neural Networks (CNNs) use is rare in time series forecasting problems when compared to Long Short Term Memory (LSTM) and Gated Recurrent Unit (GRU), and the efficiency of CNN has been rather remarkable for pattern extraction. Hence, a new method that uses one-dimensional CNNs based on Video Pixel Networks (VPNs) in this study, in which the gating mechanism of Multiplicative Units of the VPNs is modified in some sense, for short term load forecasting. Specifically, the proposed one-dimensional CNNs, LSTM and GRU variants are applied to real-world electricity load data for 1-hour-ahead and 24-hour-ahead prediction tasks which they are the main concerns for the electricity provider firms for short term load forecasting. Statistical tests were conducted to spot the significance of the performance differences in analyses for which ten ensemble predictions of each method were experimented. According to the results of the comparative analyses, the proposed one-dimensional CNN model yielded the best result in total with 2.21% mean absolute percentage error for 24-h ahead predicitions. On the other hand, not a noteworthy difference between the methods was spotted even the proposed one-dimensional CNN method yielded the best results with approximately 1% mean absolute percentage error for 1-h ahead predictions.
TL;DR: Wang et al. as mentioned in this paper proposed the Jensen-Shannon divergence of information volume (IJS) to measure the conflict between bodies of evidence in terms of the differences between the support of propositions and the elements.
Abstract: Dempster–Shafer evidence theory provides a powerful method for the expression and fusion of uncertain information. When handling the high conflict information, traditional Dempster combination rule can produce counterintuitive results. Hence, the reasonable conflict measure is essential in information fusion. Inspired by this view, the paper propose the new method to measure conflict between bodies of evidence. Firstly, we define a new information volume of mass function for the perspective of information discord and non-specificity. Second, we propose a generalized divergence based on information volume of mass function, denoted as Jensen–Shannon divergence of information volume ( I J S ) . I J S can effectively measure the conflict between bodies of evidence. I J S reflects the conflict between bodies of evidence in terms of the differences between the support of propositions and the elements. That is, compared to the current approach, I J S not only fully considers the differences between the support degree of propositions, but also the differences of elements in propositions from the perspective of information non-specificity. When the mass function degenerates to a probabilistic distribution, I J S also degenerates to the classical Jensen–Shannon divergence. Meanwhile, I J S also satisfies the axioms of distance measure, such as non-negativity, symmetry and etc. Further, we proved these axioms based on mathematical derivation, and some numerical examples are applied to explain axioms and advantages. Based on the proposed divergence measure, we propose a multi-source information fusion method in the real world, and several data sets can be used to show that the proposed fusion method is superior to current method under the framework of evidence theory.
TL;DR: In this paper , the state-of-the-art studies of visual and visual-based (i.e., visual-inertial, visual-LIDAR, visual LIDAR-IMU) SLAM are completely reviewed, as well the positioning accuracy of previous work are compared with the well-known frameworks on the public datasets.
Abstract: Autonomous driving vehicles require both a precise localization and mapping solution in different driving environment. In this context, Simultaneous Localization and Mapping (SLAM) technology is a well-study settlement. Light Detection and Ranging (LIDAR) and camera sensors are commonly used for localization and perception. However, through ten or twenty years of evolution, the LIDAR-SLAM method does not seem to have changed much. Compared with the LIDAR based schemes, the visual SLAM has a strong scene recognition ability with the advantages of low cost and easy installation. Indeed, people are trying to replace LIDAR sensors with camera only, or integrating other sensors on the basis of camera in the field of autonomous driving. Based on the current research situation of visual SLAM, this review covers the visual SLAM technologies. In particular, we firstly illustrated the typical structure of visual SLAM. Secondly, the state-of-the-art studies of visual and visual-based (i.e. visual-inertial, visual-LIDAR, visual-LIDAR-IMU) SLAM are completely reviewed, as well the positioning accuracy of our previous work are compared with the well-known frameworks on the public datasets. Finally, the key issues and the future development trend of visual SLAM technologies for autonomous driving vehicles applications are discussed.
TL;DR: In this article , the authors combine the Modified Grasshopper Optimization Algorithm (MGOA) and the Improved Harris Hawks Optimizer (IHHO) for attaining a better balance between the beginning stages of global search and the latter stages of convergence.
Abstract: The Combined Heat and Power Economic Dispatch (CHPED) is a real-world optimization problem with several complex constraints that has been a topic of studies around energy systems and optimization processes. This paper attempts to conceptualize a potent algorithm by combining the Modified Grasshopper Optimization Algorithm (MGOA) and the Improved Harris Hawks Optimizer (IHHO) for attaining a better balance between the beginning stages of global search and the latter stages of global convergence. The proposed attempt is abbreviated as MGOA-IHHO. Firstly, the chaotic and Opposition-Based Learning (OBL) methods are invoked to generate the initial population. Second, the mathematical model of the conventional Grasshopper Optimization Algorithm (GOA) is modified using Sine–Cosine Acceleration Coefficients (SCAC) to simulate the global exploration at the initial iterations and graduating to the global convergence at the final stages of optimization. Hence, it is named MGOA. Finally, the adaptive search mechanism integrates the two improved search phases of HHO with a search phase of MGOA to improve the performance of the proposed optimization method. This mechanism investigates the best solution for the aging level of the individual during the optimal evaluation process for choosing an appropriate search phase in MGOA-IHHO. The intended effect of the proposed MGOA-IHHO method is verified with other nature-inspired methods on standard single-objective test functions including 23 benchmark problems, 30 test suits of IEEE Congress on Evolutionary Computation 2017 (CEC2017), and four CHPED problems. The statistical results ascertain that the proposed hybridized MGOA-IHHO is capable of providing promising results when compared with its variants and optimization algorithms introduced in the literature.
TL;DR: The Linked Open Terms (LOT) methodology as mentioned in this paper is an overall and lightweight methodology for building ontologies based on existing methodologies and oriented to semantic web developments and technologies.
Abstract: Ontology Engineering has captured much attention during the last decades leading to the proliferation of numerous works regarding methodologies, guidelines, tools, resources, etc. including topics which are still being investigated. Even though, there are still many open questions when addressing a new ontology development project, regarding how to manage the overall project and articulate transitions between activities or which tasks and tools are recommended for each step. In this work we propose the Linked Open Terms (LOT) methodology, an overall and lightweight methodology for building ontologies based on existing methodologies and oriented to semantic web developments and technologies. The LOT methodology focuses on the alignment with industrial development, in addition to academic and research projects, and software development, that is making ontology development part of the software industry. This methodology includes lessons learnt from more than 20 years in ontological engineering and its application on 18 projects is reported.
TL;DR: In this article , an adaptive Harris hawk optimization with persistent trigonometric (sinecosine)-differences (ADHHO) is proposed for parameter identification of photovoltaic (PV) systems.
Abstract: In this paper, an adaptive Harris hawk optimization with persistent trigonometric (sine–cosine)-differences (ADHHO) is proposed for parameter identification of Photovoltaic (PV) systems. In the optimized version of HHO, we innovatively propose the persistent-trigonometric-differences mechanism for improving the global search capability of HHO; moreover, we improve the energy factor in the original algorithm so that ADHHO obtains a better balance between exploration and exploitation. Note that the proposed method can obtain lower CPU time in parameter extraction for the three-diode and PV module models with an enhanced parameter extraction performance. To validate the performance of ADHHO, we verified the parameter extraction capability of the single-diode model (SDM), double-diode model (DDM), triple-diode model (TDM), and PV module model (PVM), respectively. Further, we verified the parameter extraction effect of ADHHO in three commercial cells with different light intensity and temperature conditions. Experiments show that the method proposed in this paper can reasonably simulate the output performance of solar PV cells and can be used as a trustworthy method for the extraction of unknown parameters of solar PV systems.
TL;DR: In this paper , the authors present a comprehensive analysis of defect datasets, dataset validation, detection, prediction approaches, and tools for software defect prediction, and the survey exhibits the futuristic recommendations that will allow researchers to develop a tool for Software Defect Prediction.
Abstract: Delivering high-quality software products is a challenging task. It needs proper coordination from various teams in planning, execution, and testing. Many software products have high numbers of defects revealed in a production environment. Software failures are costly regarding money, time, and reputation for a business and even life-threatening if utilized in critical applications. Identifying and fixing software defects in the production system is costly, which could be a trivial task if detected before shipping the product. Binary classification is commonly used in existing software defect prediction studies. With the advancements in Artificial Intelligence techniques, there is a great potential to provide meaningful information to software development teams for producing quality software products. An extensive survey for Software Defect Prediction is necessary for exploring datasets, data validation methods, defect detection, and prediction approaches and tools. The survey infers standard datasets utilized in early studies lack adequate features and data validation techniques. According to the finding of the literature survey, the standard datasets has few labels, resulting in insufficient details regarding defects. Systematic Literature Reviews (SLR) on Software Defect Prediction are limited. Hence this SLR presents a comprehensive analysis of defect datasets, dataset validation, detection, prediction approaches, and tools for Software Defect Prediction. The survey exhibits the futuristic recommendations that will allow researchers to develop a tool for Software Defect Prediction. The survey introduces the architecture for developing a software prediction dataset with adequate features and statistical data validation techniques for multi-label classification for software defects.
TL;DR: In this paper , an efficient PV fault detection method is proposed to classify different types of PV module anomalies using thermographic images, which is designed as a multi-scale convolutional neural network (CNN) with three branches based on the transfer learning strategy.
Abstract: Photovoltaic (PV) power generation is one of the remarkable energy types to provide clean and sustainable energy. Therefore, rapid fault detection and classification of PV modules can help to increase the reliability of the PV systems and reduce operating costs. In this study, an efficient PV fault detection method is proposed to classify different types of PV module anomalies using thermographic images. The proposed method is designed as a multi-scale convolutional neural network (CNN) with three branches based on the transfer learning strategy. The convolutional branches include multi-scale kernels with levels of visual perception and utilize pre-trained knowledge of the transferred network to improve the representation capability of the network. To overcome the imbalanced class distribution of the raw dataset, the oversampling technique is performed with the offline augmentation method, and the network performance is increased. In the experiments, 11 types of PV module faults such as cracking, diode, hot spot, offline module, and other classes are utilized. The average accuracy is obtained as 97.32% for fault detection and 93.51% for 11 anomaly types. The experimental results indicate that the proposed method gives higher classification accuracy and robustness in PV panel faults and outperforms the other deep learning methods and existing studies.
TL;DR: In this paper , the authors present state-of-the-art machine learning, deep learning and statistical analysis models that have been used in the area of forecasting building energy consumption.
Abstract: The building sector accounts for 36 % of the total global energy usage and 40% of associated Carbon Dioxide emissions. Therefore, the forecasting of building energy consumption plays a key role for different building energy management applications (e.g., demand-side management and promoting energy efficiency measures), and implementing intelligent control strategies. Thanks to the advancement of Internet of Things in the last few years, this has led to an increase in the amount of buildings energy related-data. The accessibility of this data has inspired the interest of researchers to utilize different data-driven approaches to forecast building energy consumption. In this study, we first present state of-the-art Machine Learning, Deep Learning and Statistical Analysis models that have been used in the area of forecasting building energy consumption. In addition, we also introduce a comprehensive review of the existing research publications that have been published since 2015. The reviewed literature has been categorized according to the following scopes: (I) building type and location; (II) data components; (III) temporal granularity; (IV) data pre-processing methods; (V) features selection and extraction techniques; (VI) type of approaches; (VII) models used; and (VIII) key performance indicators. Finally, gaps and current challenges with respect to data-driven building energy consumption forecasting have been highlighted, and promising future research directions are also recommended.
TL;DR: In this paper , the authors provided an in-depth analysis of both 1st and 2nd DeepFakes generations in terms of fake detection performance and compared the performance of two different methods: (i) the traditional one followed in the literature based on selecting the entire face as input to the fake detection system, and (ii) a novel approach based on the selection of specific facial regions as input.
Abstract: Media forensics has attracted a tremendous attention in the last years in part due to the increasing concerns around DeepFakes. Since the release of the initial DeepFakes databases of the 1st generation such as UADFV and FaceForensics++ up to the latest databases of the 2nd generation such as Celeb-DF and DFDC, many visual improvements have been carried out, making fake videos almost indistinguishable to the human eye. This study provides an in-depth analysis of both 1st and 2nd DeepFakes generations in terms of fake detection performance. Two different methods are considered in our experimental framework: (i) the traditional one followed in the literature based on selecting the entire face as input to the fake detection system, and (ii) a novel approach based on the selection of specific facial regions as input to the fake detection system. Fusion techniques are applied both to the facial regions and also to three different state-of-the-art fake detection systems (Xception, Capsule Network, and DSP-FWA) in order to further increase the robustness of the detectors considered. Finally, experiments regarding intra- and inter-database scenarios are performed. Among all the findings resulting from our experiments, we highlight: (i) the very good results achieved using facial regions and fusion techniques with fake detection results above 99% Area Under the Curve (AUC) for UADFV, FaceForensics++, and Celeb-DF v2 databases, and (ii) the necessity to put more efforts on the analysis of inter-database scenarios to improve the ability of the fake detectors against attacks unseen during learning.
TL;DR: In this paper , the authors proposed a method for generating meta-tracklets and recognition of dominant motion patterns as a basis for automatic crowd behaviour analysis at the macroscopic level, where a crowd is treated as an entity.
Abstract: Automatic analysis and the recognition and prediction of the behaviour of large-scale crowds in video-surveillance data is a research field of paramount importance for the security of modern societies. It serves to predict and help prevent disasters in public places where crowds of people gather. The paper proposes a novel method for generating meta-tracklets and recognition of dominant motion patterns as a basis for automatic crowd behaviour analysis at the macroscopic level, where a crowd is treated as an entity. The basic characteristic of macroscopic crowd scenes is that it is impossible to detect and track individuals in the scene. The idea of the method proposed in this paper is to recognize dominant crowd motion patterns, by avoiding time-consuming and error-sensitive crowd segmentation, crowd tracking and detection of regions of interest. Thus, the process of determining dominant motion patterns and recognizing crowd behaviour is accelerated. The method is inspired by a quantum mechanical approach. It combines a set of particles, which are considered as particles in quantum mechanics, tracklets of particles’ advection in a video clip, and the interaction of wave functions spread out from particle positions. A wave function is expressed in the form of an asymmetric potential function. Peaks of the wave field define the most probable particle flow, which defines a meta-tracklet. Dominant motion patterns are recognized by applying the functions of fuzzy predicates, which represent a combination of common-sense and human expert knowledge about crowd motions, to the meta-tracklets. The experimental results of the proposed method are presented for a subset of UCF dataset and AGORASET crowd simulation videos and have shown promising results in dominant motion pattern recognition.