Showing papers on "Swarm intelligence published in 2021"
TL;DR: A review of swarm intelligence algorithms can be found in this paper, where the authors highlight the functions and strengths from 127 research literatures and briefly provide the description of their successful applications in optimization problems of engineering fields.
Abstract: Swarm intelligence algorithms are a subset of the artificial intelligence (AI) field, which is increasing popularity in resolving different optimization problems and has been widely utilized in various applications. In the past decades, numerous swarm intelligence algorithms have been developed, including ant colony optimization (ACO), particle swarm optimization (PSO), artificial fish swarm (AFS), bacterial foraging optimization (BFO), and artificial bee colony (ABC). This review tries to review the most representative swarm intelligence algorithms in chronological order by highlighting the functions and strengths from 127 research literatures. It provides an overview of the various swarm intelligence algorithms and their advanced developments, and briefly provides the description of their successful applications in optimization problems of engineering fields. Finally, opinions and perspectives on the trends and prospects in this relatively new research domain are represented to support future developments.
247 citations
TL;DR: A comparative analysis of different feature selection methods is presented, and a general categorization of these methods is performed, which shows the strengths and weaknesses of the different studied swarm intelligence-based feature selection Methods.
200 citations
TL;DR: A rigorous yet systematic review is presented to organize and summarize the information on the PSO algorithm and the developments and trends of its most basic as well as of some of the very notable implementations that have been introduced recently, bearing in mind the coverage of paradigm, theory, hybridization, parallelization, complex optimization, and the diverse applications of the algorithm.
Abstract: Over the ages, nature has constantly been a rich source of inspiration for science, with much still to discover about and learn from. Swarm Intelligence (SI), a major branch of artificial intelligence, was rendered to model the collective behavior of social swarms in nature. Ultimately, Particle Swarm Optimization algorithm (PSO) is arguably one of the most popular SI paradigms. Over the past two decades, PSO has been applied successfully, with good return as well, in a wide variety of fields of science and technology with a wider range of complex optimization problems, thereby occupying a prominent position in the optimization field. However, through in-depth studies, a number of problems with the algorithm have been detected and identified; e.g., issues regarding convergence, diversity, and stability. Consequently, since its birth in the mid-1990s, PSO has witnessed a myriad of enhancements, extensions, and variants in various aspects of the algorithm, specifically after the twentieth century, and the related research has therefore now reached an impressive state. In this paper, a rigorous yet systematic review is presented to organize and summarize the information on the PSO algorithm and the developments and trends of its most basic as well as of some of the very notable implementations that have been introduced recently, bearing in mind the coverage of paradigm, theory, hybridization, parallelization, complex optimization, and the diverse applications of the algorithm, making it more accessible. Ease for researchers to determine which PSO variant is currently best suited or to be invented for a given optimization problem or application. This up-to-date review also highlights the current pressing issues and intriguing open challenges haunting PSO, prompting scholars and researchers to conduct further research both on the theory and application of the algorithm in the forthcoming years.
169 citations
TL;DR: This review study serves as a solid reference for future studies in the arena of SI and in particular the MBO algorithm including its modifications, hybridizations, variants, and applications.
Abstract: Swarm intelligence (SI) is the collective behavior of decentralized, self-organized natural or artificial systems. Monarch butterfly optimization (MBO) algorithm is a class of swarm intelligence metaheuristic algorithm inspired by the migration behavior of monarch butterflies. Through the migration operation and butterfly adjusting operation, individuals in MBO are updated. MBO can outperform many state-of-the-art optimization techniques when solving global numerical optimization and engineering problems. This paper presents a comprehensive review of the MBO algorithm including its modifications, hybridizations, variants, and applications. Additionally, further research directions for MBO are discussed. This review study serves as a solid reference for future studies in the arena of SI and in particular the MBO algorithm.
136 citations
TL;DR: A comprehensive review of GOA based on more than 120 scientific articles published by leading publishers: IEEE, Springer, Elsevier, IET, Hindawi, and others is presented in this article.
Abstract: Grasshopper Optimization Algorithm (GOA) is a recent swarm intelligence algorithm inspired by the foraging and swarming behavior of grasshoppers in nature. The GOA algorithm has been successfully applied to solve various optimization problems in several domains and demonstrated its merits in the literature. This paper proposes a comprehensive review of GOA based on more than 120 scientific articles published by leading publishers: IEEE, Springer, Elsevier, IET, Hindawi, and others. It provides the GOA variants, including multi-objective and hybrid variants. It also discusses the main applications of GOA in various fields such as scheduling, economic dispatch, feature selection, load frequency control, distributed generation, wind energy system, and other engineering problems. Finally, the paper provides some possible future research directions in this area.
98 citations
TL;DR: In this paper, a novel DE algorithm named quantum-based avian navigation optimizer algorithm (QANA) was proposed, which is inspired by the extraordinary precision navigation of migratory birds during long-distance aerial paths.
87 citations
TL;DR: This paper builds a bridge between the swarm intelligence and cyber-physical systems communities to view the research problems of swarm intelligence from a broader perspective and motivate future research activities in modeling, design, validation/verification, and human-in-the-loop concepts.
Abstract: Swarm Intelligence (SI) is a popular multi-agent framework that has been originally inspired by swarm behaviors observed in natural systems, such as ant and bee colonies. In a system designed after swarm intelligence, each agent acts autonomously, reacts on dynamic inputs, and, implicitly or explicitly, works collaboratively with other swarm members without a central control. The system as a whole is expected to exhibit global patterns and behaviors. Although well-designed swarms can show advantages in adaptability, robustness, and scalability, it must be noted that SI system have not really found their way from lab demonstrations to real-world applications, so far. This is particularly true for embodied SI, where the agents are physical entities, such as in swarm robotics scenarios. In this paper, we start from these observations, outline different definitions and characterizations, and then discuss present challenges in the perspective of future use of swarm intelligence. These include application ideas, research topics, and new sources of inspiration from biology, physics, and human cognition. To motivate future applications of swarms, we make use of the notion of cyber-physical systems (CPS). CPSs are a way to encompass the large spectrum of technologies including robotics, internet of things (IoT), Systems on Chip (SoC), embedded systems, and so on. Thereby, we give concrete examples for visionary applications and their challenges representing the physical embodiment of swarm intelligence in autonomous driving and smart traffic, emergency response, environmental monitoring, electric energy grids, space missions, medical applications, and human networks. We do not aim to provide new solutions for the swarm intelligence or CPS community, but rather build a bridge between these two communities. This allows us to view the research problems of swarm intelligence from a broader perspective and motivate future research activities in modeling, design, validation/verification, and human-in-the-loop concepts.
87 citations
TL;DR: Intelligent feature selection methods and intrusion detection (ISTID) organization in webs based on neuron-genetic algorithms, intelligent software agents, genetic algorithms, particulate swarm intelligence and neural networks, rough-set are proposed.
Abstract: Rapid Internet use growth and applications of diverse military have managed researchers to develop smart systems to help applications and users achieve the facilities through the provision of required service quality in networks. Any smart technologies offer protection in interactions in dispersed locations such as, e-commerce, mobile networking, telecommunications and management of network. Furthermore, this article proposed on intelligent feature selection methods and intrusion detection (ISTID) organization in webs based on neuron-genetic algorithms, intelligent software agents, genetic algorithms, particulate swarm intelligence and neural networks, rough-set. These techniques were useful to identify and prevent network intrusion to provide Internet safety and improve service value and accuracy, performance and efficiency. Furthermore, new algorithms of intelligent rules-based attributes collection algorithm for efficient function and rules-based improved vector support computer, were proposed in this article, along with a survey into the current smart techniques for intrusion detection systems.
82 citations
TL;DR: It was found that the stacking ensemble learning method can effectively integrate the prediction results of the base learner to improve the performance of the model, and the improved swarm intelligence algorithm can further improve the prediction accuracy.
63 citations
TL;DR: This work analyzes the behavior of nine metaheuristic algorithms, namely, salp swarm algorithm (SSA), multi-verse optimizer (MVO), moth-flame optimizers (MFO), atom search optimization (ASO), ecogeography-based optimization (EBO), queuing search algorithm (QSA), equilibrium Optimizer (EO), evolutionary strategy (ES) and hybrid self-adaptive orthogonal genetic algorithm (HSOGA).
Abstract: Determining the solution for real mechanical design problems is a challenging task when using the newly developed and efficient swarm intelligence algorithms. There are so many difficulties to be addressed, including but not limited to mixed decision variables, diverse constraints, inherent errors, conflicting objectives, and numerous locally optimal solutions. This work analyzes the behavior of nine metaheuristic algorithms, namely, salp swarm algorithm (SSA), multi-verse optimizer (MVO), moth-flame optimizer (MFO), atom search optimization (ASO), ecogeography-based optimization (EBO), queuing search algorithm (QSA), equilibrium optimizer (EO), evolutionary strategy (ES) and hybrid self-adaptive orthogonal genetic algorithm (HSOGA). The efficiency of these algorithms is evaluated on eight mechanical design problems using the solution quality and convergence analysis, which verifies the wide applicability of these algorithms to real-world application problems.
62 citations
25 Oct 2021
TL;DR: In this paper, the authors proposed an enhanced version of the firefly algorithm that corrects the acknowledged drawbacks of the original method by an explicit exploration mechanism and a chaotic local search strategy, theoretically tested on two sets of bound-constrained benchmark functions from the CEC suites and practically validated for automatically selecting the optimal dropout rate for the regularization of deep neural networks.
Abstract: Swarm intelligence techniques have been created to respond to theoretical and practical global optimization problems. This paper puts forward an enhanced version of the firefly algorithm that corrects the acknowledged drawbacks of the original method, by an explicit exploration mechanism and a chaotic local search strategy. The resulting augmented approach was theoretically tested on two sets of bound-constrained benchmark functions from the CEC suites and practically validated for automatically selecting the optimal dropout rate for the regularization of deep neural networks. Despite their successful applications in a wide spectrum of different fields, one important problem that deep learning algorithms face is overfitting. The traditional way of preventing overfitting is to apply regularization; the first option in this sense is the choice of an adequate value for the dropout parameter. In order to demonstrate its ability in finding an optimal dropout rate, the boosted version of the firefly algorithm has been validated for the deep learning subfield of convolutional neural networks, with respect to five standard benchmark datasets for image processing: MNIST, Fashion-MNIST, Semeion, USPS and CIFAR-10. The performance of the proposed approach in both types of experiments was compared with other recent state-of-the-art methods. To prove that there are significant improvements in results, statistical tests were conducted. Based on the experimental data, it can be concluded that the proposed algorithm clearly outperforms other approaches.
TL;DR: Investigation of the potential of two state-of-the-art hybrid methods, namely grasshopper optimization algorithm (GOA) and gray wolf optimization (GWO) in improving the neural assessment of heating load of residential buildings showed that utilizing both GOA and GWO algorithms results in increasing the accuracy of the neural network.
Abstract: The advent of new data-mining techniques and, more recently, swarm-based optimization algorithms have antiquated traditional models in the field of energy performance analysis. This paper investigates the potential of two state-of-the-art hybrid methods, namely grasshopper optimization algorithm (GOA) and gray wolf optimization (GWO) in improving the neural assessment of heating load (HL) of residential buildings. To achieve this goal, eight HL influential factors including glazing area distribution, relative compactness, overall height, surface area, roof area, wall area, orientation, and glazing area are considered for preparing the required dataset. A population-based sensitivity analysis is then carried out to use the best-fitted structures of each ensemble. The results showed that utilizing both GOA and GWO algorithms results in increasing the accuracy of the neural network. From comparison viewpoint, it was found that the GWO (error = 2.2899 and correlation = 0.9551) surpasses GOA (error = 2.4459 and correlation = 0.9486) in adjusting the computational parameters of the proposed neural system.
TL;DR: Wang et al. as discussed by the authors proposed a hybrid algorithm based on PSO and GWO (Hybrid GWO with PSO, HGWOP) to improve the global search ability while retaining the strong exploitation ability of GWO.
TL;DR: An effective approach for damage detection by using a recently developed novel swarm intelligence algorithm, i.e. the marine predator algorithm (MPA), and the superior and stable performance of MPAFNN proves its effectiveness.
TL;DR: This paper presents a comprehensive survey of the meta-heuristic optimization algorithms on the text clustering applications and highlights its main procedures, its advantages and disadvantages, and recommends potential future research paths.
Abstract: This paper presents a comprehensive survey of the meta-heuristic optimization algorithms on the text clustering applications and highlights its main procedures. These Artificial Intelligence (AI) algorithms are recognized as promising swarm intelligence methods due to their successful ability to solve machine learning problems, especially text clustering problems. This paper reviews all of the relevant literature on meta-heuristic-based text clustering applications, including many variants, such as basic, modified, hybridized, and multi-objective methods. As well, the main procedures of text clustering and critical discussions are given. Hence, this review reports its advantages and disadvantages and recommends potential future research paths. The main keywords that have been considered in this paper are text, clustering, meta-heuristic, optimization, and algorithm.
TL;DR: It can be concluded that the newly constructed ELM-based ANSI models can solve the difficulties in tuning the acceleration coefficients of SPSO by the trial-and-error method for predicting the CBR of soils and be further applied to other real-time problems of geotechnical engineering.
TL;DR: The proposed algorithm is a hybrid binary‐real PSO, which includes the combination of categorical and numerical encoding of a particle and a different approach for calculating the velocity of particles, and has the ability to produce effective rules with highest accuracy for the detection of CAD.
Abstract: Background: Coronary artery disease (CAD) is one of the major and important causes of mortality worldwide. The knowledge about the risk factors which increases the probability of developing CAD can help to understand the disease better and also its treatment. Nowadays, many computer-aided approaches have been used for the prediction and diagnosis of diseases. The swarm intelligence algorithms like particle swarm optimization (PSO) have demonstrated great performance in solving different optimization problems. As rule discovery can be modeled as an optimization problem, it can be mapped to an optimization problem and solved by means of an evolutionary algorithm like PSO. Methods: An approach for discovering classification rules of CAD is proposed. The work is based on the real-world CAD dataset and aims at the detection of this disease by producing the accurate and effective rules. An approach based on a hybrid binary-real PSO algorithm is proposed which includes the combination of binary and realvalued encoding of a particle and a different approach for calculating the velocity of particles. The rules were developed from randomly generated particles which take random values in the range of each attribute in the rule. Two different feature selection approaches based on multi-objective evolutionary search and PSO were applied on the dataset and the most relevant features were selected by the algorithms. Results: The accuracy of two different rule sets were evaluated. The rule set with 11 features obtained more accurate results than the rule set with 13 features. Our results show that the proposed approach has the ability to produce effective rules with highest accuracy for the detection of CAD.
TL;DR: In this article, the role of evolutionary machine learning (EC) algorithms in solving different ML challenges has been investigated, including feature selection, resampling, classifiers, neural networks, reinforcement learning, clustering, association rule mining, and ensemble methods.
Abstract: Evolutionary Computation (EC) approaches are inspired by nature and solve optimization problems in a stochastic manner. They can offer a reliable and effective approach to address complex problems in real-world applications. EC algorithms have recently been used to improve the performance of Machine Learning (ML) models and the quality of their results. Evolutionary approaches can be used in all three parts of ML: preprocessing (e.g., feature selection and resampling), learning (e.g., parameter setting, membership functions, and neural network topology), and postprocessing (e.g., rule optimization, decision tree/support vectors pruning, and ensemble learning). This article investigates the role of EC algorithms in solving different ML challenges. We do not provide a comprehensive review of evolutionary ML approaches here; instead, we discuss how EC algorithms can contribute to ML by addressing conventional challenges of the artificial intelligence and ML communities. We look at the contributions of EC to ML in nine sub-fields: feature selection, resampling, classifiers, neural networks, reinforcement learning, clustering, association rule mining, and ensemble methods. For each category, we discuss evolutionary machine learning in terms of three aspects: problem formulation, search mechanisms, and fitness value computation. We also consider open issues and challenges that should be addressed in future work.
TL;DR: The Generalized Particle Swarm Optimization (GEPSO) algorithm is introduced as a new version of the PSO algorithm for continuous space optimization, which enriches the original PSO by incorporating two new terms into the velocity updating equation, which aim to deepen the interrelations of particles and their knowledge sharing, increase variety in the swarm, and provide a better search in unexplored areas of the search space.
01 Jan 2021
TL;DR: The Databionic swarm, called DBS, is introduced which is able to adapt itself to structures of high-dimensional data characterized by distance and/or density-based structures in the data space and can outperform common clustering methods such as K-means, PAM, single linkage, spectral clustering, model-based clustering and Ward.
Abstract: Algorithms implementing populations of agents which interact with one another and sense their environment may exhibit emergent behavior such as self-organization and swarm intelligence. Here a swarm system, called Databionic swarm (DBS), is introduced which is able to adapt itself to structures of high-dimensional data characterized by distance and/or density-based structures in the data space. By exploiting the interrelations of swarm intelligence, self-organization and emergence, DBS serves as an alternative approach to the optimization of a global objective function in the task of clustering. The swarm omits the usage of a global objective function and is parameter-free because it searches for the Nash equilibrium during its annealing process. To our knowledge, DBS is the first swarm combining these approaches. Its clustering can outperform common clustering methods such as K-means, PAM, single linkage, spectral clustering, model-based clustering, and Ward, if no prior knowledge about the data is available. A central problem in clustering is the correct estimation of the number of clusters. This is addressed by a DBS visualization called topographic map which allows assessing the number of clusters. It is known that all clustering algorithms construct clusters, irrespective of the data set contains clusters or not. In contrast to most other clustering algorithms, the topographic map identifies, that clustering of the data is meaningless if the data contains no (natural) clusters. The performance of DBS is demonstrated on a set of benchmark data, which are constructed to pose difficult clustering problems and in two real-world applications.
TL;DR: The Whale Optimization Algorithm is improved and extended to solve multi-objective optimization problems with the purpose of alleviating these drawbacks and demonstrating the superiority of the proposed algorithm compared to some of the existing multi- objective algorithms in the literature.
Abstract: Recently, several meta-heuristics and evolutionary algorithms have been proposed for tackling optimization problems. Such methods tend to suffer from degraded performance when solving multi-objective optimization problems due to addressing the conflicting goals of finding accurate estimation of Pareto optimal solutions and increasing their distribution across all objectives. In this paper, the Whale Optimization Algorithm (WOA) is improved and extended to solve such multi-objective optimization problems with the purpose of alleviating these drawbacks. The improvements include: (1) modifying the distance control factor of the standard WOA to contain values generated dynamically instead of a fixed one, (2) the trade-off between moving toward the opposite of the best solution and its original values based on a certain probability to prevent stuck into local minima, and (3) accelerating the convergence and coverage using Nelder–Mead method and the Pareto Archived Evolution Strategy (PAES). The proposed algorithm is tested on three benchmark multi-objective test functions (DTLZ, CEC 2009, and GLT), including 25 test functions, to verify its effectiveness by comparing with nine robust multi-objective algorithms. The experiments demonstrate the superiority of the proposed algorithm compared to some of the existing multi-objective algorithms in the literature.
TL;DR: A modified COA (MCOA) approach based on chaotic sequences generated by Tinkerbell map to scatter and association probabilities tuning and an adaptive procedure of updating parameters related to social condition is proposed that presented competitive solutions when compared with other state-of-the-art metaheuristic algorithms in terms of results quality.
TL;DR: The statistical analysis revealed that the proposed DWT-PSO-RBFNN method performed better based on MAPE, MAD, and RMSE emphasizing its great potential.
TL;DR: A comprehensive review of firefly algorithm is presented and various characteristics are discussed and the possible future research direction of FA is provided.
Abstract: Firefly Algorithm (FA) is one of the popular algorithm of Swarm Intelligence domain that can be used in most of the areas of optimization. FA and its variants are simple to implement and easily understood. These can be used to successfully solve the problems of different areas. Modification in original FA or hybrid FA algorithms are required to solve diverse range of engineering problems. In this paper, a comprehensive review of firefly algorithm is presented and various characteristics are discussed. The various variant of FA such as binary, multi-objective and hybrid with other meta-heuristics are discussed. The applications and performance evolution metric are presented. This paper provides the possible future research direction of FA.
TL;DR: Crow Search Algorithm (CSA) is a swarm intelligence optimization algorithm inspired by the social intelligent behavior of crows for hiding food as mentioned in this paper.CSA has been widely used to solve a large variety of optimization problems in several fields and areas of research and has proved its efficiency compared to several state-of-the-art optimization algorithms available in the literature.
Abstract: Crow Search Algorithm (CSA) is a recent swarm intelligence optimization algorithm inspired by the social intelligent behavior of crows for hiding food. It has been widely used to solve a large variety of optimization problems in several fields and areas of research and has proved its efficiency compared to several state-of-the-art optimization algorithms available in the literature. This paper presents a comprehensive overview of Crow Search Algorithm and its new variants categorized into modified and hybridized versions. It also describes the several applications of CSA in various domains such as feature selection, image processing, scheduling, economic dispatch, distributed generation, and other engineering problems. In addition, the paper suggests some interesting research areas related to CSA enhancement, CSA hybridization, and possible new applications.
TL;DR: A novel swarm intelligence-based algorithm is proposed for producing the multiobjective optimal overtaking trajectory of autonomous ground vehicles and the designed strategy tends to be more capable in terms of producing a set of widespread and high-quality Pareto-optimal solutions.
Abstract: Constrained autonomous vehicle overtaking trajectories are usually difficult to generate due to certain practical requirements and complex environmental limitations. This problem becomes more challenging when multiple contradicting objectives are required to be optimized and the on-road objects to be overtaken are irregularly placed. In this article, a novel swarm intelligence-based algorithm is proposed for producing the multiobjective optimal overtaking trajectory of autonomous ground vehicles. The proposed method solves a multiobjective optimal control model in order to optimize the maneuver time duration, the trajectory smoothness, and the vehicle visibility, while taking into account different types of mission-dependent constraints. However, one problem that could have an impact on the optimization process is the selection of algorithm control parameters. To desensitize the negative influence, a novel fuzzy adaptive strategy is proposed and embedded in the algorithm framework. This allows the optimization process to dynamically balance the local exploitation and global exploration, thereby exploring the tradeoff between objectives more effectively. The performance of using the designed fuzzy adaptive multiobjective method is analyzed and validated by executing a number of simulation studies. The results confirm the effectiveness of applying the proposed algorithm to produce multiobjective optimal overtaking trajectories for autonomous ground vehicles. Moreover, the comparison to other state-of-the-art multiobjective optimization schemes shows that the designed strategy tends to be more capable in terms of producing a set of widespread and high-quality Pareto-optimal solutions.
TL;DR: In this article, the use of metaheuristic search for optimal power flow (OPF), scheduling and planning in the context of the smart grid has been reviewed and discussed extensively with regard to problem handling, multi-objective optimization performance and method accuracy in relation to computational complexity.
Abstract: A multitude of optimization tasks ensue in the context of the smart grid, often exhibiting undesirable characteristics like non-convexity, mixed types of design variables and multiple - and often conflicting - objectives. These tasks can be broadly categorized into three classes of problems, namely optimal power flow (OPF), scheduling and planning. Metaheuristic search methods form a generic class of optimization techniques, that have been shown to work successfully for complex problems. Not surprisingly, they have been widely applied in the smart grid, their use spanning almost every smart grid-related optimization task. In this work, we review the use of metaheuristic search for OPF, scheduling and planning through a unified approach, keeping in mind that these problems share many common challenges and objectives. The use of different metaheuristic methods is discussed extensively with regard to problem handling, multi-objective optimization performance and method accuracy in relation to computational complexity. An attempt to arrive at quantitative conclusions is also being made, by compiling tables which present collective results on common test grids. Lastly, the paper identifies promising directions for future research, concerning metaheuristic search application practices, method development and new challenges that we believe will shape the future of smart grid optimization.
TL;DR: In this paper, a new ant colony optimization with the Cauchy mutation and the greedy Levy mutation, termed CLACO, was presented for continuous domains. But the performance of the algorithm was not evaluated in terms of search capability and convergence speed.
TL;DR: A Local-Global best Bat Algorithm for Neural Networks (LGBA-NN) to select both feature subsets and hyperparameters for efficient detection of botnet attacks, inferred from 9 commercial IoT devices infected by two botnets: Gafgyt and Mirai.
Abstract: The need for timely identification of Distributed Denial-of-Service (DDoS) attacks in the Internet of Things (IoT) has become critical in minimizing security risks as the number of IoT devices deployed rapidly grows globally and the volume of such attacks rises to unprecedented levels. Instant detection facilitates network security by speeding up warning and disconnection from the network of infected IoT devices, thereby preventing the botnet from propagating and thereby stopping additional attacks. Several methods have been developed for detecting botnet attacks, such as Swarm Intelligence (SI) and Evolutionary Computing (EC)-based algorithms. In this study, we propose a Local-Global best Bat Algorithm for Neural Networks (LGBA-NN) to select both feature subsets and hyperparameters for efficient detection of botnet attacks, inferred from 9 commercial IoT devices infected by two botnets: Gafgyt and Mirai. The proposed Bat Algorithm (BA) adopted the local-global best-based inertia weight to update the bat’s velocity in the swarm. To tackle with swarm diversity of BA, we proposed Gaussian distribution used in the population initialization. Furthermore, the local search mechanism was followed by the Gaussian density function and local-global best function to achieve better exploration during each generation. Enhanced BA was further employed for neural network hyperparameter tuning and weight optimization to classify ten different botnet attacks with an additional one benign target class. The proposed LGBA-NN algorithm was tested on an N-BaIoT data set with extensive real traffic data with benign and malicious target classes. The performance of LGBA-NN was compared with several recent advanced approaches such as weight optimization using Particle Swarm Optimization (PSO-NN) and BA-NN. The experimental results revealed the superiority of LGBA-NN with 90% accuracy over other variants, i.e., BA-NN (85.5% accuracy) and PSO-NN (85.2% accuracy) in multi-class botnet attack detection.
TL;DR: A comprehensive survey of the recent optimization methods employed to enhance DNNs performance on various tasks is presented and the importance of optimization methods in generating the optimal hyper-parameters and structures of Dnns in taking into consideration massive-scale data is analyzed.
Abstract: Deep neural networks (DNNs) have evolved as a beneficial machine learning method that has been successfully used in various applications. Currently, DNN is a superior technique of extracting information from massive sets of data in a self-organized method. DNNs have different structures and parameters, which are usually produced for particular applications. Nevertheless, the training procedures of DNNs can be protracted depending on the given application and the size of the training set. Further, determining the most precise and practical structure of a deep learning method in a reasonable time is a possible problem related to this procedure. Meta-heuristics techniques, such as swarm intelligence (SI) and evolutionary computing (EC), represent optimization frames with specific theories and objective functions. These methods are adjustable and have been demonstrated their effectiveness in various applications; hence, they can optimize the DNNs models. This paper presents a comprehensive survey of the recent optimization methods (i.e., SI and EC) employed to enhance DNNs performance on various tasks. This paper also analyzes the importance of optimization methods in generating the optimal hyper-parameters and structures of DNNs in taking into consideration massive-scale data. Finally, several potential directions that still need improvements and open problems in evolutionary DNNs are identified.