Differential evolution (DE) is arguably one of the most powerful stochastic real-parameter optimization algorithms in current use. DE operates through similar computational steps as employed by a standard evolutionary algorithm (EA). However, unlike traditional EAs, the DE-variants perturb the current-generation population members with the scaled differences of randomly selected and distinct population members. Therefore, no separate probability distribution has to be used for generating the offspring. Since its inception in 1995, DE has drawn the attention of many researchers all over the world resulting in a lot of variants of the basic algorithm with improved performance. This paper presents a detailed review of the basic concepts of DE and a survey of its major variants, its application to multiobjective, constrained, large scale, and uncertain optimization problems, and the theoretical studies conducted on DE so far. Also, it provides an overview of the significant engineering applications that have benefited from the powerful nature of DE.

/pdf/differential-evolution-a-survey-of-the-state-of-the-art-3lz840pz2t.pdf

Differential Evolution: A Survey of the State-of-the-Art

Differential evolution (DE) is an efficient and powerful population-based stochastic search technique for solving optimization problems over continuous space, which has been widely applied in many scientific and engineering fields. However, the success of DE in solving a specific problem crucially depends on appropriately choosing trial vector generation strategies and their associated control parameter values. Employing a trial-and-error scheme to search for the most suitable strategy and its associated parameter settings requires high computational costs. Moreover, at different stages of evolution, different strategies coupled with different parameter settings may be required in order to achieve the best performance. In this paper, we propose a self-adaptive DE (SaDE) algorithm, in which both trial vector generation strategies and their associated control parameter values are gradually self-adapted by learning from their previous experiences in generating promising solutions. Consequently, a more suitable generation strategy along with its parameter settings can be determined adaptively to match different phases of the search process/evolution. The performance of the SaDE algorithm is extensively evaluated (using codes available from P. N. Suganthan) on a suite of 26 bound-constrained numerical optimization problems and compares favorably with the conventional DE and several state-of-the-art parameter adaptive DE variants.

Differential Evolution Algorithm With Strategy Adaptation for Global Numerical Optimization

A new differential evolution (DE) algorithm, JADE, is proposed to improve optimization performance by implementing a new mutation strategy ldquoDE/current-to-p bestrdquo with optional external archive and updating control parameters in an adaptive manner. The DE/current-to-pbest is a generalization of the classic ldquoDE/current-to-best,rdquo while the optional archive operation utilizes historical data to provide information of progress direction. Both operations diversify the population and improve the convergence performance. The parameter adaptation automatically updates the control parameters to appropriate values and avoids a user's prior knowledge of the relationship between the parameter settings and the characteristics of optimization problems. It is thus helpful to improve the robustness of the algorithm. Simulation results show that JADE is better than, or at least comparable to, other classic or adaptive DE algorithms, the canonical particle swarm optimization, and other evolutionary algorithms from the literature in terms of convergence performance for a set of 20 benchmark problems. JADE with an external archive shows promising results for relatively high dimensional problems. In addition, it clearly shows that there is no fixed control parameter setting suitable for various problems or even at different optimization stages of a single problem.

JADE: Adaptive Differential Evolution With Optional External Archive

Recent research has revealed that the output of deep neural networks (DNNs) can be easily altered by adding relatively small perturbations to the input vector. In this paper, we analyze an attack in an extremely limited scenario where only one pixel can be modified. For that we propose a novel method for generating one-pixel adversarial perturbations based on differential evolution (DE). It requires less adversarial information (a black-box attack) and can fool more types of networks due to the inherent features of DE. The results show that 67.97% of the natural images in Kaggle CIFAR-10 test dataset and 16.04% of the ImageNet (ILSVRC 2012) test images can be perturbed to at least one target class by modifying just one pixel with 74.03% and 22.91% confidence on average. We also show the same vulnerability on the original CIFAR-10 dataset. Thus, the proposed attack explores a different take on adversarial machine learning in an extreme limited scenario, showing that current DNNs are also vulnerable to such low dimension attacks. Besides, we also illustrate an important application of DE (or broadly speaking, evolutionary computation) in the domain of adversarial machine learning: creating tools that can effectively generate low-cost adversarial attacks against neural networks for evaluating robustness.

One Pixel Attack for Fooling Deep Neural Networks

Evolutionary algorithms (EAs) are well-known optimization approaches to deal with nonlinear and complex problems. However, these population-based algorithms are computationally expensive due to the slow nature of the evolutionary process. This paper presents a novel algorithm to accelerate the differential evolution (DE). The proposed opposition-based DE (ODE) employs opposition-based learning (OBL) for population initialization and also for generation jumping. In this work, opposite numbers have been utilized to improve the convergence rate of DE. A comprehensive set of 58 complex benchmark functions including a wide range of dimensions is employed for experimental verification. The influence of dimensionality, population size, jumping rate, and various mutation strategies are also investigated. Additionally, the contribution of opposite numbers is empirically verified. We also provide a comparison of ODE to fuzzy adaptive DE (FADE). Experimental results confirm that the ODE outperforms the original DE and FADE in terms of convergence speed and solution accuracy.

/pdf/opposition-based-differential-evolution-14k11phz34.pdf

Opposition-Based Differential Evolution

We describe an efficient technique for adapting control parameter settings associated with differential evolution (DE). The DE algorithm has been used in many practical cases and has demonstrated good convergence properties. It has only a few control parameters, which are kept fixed throughout the entire evolutionary process. However, it is not an easy task to properly set control parameters in DE. We present an algorithm-a new version of the DE algorithm-for obtaining self-adaptive control parameter settings that show good performance on numerical benchmark problems. The results show that our algorithm with self-adaptive control parameter settings is better than, or at least comparable to, the standard DE algorithm and evolutionary algorithms from literature when considering the quality of the solutions obtained

Self-Adapting Control Parameters in Differential Evolution: A Comparative Study on Numerical Benchmark Problems

Solving single objective real-parameter optimization problems, also known as a bound-constrained optimization, is still a challenging task. We can find such problems in engineering optimization, scientific applications, and in other real-world problems. Usually, these problems are very complex and computationally expensive. A new algorithm, called jSO, is presented in this paper. The algorithm is an improved variant of the iL-SHADE algorithm, mainly with a new weighted version of mutation strategy. The experiments were performed on CEC 2017 benchmark functions, which are different from previous competition benchmark functions. A comparison of the proposed jSO algorithm and the L-SHADE algorithm is presented first. From the obtained results we can conclude that jSO performs better in comparison with the L-SHADE algorithm. Next, a comparison of jSO and iL-SHADE is also performed, and jSO obtained better or competitive results. Using the CEC 2017 evaluation method, jSO obtained the best final score among these three algorithms.

Single objective real-parameter optimization: Algorithm jSO

Differential evolution (DE) has been shown to be a simple, yet powerful, evolutionary algorithm for global optimization for many real problems. Adaptation, especially self-adaptation, has been found to be highly beneficial for adjusting control parameters, especially when done without any user interaction. This paper presents differential evolution algorithms, which use different adaptive or self-adaptive mechanisms applied to the control parameters. Detailed performance comparisons of these algorithms on the benchmark functions are outlined.

Performance comparison of self-adaptive and adaptive differential evolution algorithms

In this paper we investigate a Self-Adaptive Differential Evolution algorithm (jDE) where F and CR control parameters are self-adapted and a multi-population method with aging mechanism is used. The performance of the jDE algorithm is evaluated on the set of benchmark functions provided for the CEC 2009 special session on evolutionary computation in dynamic and uncertain environments.

https://www.cs.le.ac.uk/people/syang/ECiDUE/Competition09-Papers/4982976.pdf

Dynamic optimization using Self-Adaptive Differential Evolution

In this paper we investigate a self-adaptive differential evolution algorithm (jDEdynNP-F) where F and CR control parameters are self-adapted and a population size reduction method is used. Additionally the proposed jDEdynNP-F algorithm uses a mechanism for sign changing of F control parameter with some probability based on the fitness values of randomly chosen vectors, which are multiplied by the F control parameter (scaling factor) in the mutation operation of DE algorithm. The performance of the jDEdynNP-F algorithm is evaluated on the set of 7 benchmark functions provided for the CECpsila2008 special session on high-dimensional real-parameter optimization.

/pdf/high-dimensional-real-parameter-optimization-using-self-10vypcyq3o.pdf

Borko Boskovic

Papers

Self-Adapting Control Parameters in Differential Evolution: A Comparative Study on Numerical Benchmark Problems

Single objective real-parameter optimization: Algorithm jSO

Performance comparison of self-adaptive and adaptive differential evolution algorithms

Dynamic optimization using Self-Adaptive Differential Evolution

High-dimensional real-parameter optimization using Self-Adaptive Differential Evolution algorithm with population size reduction