Differential evolution (DE) is arguably one of the most powerful stochastic real-parameter optimization algorithms in current use. DE operates through similar computational steps as employed by a standard evolutionary algorithm (EA). However, unlike traditional EAs, the DE-variants perturb the current-generation population members with the scaled differences of randomly selected and distinct population members. Therefore, no separate probability distribution has to be used for generating the offspring. Since its inception in 1995, DE has drawn the attention of many researchers all over the world resulting in a lot of variants of the basic algorithm with improved performance. This paper presents a detailed review of the basic concepts of DE and a survey of its major variants, its application to multiobjective, constrained, large scale, and uncertain optimization problems, and the theoretical studies conducted on DE so far. Also, it provides an overview of the significant engineering applications that have benefited from the powerful nature of DE.

/pdf/differential-evolution-a-survey-of-the-state-of-the-art-3lz840pz2t.pdf

Differential Evolution: A Survey of the State-of-the-Art

This paper presents a variant of particle swarm optimizers (PSOs) that we call the comprehensive learning particle swarm optimizer (CLPSO), which uses a novel learning strategy whereby all other particles' historical best information is used to update a particle's velocity. This strategy enables the diversity of the swarm to be preserved to discourage premature convergence. Experiments were conducted (using codes available from http://www.ntu.edu.sg/home/epnsugan) on multimodal test functions such as Rosenbrock, Griewank, Rastrigin, Ackley, and Schwefel and composition functions both with and without coordinate rotation. The results demonstrate good performance of the CLPSO in solving multimodal problems when compared with eight other recent variants of the PSO.

Comprehensive learning particle swarm optimizer for global optimization of multimodal functions

A new differential evolution (DE) algorithm, JADE, is proposed to improve optimization performance by implementing a new mutation strategy ldquoDE/current-to-p bestrdquo with optional external archive and updating control parameters in an adaptive manner. The DE/current-to-pbest is a generalization of the classic ldquoDE/current-to-best,rdquo while the optional archive operation utilizes historical data to provide information of progress direction. Both operations diversify the population and improve the convergence performance. The parameter adaptation automatically updates the control parameters to appropriate values and avoids a user's prior knowledge of the relationship between the parameter settings and the characteristics of optimization problems. It is thus helpful to improve the robustness of the algorithm. Simulation results show that JADE is better than, or at least comparable to, other classic or adaptive DE algorithms, the canonical particle swarm optimization, and other evolutionary algorithms from the literature in terms of convergence performance for a set of 20 benchmark problems. JADE with an external archive shows promising results for relatively high dimensional problems. In addition, it clearly shows that there is no fixed control parameter setting suitable for various problems or even at different optimization stages of a single problem.

JADE: Adaptive Differential Evolution With Optional External Archive

Many areas in power systems require solving one or more nonlinear optimization problems. While analytical methods might suffer from slow convergence and the curse of dimensionality, heuristics-based swarm intelligence can be an efficient alternative. Particle swarm optimization (PSO), part of the swarm intelligence family, is known to effectively solve large-scale nonlinear optimization problems. This paper presents a detailed overview of the basic concepts of PSO and its variants. Also, it provides a comprehensive survey on the power system applications that have benefited from the powerful nature of PSO as an optimization technique. For each application, technical details that are required for applying PSO, such as its type, particle formulation (solution representation), and the most efficient fitness functions are also discussed.

/pdf/particle-swarm-optimization-basic-concepts-variants-and-4h7hr86hk4.pdf

Particle Swarm Optimization: Basic Concepts, Variants and Applications in Power Systems

The particle swarm optimizer (PSO) is a stochastic, population-based optimization technique that can be applied to a wide range of problems, including neural network training. This paper presents a variation on the traditional PSO algorithm, called the cooperative particle swarm optimizer, or CPSO, employing cooperative behavior to significantly improve the performance of the original algorithm. This is achieved by using multiple swarms to optimize different components of the solution vector cooperatively. Application of the new PSO algorithm on several benchmark optimization problems shows a marked improvement in performance over the traditional PSO.

A Cooperative approach to particle swarm optimization

The optimization of biotechnological processes is a complex problem that has been intensively studied in the past few years due to the economic impact of the products obtained from fermentations. In fed-batch processes, the goal is to fi nd the optimal feeding trajectory that maximizes the fi nal productivity. Several methods, including Evolutionary Algorithms (EAs) have been applied to this task in a number of different fermentation processes. This paper performs an experimental comparison between Particle Swarm Optimization, Differential Evolution and a real-valued EA in three distinct case studies, taken from previous work by the authors and literature, all considering the optimization of fed-batch fermentation processes.

/pdf/a-comparison-of-algorithms-for-the-optimization-of-3c0itdpefl.pdf

A Comparison of Algorithms for the Optimization of Fermentation Processes

The worldwide adoption of mobile devices is raising the value of Mobile Performance Marketing, which is supported by Demand-Side Platforms (DSP) that match mobile users to advertisements. In these markets, monetary compensation only occurs when there is a user conversion. Thus, a key DSP issue is the design of a data-driven model to predict user conversion. To handle this nontrivial task, we propose a novel Multi-objective Optimization (MO) approach to evolve Decision Trees (DT) using a Grammatical Evolution (GE), under two main variants: a pure GE method (MGEDT) and a GE with Lamarckian Evolution (MGEDTL). Both variants evolve variable-length DTs and perform a simultaneous optimization of the predictive performance and model complexity. To handle big data, the GE methods include a training sampling and parallelism evaluation mechanism. The algorithms were applied to a recent database with around 6 million records from a real-world DSP. Using a realistic Rolling Window (RW) validation, the two GE variants were compared with a standard DT algorithm (CART), a Random Forest and a state-of-the-art Deep Learning (DL) model. Competitive results were obtained by the GE methods, which present affordable training times and very fast predictive response times.

Multi-objective grammatical evolution of decision trees for mobile marketing user conversion prediction

Mobile performance marketing is a growing industry due to the massive adoption of smartphones and tablets. In this paper, we explore Deep Multilayer Perceptrons (MLP) to predict the Conversion Rate (CVR) of mobile users that are redirected to ad campaigns (i.e., if there will be a sale). We analyze recent real-world big data provided by a global mobile marketing company. Using a realistic rolling window validation, we conducted several experiments with different datasets (two sampling and two data traffic modes), in which we measure both the predictive binary classification performance and the computational effort. The modeling experiments include: two data preprocessing methods, the popular one-hot encoding and a proposed Percentage Categorical Pruning (PCP); and two MLP learning modes, offline (reset) and online (reuse). Overall, competitive classification results were achieved by the PCP transform and the two MLP learning modes, producing real-time predictions and comparing favorably against a Convolutional Neural Network and a Logistic Regression.

Using Deep Learning for Mobile Marketing User Conversion Prediction

Multi-step time series forecasting (TSF) is a crucial element to support tactical decisions (e.g., designing production or marketing plans several months in advance). While most TSF research addresses only single-point prediction, prediction intervals (PIs) are useful to reduce uncertainty related to important decision making variables. In this paper, we explore a large set of neural network methods for multi-step TSF and that directly optimize PIs. This includes multi-step adaptations of recently proposed PI methods, such as lower–upper bound estimation (LUBET), its ensemble extension (LUBEXT), a multi-objective evolutionary algorithm LUBE (MLUBET) and a two-phase learning multi-objective evolutionary algorithm (M2LUBET). We also explore two new ensemble variants for the evolutionary approaches based on two PI coverage–width split methods (radial slices and clustering), leading to the MLUBEXT, M2LUBEXT, MLUBEXT2 and M2LUBEXT2 methods. A robust comparison was held by considering the rolling window procedure, nine time series from several real-world domains and with different characteristics, two PI quality measures (coverage error and width) and the Wilcoxon statistic. Overall, the best results were achieved by the M2LUBET neuroevolution method, which requires a reasonable computational effort for time series with a few hundreds of observations.

/pdf/multi-step-time-series-prediction-intervals-using-l0w0tdxu93.pdf

Multi-step time series prediction intervals using neuroevolution

Previous studies in Particle Swarm Optimization (PSO) have emphasized the role of population topologies in particle swarms. These studies have shown that a relationship between the way individuals in a population are organized and their aptitude to find global optima exists. A study of what graph statistics are relevant is of paramount importance. This work presents such a study, which will provide guidelines that can be used by researchers in the field of PSO in particular and in the Evolutionary Computation arena in general.

Rui Mendes

Papers

A Comparison of Algorithms for the Optimization of Fermentation Processes

Multi-objective grammatical evolution of decision trees for mobile marketing user conversion prediction

Using Deep Learning for Mobile Marketing User Conversion Prediction

Multi-step time series prediction intervals using neuroevolution

What Makes a Successful Society? Experiments with Population Topologies in Particle Swarms