Showing papers in "IEEE Transactions on Evolutionary Computation in 2020"

Evolving Deep Convolutional Neural Networks for Image Classification

Abstract: Evolutionary paradigms have been successfully applied to neural network designs for two decades. Unfortunately, these methods cannot scale well to the modern deep neural networks due to the complicated architectures and large quantities of connection weights. In this paper, we propose a new method using genetic algorithms for evolving the architectures and connection weight initialization values of a deep convolutional neural network to address image classification problems. In the proposed algorithm, an efficient variable-length gene encoding strategy is designed to represent the different building blocks and the potentially optimal depth in convolutional neural networks. In addition, a new representation scheme is developed for effectively initializing connection weights of deep convolutional neural networks, which is expected to avoid networks getting stuck into local minimum that is typically a major issue in the backward gradient-based optimization. Furthermore, a novel fitness evaluation method is proposed to speed up the heuristic search with substantially less computational resource. The proposed algorithm is examined and compared with 22 existing algorithms on nine widely used image classification tasks, including the state-of-the-art methods. The experimental results demonstrate the remarkable superiority of the proposed algorithm over the state-of-the-art designs in terms of classification error rate and the number of parameters (weights).

Multifactorial Evolutionary Algorithm With Online Transfer Parameter Estimation: MFEA-II

Abstract: Humans rarely tackle every problem from scratch. Given this observation, the motivation for this paper is to improve optimization performance through adaptive knowledge transfer across related problems. The scope for spontaneous transfers under the simultaneous occurrence of multiple problems unveils the benefits of multitasking. Multitask optimization has recently demonstrated competence in solving multiple (related) optimization tasks concurrently. Notably, in the presence of underlying relationships between problems, the transfer of high-quality solutions across them has shown to facilitate superior performance characteristics. However, in the absence of any prior knowledge about the intertask synergies (as is often the case with general black-box optimization), the threat of predominantly negative transfer prevails. Susceptibility to negative intertask interactions can impede the overall convergence behavior. To allay such fears, in this paper, we propose a novel evolutionary computation framework that enables online learning and exploitation of the similarities (and discrepancies) between distinct tasks in multitask settings, for an enhanced optimization process. Our proposal is based on the principled theoretical arguments that seek to minimize the tendency of harmful interactions between tasks, based on a purely data-driven learning of relationships among them. The efficacy of our proposed method is validated experimentally on a series of synthetic benchmarks, as well as a practical study that provides insights into the behavior of the method in the face of several tasks occurring at once.

A Survey of Automatic Parameter Tuning Methods for Metaheuristics

Abstract: Parameter tuning, that is, to find appropriate parameter settings (or configurations) of algorithms so that their performance is optimized, is an important task in the development and application of metaheuristics. Automating this task, i.e., developing algorithmic procedure to address parameter tuning task, is highly desired and has attracted significant attention from the researchers and practitioners. During last two decades, many automatic parameter tuning approaches have been proposed. This paper presents a comprehensive survey of automatic parameter tuning methods for metaheuristics. A new classification (or taxonomy) of automatic parameter tuning methods is introduced according to the structure of tuning methods. The existing automatic parameter tuning approaches are consequently classified into three categories: 1) simple generate-evaluate methods; 2) iterative generate-evaluate methods; and 3) high-level generate-evaluate methods. Then, these three categories of tuning methods are reviewed in sequence. In addition to the description of each tuning method, its main strengths and weaknesses are discussed, which is helpful for new researchers or practitioners to select appropriate tuning methods to use. Furthermore, some challenges and directions of this field are pointed out for further research.

Variable-Size Cooperative Coevolutionary Particle Swarm Optimization for Feature Selection on High-Dimensional Data

Abstract: Evolutionary feature selection (FS) methods face the challenge of “curse of dimensionality” when dealing with high-dimensional data. Focusing on this challenge, this article studies a variable-size cooperative coevolutionary particle swarm optimization algorithm (VS-CCPSO) for FS. The proposed algorithm employs the idea of “divide and conquer” in cooperative coevolutionary approach, but several new developed problem-guided operators/strategies make it more suitable for FS problems. First, a space division strategy based on the feature importance is presented, which can classify relevant features into the same subspace with a low computational cost. Following that, an adaptive adjustment mechanism of subswarm size is developed to maintain an appropriate size for each subswarm, with the purpose of saving computational cost on evaluating particles. Moreover, a particle deletion strategy based on fitness-guided binary clustering, and a particle generation strategy based on feature importance and crossover both are designed to ensure the quality of particles in the subswarms. We apply VS-CCPSO to 12 typical datasets and compare it with six state-of-the-art methods. The experimental results show that VS-CCPSO has the capability of obtaining good feature subsets, suggesting its competitiveness for tackling FS problems with high dimensionality.

An Evolutionary Algorithm for Large-Scale Sparse Multiobjective Optimization Problems

Abstract: In the last two decades, a variety of different types of multiobjective optimization problems (MOPs) have been extensively investigated in the evolutionary computation community. However, most existing evolutionary algorithms encounter difficulties in dealing with MOPs whose Pareto optimal solutions are sparse (i.e., most decision variables of the optimal solutions are zero), especially when the number of decision variables is large. Such large-scale sparse MOPs exist in a wide range of applications, for example, feature selection that aims to find a small subset of features from a large number of candidate features, or structure optimization of neural networks whose connections are sparse to alleviate overfitting. This paper proposes an evolutionary algorithm for solving large-scale sparse MOPs. The proposed algorithm suggests a new population initialization strategy and genetic operators by taking the sparse nature of the Pareto optimal solutions into consideration, to ensure the sparsity of the generated solutions. Moreover, this paper also designs a test suite to assess the performance of the proposed algorithm for large-scale sparse MOPs. The experimental results on the proposed test suite and four application examples demonstrate the superiority of the proposed algorithm over seven existing algorithms in solving large-scale sparse MOPs.

Automatic Niching Differential Evolution With Contour Prediction Approach for Multimodal Optimization Problems

Abstract: Niching techniques have been widely incorporated into evolutionary algorithms (EAs) for solving multimodal optimization problems (MMOPs). However, most of the existing niching techniques are either sensitive to the niching parameters or require extra fitness evaluations (FEs) to maintain the niche detection accuracy. In this paper, we propose a new automatic niching technique based on the affinity propagation clustering (APC) and design a novel niching differential evolution (DE) algorithm, termed as automatic niching DE (ANDE), for solving MMOPs. In the proposed ANDE algorithm, APC acts as a parameter-free automatic niching method that does not need to predefine the number of clusters or the cluster size. Also, it can facilitate locating multiple peaks without extra FEs. Furthermore, the ANDE algorithm is enhanced by a contour prediction approach (CPA) and a two-level local search (TLLS) strategy. First, the CPA is a predictive search strategy. It exploits the individual distribution information in each niche to estimate the contour landscape, and then predicts the rough position of the potential peak to help accelerate the convergence speed. Second, the TLLS is a solution refine strategy to further increase the solution accuracy after the CPA roughly predicting the peaks. Compared with the other state-of-the-art DE and non-DE multimodal algorithms, even the winner of competition on multimodal optimization, the experimental results on 20 widely used benchmark functions illustrate the superiority of the proposed ANDE algorithm.

Surrogate-Assisted Evolutionary Deep Learning Using an End-to-End Random Forest-Based Performance Predictor

Abstract: Convolutional neural networks (CNNs) have shown remarkable performance in various real-world applications. Unfortunately, the promising performance of CNNs can be achieved only when their architectures are optimally constructed. The architectures of state-of-the-art CNNs are typically handcrafted with extensive expertise in both CNNs and the investigated data, which consequently hampers the widespread adoption of CNNs for less experienced users. Evolutionary deep learning (EDL) is able to automatically design the best CNN architectures without much expertise. However, the existing EDL algorithms generally evaluate the fitness of a new architecture by training from scratch, resulting in the prohibitive computational cost even operated on high-performance computers. In this paper, an end-to-end offline performance predictor based on the random forest is proposed to accelerate the fitness evaluation in EDL. The proposed performance predictor shows the promising performance in term of the classification accuracy and the consumed computational resources when compared with 18 state-of-the-art peer competitors by integrating into an existing EDL algorithm as a case study. The proposed performance predictor is also compared with the other two representatives of existing performance predictors. The experimental results show the proposed performance predictor not only significantly speeds up the fitness evaluations but also achieves the best prediction among the peer performance predictors.

Self-Regulated Evolutionary Multitask Optimization

Abstract: Evolutionary multitask optimization (EMTO) is a newly emerging research area in the field of evolutionary computation. It investigates how to solve multiple optimization problems (tasks) at the same time via evolutionary algorithms (EAs) to improve on the performance of solving each task independently, assuming if some component tasks are related then the useful knowledge (e.g., promising candidate solutions) acquired during the process of solving one task may assist in (and also benefit from) solving the other tasks. In EMTO, task relatedness is typically unknown in advance and needs to be captured via EA’s population. Since the population of an EA can only cover a subregion of the solution space and keeps evolving during the search, thus captured task relatedness is local and dynamic. The multifactorial EA (MFEA) is one of the most representative EMTO techniques, inspired by the bio-cultural model of multifactorial inheritance, which transmits both biological and cultural traits from the parents to the offspring. MFEA has succeeded in solving various multitask optimization (MTO) problems. However, the intensity of knowledge transfer in MFEA is determined via its algorithmic configuration without considering the degree of task relatedness, which may prevent the effective sharing and utilization of the useful knowledge acquired in related tasks. To address this issue, we propose a self-regulated EMTO (SREMTO) algorithm to automatically adapt the intensity of cross-task knowledge transfer to different and varying degrees of relatedness between different tasks as the search proceeds so that the useful knowledge in common for solving related tasks can be captured, shared, and utilized to a great extent. We compare SREMTO with MFEA and its variants as well as the single-task optimization counterpart of SREMTO on two MTO test suites, which demonstrates the superiority of SREMTO.

A Similarity-Based Cooperative Co-Evolutionary Algorithm for Dynamic Interval Multiobjective Optimization Problems

Abstract: Dynamic interval multiobjective optimization problems (DI-MOPs) are very common in real-world applications. However, there are few evolutionary algorithms (EAs) that are suitable for tackling DI-MOPs up to date. A framework of dynamic interval multiobjective cooperative co-evolutionary optimization based on the interval similarity is presented in this paper to handle DI-MOPs. In the framework, a strategy for decomposing decision variables is first proposed, through which all the decision variables are divided into two groups according to the interval similarity between each decision variable and interval parameters. Following that, two subpopulations are utilized to cooperatively optimize decision variables in the two groups. Furthermore, two response strategies, i.e., a strategy based on the change intensity and a random mutation strategy, are employed to rapidly track the changing Pareto front of the optimization problem. The proposed algorithm is applied to eight benchmark optimization instances as well as a multiperiod portfolio selection problem and compared with five state-of-the-art EAs. The experimental results reveal that the proposed algorithm is very competitive on most optimization instances.

Boosting Data-Driven Evolutionary Algorithm With Localized Data Generation

Abstract: By efficiently building and exploiting surrogates, data-driven evolutionary algorithms (DDEAs) can be very helpful in solving expensive and computationally intensive problems. However, they still often suffer from two difficulties. First, many existing methods for building a single ad hoc surrogate are suitable for some special problems but may not work well on some other problems. Second, the optimization accuracy of DDEAs deteriorates if available data are not enough for building accurate surrogates, which is common in expensive optimization problems. To this end, this article proposes a novel DDEA with two efficient components. First, a boosting strategy (BS) is proposed for self-aware model managements, which can iteratively build and combine surrogates to obtain suitable surrogate models for different problems. Second, a localized data generation (LDG) method is proposed to generate synthetic data to alleviate data shortage and increase data quantity, which is achieved by approximating fitness through data positions. By integrating the BS and the LDG, the BDDEA-LDG algorithm is able to improve model accuracy and data quantity at the same time automatically according to the problems at hand. Besides, a tradeoff is empirically considered to strike a better balance between the effectiveness of surrogates and the time cost for building them. The experimental results show that the proposed BDDEA-LDG algorithm can generally outperform both traditional methods without surrogates and other state-of-the-art DDEA son widely used benchmarks and an arterial traffic signal timing real-world optimization problem. Furthermore, the proposed BDDEA-LDG algorithm can use only about 2% computational budgets of traditional methods for producing competitive results.

Novel Interactive Preference-Based Multiobjective Evolutionary Optimization for Bolt Supporting Networks

Abstract: Previous methods of designing a bolt supporting network, which depend on engineering experiences, seek optimal bolt supporting schemes in terms of supporting quality. The supporting cost and time, however, have not been considered, which restricts their applications in real-world situations. We formulate the problem of designing a bolt supporting network as a three-objective optimization model by simultaneously considering such indicators as quality, economy, and efficiency. Especially, two surrogate models are constructed by support vector regression for roof-to-floor convergence and the two-sided displacement, respectively, so as to rapidly evaluate supporting quality during optimization. To solve the formulated model, a novel interactive preference-based multiobjective evolutionary algorithm is proposed. The highlight of generic methods which interactively articulate preferences is to systematically manage the regions of interest by three steps, that is, “partitioning-updating-tracking” in accordance with the cognition process of human. The preference regions of a decision-maker (DM) are first articulated and employed to narrow down the feasible objective space before the evolution in terms of nadir point, not the commonly used ideal point. Then, the DM’s preferences are tracked by dynamically updating these preference regions based on satisfactory candidates during the evolution. Finally, individuals in the population are evaluated based on the preference regions. We apply the proposed model and algorithm to design the bolt supporting network of a practical roadway. The experimental results show that the proposed method can generate an optimal bolt supporting scheme with a good balance between supporting quality and the other demands, besides speeding up its convergence.

Efficient Generalized Surrogate-Assisted Evolutionary Algorithm for High-Dimensional Expensive Problems

Abstract: Engineering optimization problems usually involve computationally expensive simulations and many design variables. Solving such problems in an efficient manner is still a major challenge. In this paper, a generalized surrogate-assisted evolutionary algorithm is proposed to solve such high-dimensional expensive problems. The proposed algorithm is based on the optimization framework of the genetic algorithm (GA). This algorithm proposes to use a surrogate-based trust region local search method, a surrogate-guided GA (SGA) updating mechanism with a neighbor region partition strategy and a prescreening strategy based on the expected improvement infilling criterion of a simplified Kriging in the optimization process. The SGA updating mechanism is a special characteristic of the proposed algorithm. This mechanism makes a fusion between surrogates and the evolutionary algorithm. The neighbor region partition strategy effectively retains the diversity of the population. Moreover, multiple surrogates used in the SGA updating mechanism make the proposed algorithm optimize robustly. The proposed algorithm is validated by testing several high-dimensional numerical benchmark problems with dimensions varying from 30 to 100, and an overall comparison is made between the proposed algorithm and other optimization algorithms. The results show that the proposed algorithm is very efficient and promising for optimizing high-dimensional expensive problems.

A Survey of Weight Vector Adjustment Methods for Decomposition-Based Multiobjective Evolutionary Algorithms

Abstract: Multiobjective evolutionary algorithms based on decomposition (MOEA/D) have attracted tremendous attention and achieved great success in the fields of optimization and decision-making. MOEA/Ds work by decomposing the target multiobjective optimization problem (MOP) into multiple single-objective subproblems based on a set of weight vectors. The subproblems are solved cooperatively in an evolutionary algorithm framework. Since weight vectors define the search directions and, to a certain extent, the distribution of the final solution set, the configuration of weight vectors is pivotal to the success of MOEA/Ds. The most straightforward method is to use predefined and uniformly distributed weight vectors. However, it usually leads to the deteriorated performance of MOEA/Ds on solving MOPs with irregular Pareto fronts. To deal with this issue, many weight vector adjustment methods have been proposed by periodically adjusting the weight vectors in a random, predefined, or adaptive way. This article focuses on weight vector adjustment on a simplex and presents a comprehensive survey of these weight vector adjustment methods covering the weight vector adaptation strategies, theoretical analyses, benchmark test problems, and applications. The current limitations, new challenges, and future directions of weight vector adjustment are also discussed.

Distributed Individuals for Multiple Peaks: A Novel Differential Evolution for Multimodal Optimization Problems

Abstract: Locating more peaks and refining the solution accuracy on the found peaks are two challenging issues in solving multimodal optimization problems (MMOPs). To deal with these two challenges, a distributed individuals differential evolution (DIDE) algorithm is proposed in this article based on a distributed individuals for multiple peaks (DIMP) framework and two novel mechanisms. First, the DIMP framework provides sufficient diversity by letting each individual act as a distributed unit to track a peak. Based on the DIMP framework, each individual uses a virtual population controlled by an adaptive range adjustment strategy to explore the search space sufficiently for locating a peak and then gradually approach it. Second, the two novel mechanisms named lifetime mechanism and elite learning mechanism (ELM) cooperate with the DIMP framework. The lifetime mechanism is inspired by the natural phenomenon that every organism will gradually age and has a limited lifespan. When an individual runs out of its lifetime and also has good fitness, it is regarded as an elite solution and will be added to an archive. Then the individual restarts a new lifetime, so as to bring further diversity to locate more peaks. The ELM is proposed to refine the accuracy of those elite solutions in the archive, being efficient in dealing with the solution accuracy issue on the found peaks. The experimental results on 20 multimodal benchmark test functions show that the proposed DIDE algorithm has generally better or competitive performance compared with the state-of-the-art multimodal optimization algorithms.

Evolutionary Dynamic Multiobjective Optimization Assisted by a Support Vector Regression Predictor

Abstract: Dynamic multiobjective optimization problems (DMOPs) challenge multiobjective evolutionary algorithms (MOEAs) because those problems change rapidly over time. The class of DMOPs whose objective functions change over time steps, in ways that exhibit some hidden patterns has gained much attention. Their predictability indicates that the problem exhibits some correlations between solutions obtained in sequential time periods. Most of the current approaches use linear models or similar strategies to describe the correlations between historical solutions obtained, and predict the new solutions in the following time period as an initial population from which the MOEA can begin searching in order to improve its efficiency. However, nonlinear correlations between historical solutions and current solutions are more common in practice, and a linear model may not be suitable for the nonlinear case. In this paper, we present a support vector regression (SVR)-based predictor to generate the initial population for the MOEA in the new environment. The basic idea of this predictor is to map the historical solutions into a high-dimensional feature space via a nonlinear mapping, and to do linear regression in this space. SVR is used to implement this process. We incorporate this predictor into the MOEA based on decomposition (MOEA/D) to construct a novel algorithm for solving the aforementioned class of DMOPs. Comprehensive experiments have shown the effectiveness and competitiveness of our proposed predictor, comparing with the state-of-the-art methods.

A Constrained Multiobjective Evolutionary Algorithm With Detect-and-Escape Strategy

Abstract: Overall constraint violation functions are commonly used in multiobjective evolutionary algorithms (MOEAs) for handling constraints. Constraints could cause these algorithms stuck in two stagnation states: 1) since the feasible region of a multiobjective optimization problem can consist of several disconnected feasible subregions, the search can be easily trapped in a feasible subregion which does not contain all the global Pareto optimal solutions and 2) an overall constraint violation function may have many nonzero minimal points, it can make the search stuck in an unfeasible area. To address these two issues, this article proposes a strategy to detect whether or not the search is stuck in these two stagnation states and then escape from them. Our proposed detect-and-escape strategy uses the feasible ratio and the change rate of overall constraint violation to detect stagnation, and adjusts the weight of the constraint violation for guiding the search to escape from stagnation states. We develop and implement a decomposition-based constrained MOEA with this strategy. Extensive experiments on a number of benchmark problems demonstrate the competitiveness of our proposed algorithm when compared to five other state-of-the-art constrained evolutionary algorithms.

An Estimation of Distribution Algorithm for Mixed-Variable Newsvendor Problems

Abstract: As one of the classical problems in the economic market, the newsvendor problem aims to make maximal profit by determining the optimal order quantity of products. However, the previous newsvendor models assume that the selling price of a product is a predefined constant and only regard the order quantity as a decision variable, which may result in an unreasonable investment decision. In this article, a new newsvendor model is first proposed, which involves of both order quantity and selling price as decision variables. In this way, the newsvendor problem is reformulated as a mixed-variable nonlinear programming problem, rather than an integer linear programming problem as in previous investigations. In order to solve the mixed-variable newsvendor problem, a histogram model-based estimation of distribution algorithm (EDA) called ${\mathrm{ EDA}}_{mvn}$ is developed, in which an adaptive-width histogram model is used to deal with the continuous variables and a learning-based histogram model is applied to deal with the discrete variables. The performance of ${\mathrm{ EDA}}_{mvn}$ was assessed on a test suite with eight representative instances generated by the orthogonal experiment design method and a real-world instance generated from real market data of Alibaba. The experimental results show that, ${\mathrm{ EDA}}_{mvn}$ outperforms not only the state-of-the-art mixed-variable evolutionary algorithms, but also a commercial software, i.e., Lingo.

Novel Prediction Strategies for Dynamic Multiobjective Optimization

Abstract: This paper proposes a new prediction-based dynamic multiobjective optimization (PBDMO) method, which combines a new prediction-based reaction mechanism and a popular regularity model-based multiobjective estimation of distribution algorithm (RM-MEDA) for solving dynamic multiobjective optimization problems. Whenever a change is detected, PBDMO reacts effectively to it by generating three subpopulations based on different strategies. The first subpopulation is created by moving nondominated individuals using a simple linear prediction model with different step sizes. The second subpopulation consists of some individuals generated by a novel sampling strategy to improve population convergence as well as distribution. The third subpopulation comprises some individuals generated using a shrinking strategy based on the probability distribution of variables. These subpopulations are tailored to form a population for the new environment. The experimental results carried out on a variety of bi- and three-objective benchmark functions demonstrate that the proposed technique has competitive performance compared with some state-of-the-art algorithms.

Multiple Reference Points-Based Decomposition for Multiobjective Feature Selection in Classification: Static and Dynamic Mechanisms

Abstract: Feature selection is an important task in machine learning that has two main objectives: 1) reducing dimensionality and 2) improving learning performance. Feature selection can be considered a multiobjective problem. However, it has its problematic characteristics, such as a highly discontinuous Pareto front, imbalance preferences, and partially conflicting objectives. These characteristics are not easy for existing evolutionary multiobjective optimization (EMO) algorithms. We propose a new decomposition approach with two mechanisms (static and dynamic) based on multiple reference points under the multiobjective evolutionary algorithm based on decomposition (MOEA/D) framework to address the above-mentioned difficulties of feature selection. The static mechanism alleviates the dependence of the decomposition on the Pareto front shape and the effect of the discontinuity. The dynamic one is able to detect regions in which the objectives are mostly conflicting, and allocates more computational resources to the detected regions. In comparison with other EMO algorithms on 12 different classification datasets, the proposed decomposition approach finds more diverse feature subsets with better performance in terms of hypervolume and inverted generational distance. The dynamic mechanism successfully identifies conflicting regions and further improves the approximation quality for the Pareto fronts.

A Many-Objective Evolutionary Algorithm With Pareto-Adaptive Reference Points

Abstract: We propose a new many-objective evolutionary algorithm with Pareto-adaptive reference points. In this algorithm, the shape of the Pareto-optimal front (PF) is estimated based on a ratio of Euclidean distances. If the estimated shape is likely to be convex, the nadir point is used as the reference point to calculate the convergence and diversity indicators for individuals. Otherwise, the reference point is set to the ideal point. In addition, the estimation of the nadir point is different from what was widely used in the literature. The nadir point, together with the ideal point, provides a feasible way to deal with dominance resistant solutions, which are difficult to be detected and eliminated in Pareto-based algorithms. The proposed algorithm is compared with the state-of-the-art many-objective optimization algorithms on a number of unconstrained and constrained test problems with up to 15 objectives. The experimental results show that it performs better than other algorithms in most of the test instances. Moreover, the new algorithm shows good performance on problems whose PFs are irregular (being discontinuous, degenerated, bent, or mixed). The observed high performance and inherent good properties (such as being free of weight vectors and control parameters) make the new proposal a promising tool for other similar problems.

Utilizing the Correlation Between Constraints and Objective Function for Constrained Evolutionary Optimization

Abstract: When solving constrained optimization problems by evolutionary algorithms, the core issue is to balance constraints and objective function. This paper is the first attempt to utilize the correlation between constraints and objective function to keep this balance. First of all, the correlation between constraints and objective function is mined and represented by a correlation index. Afterward, a weighted sum updating approach and an archiving and replacement mechanism are proposed to make use of this correlation index to guide the evolution. By the above process, a novel constrained optimization evolutionary algorithm is presented. Experiments on a broad range of benchmark test functions indicate that the proposed method shows better or at least competitive performance against other state-of-the-art methods. Moreover, the proposed method is applied to the gait optimization of humanoid robots.

Adapting Reference Vectors and Scalarizing Functions by Growing Neural Gas to Handle Irregular Pareto Fronts

Abstract: The performance of decomposition-based multiobjective evolutionary algorithms (MOEAs) often deteriorates clearly when solving multiobjective optimization problems with irregular Pareto fronts (PFs). The main reason is the improper settings of reference vectors and scalarizing functions. In this paper, we propose a decomposition-based MOEA guided by a growing neural gas network, which learns the topological structure of the PF. Both reference vectors and scalarizing functions are adapted based on the topological structure to enhance the evolutionary algorithm’s search ability. The proposed algorithm is compared with eight state-of-the-art optimizers on 34 test problems. The experimental results demonstrate that the proposed method is competitive in handling irregular PFs.

Multiobjective Multitasking Optimization Based on Incremental Learning

Abstract: Multiobjective multitasking optimization (MTO) is an emerging research topic in the field of evolutionary computation. In contrast to multiobjective optimization, MTO solves multiple optimization tasks simultaneously. MTO aims to improve the overall performance of multiple tasks through knowledge transfer among tasks. Recently, MTO has attracted the attention of many researchers, and several algorithms have been proposed in the literature. However, one of the crucial issues, finding useful knowledge, has been rarely studied. Keeping this in mind, this article proposes an MTO algorithm based on incremental learning (EMTIL). Specifically, the transferred solutions (the form of knowledge) will be selected by incremental classifiers, which are capable of finding valuable solutions for knowledge transfer. The training data are generated by the knowledge transfer at each generation. Furthermore, the search space of the tasks will be explored by the proposed mapping (among tasks) approach, which helps these tasks to escape from their local Pareto Fronts. Empirical studies have been conducted on 15 MTO problems to assess the effectiveness of EMTIL. The experimental results demonstrate that EMTIL works more effectively for MTO compared to the existing algorithms.

A New Hypervolume-Based Evolutionary Algorithm for Many-Objective Optimization

Abstract: In this article, a new hypervolume-based evolutionary multiobjective optimization algorithm (EMOA), namely, R2HCA-EMOA (R2-based hypervolume contribution approximation EMOA), is proposed for many-objective optimization. The core idea of the algorithm is to use an R2 indicator variant to approximate the hypervolume contribution. The basic framework of the proposed algorithm is the same as SMS-EMOA. In order to make the algorithm computationally efficient, a utility tensor structure is introduced for the calculation of the R2 indicator variant. Moreover, a normalization mechanism is incorporated into R2HCA-EMOA to enhance the performance of the algorithm. Through experimental studies, R2HCA-EMOA is compared with three hypervolume-based EMOAs and several other state-of-the-art EMOAs on 5-, 10-, and 15-objective DTLZ, WFG problems, and their minus versions. Our results show that R2HCA-EMOA is more efficient than the other hypervolume-based EMOAs, and is superior to all the compared state-of-the-art EMOAs.

Handling Imbalance Between Convergence and Diversity in the Decision Space in Evolutionary Multimodal Multiobjective Optimization

Abstract: There may exist more than one Pareto optimal solution with the same objective vector to a multimodal multiobjective optimization problem (MMOP). The difficulties in finding such solutions can be different. Although a number of evolutionary multimodal multiobjective algorithms (EMMAs) have been proposed, they are unable to solve such an MMOP due to their convergence-first selection criteria. They quickly converge to the Pareto optimal solutions which are easy to find and therefore lose diversity in the decision space. That is, such an MMOP features an imbalance between achieving convergence and preserving diversity in the decision space. In this article, we first present a set of imbalanced distance minimization benchmark problems. Then we propose an evolutionary algorithm using a convergence-penalized density method (CPDEA). In CPDEA, the distances among solutions in the decision space are transformed based on their local convergence quality. Their density values are estimated based on the transformed distances and used as the selection criterion. We compare CPDEA with five state-of-the-art EMMAs on the proposed benchmarks. Our experimental results show that CPDEA is clearly superior in solving these problems.

A Multimodel Prediction Method for Dynamic Multiobjective Evolutionary Optimization

Abstract: A large number of prediction strategies are specific to a dynamic multiobjective optimization problem (DMOP) with only one type of the Pareto set (PS) change. However, a continuous DMOP with more than one type of the unknown PS change has been seldom investigated. We present a multimodel prediction approach (MMP) realized in the framework of evolutionary algorithms (EAs) to tackle the problem. In this paper, we first detect the type of the PS change, followed by the selection of an appropriate prediction model to provide an initial population for the subsequent evolution. To observe the influence of MMP on EAs, optimal solutions obtained by three classical dynamic multiobjective EAs with and without MMP are investigated. Furthermore, to investigate the performance of MMP, three state-of-the-art prediction strategies are compared on a large number of dynamic test instances under the same particle swarm optimizer. The experimental results demonstrate that the proposed approach outperforms its counterparts under comparison on most optimization problems.

Does Preference Always Help? A Holistic Study on Preference-Based Evolutionary Multiobjective Optimization Using Reference Points

Abstract: The ultimate goal of multiobjective optimization is to help a decision maker (DM) identify solution(s) of interest (SOI) achieving satisfactory tradeoffs among multiple conflicting criteria. This can be realized by leveraging DM’s preference information in evolutionary multiobjective optimization (EMO). No consensus has been reached on the effectiveness brought by incorporating preference in EMO (either a priori or interactively ) versus a posteriori decision making after a complete run of an EMO algorithm. Bearing this consideration in mind, this article: 1) provides a pragmatic overview of the existing developments of preference-based EMO (PBEMO) and 2) conducts a series of experiments to investigate the effectiveness brought by preference incorporation in EMO for approximating various SOI. In particular, the DM’s preference information is elicited as a reference point, which represents her/his aspirations for different objectives. The experimental results demonstrate that preference incorporation in EMO does not always lead to a desirable approximation of SOI if the DM’s preference information is not well utilized, nor does the DM elicit invalid preference information, which is not uncommon when encountering a black-box system. To a certain extent, this issue can be remedied through an interactive preference elicitation. Last but not the least, we find that a PBEMO algorithm is able to be generalized to approximate the whole PF given an appropriate setup of preference information.

Offline Data-Driven Multiobjective Optimization: Knowledge Transfer Between Surrogates and Generation of Final Solutions

Abstract: In offline data-driven optimization, only historical data is available for optimization, making it impossible to validate the obtained solutions during the optimization. To address these difficulties, this paper proposes an evolutionary algorithm assisted by two surrogates, one coarse model and one fine model. The coarse surrogate (CS) aims to guide the algorithm to quickly find a promising subregion in the search space, whereas the fine one focuses on leveraging good solutions according to the knowledge transferred from the CS. Since the obtained Pareto optimal solutions have not been validated using the real fitness function, a technique for generating the final optimal solutions is suggested. All achieved solutions during the whole optimization process are grouped into a number of clusters according to a set of reference vectors. Then, the solutions in each cluster are averaged and outputted as the final solution of that cluster. The proposed algorithm is compared with its three variants and two state-of-the-art offline data-driven multiobjective algorithms on eight benchmark problems to demonstrate its effectiveness. Finally, the proposed algorithm is successfully applied to an operational indices optimization problem in beneficiation processes.

Scaling Up Dynamic Optimization Problems: A Divide-and-Conquer Approach

Abstract: Scalability is a crucial aspect of designing efficient algorithms. Despite their prevalence, large-scale dynamic optimization problems are not well studied in the literature. This paper is concerned with designing benchmarks and frameworks for the study of large-scale dynamic optimization problems. We start by a formal analysis of the moving peaks benchmark (MPB) and show its nonseparable nature irrespective of its number of peaks. We then propose a composite MPB suite with exploitable modularity covering a wide range of scalable partially separable functions suitable for the study of large-scale dynamic optimization problems. The benchmark exhibits modularity, heterogeneity, and imbalance features to resemble real-world problems. To deal with the intricacies of large-scale dynamic optimization problems, we propose a decomposition-based coevolutionary framework which breaks a large-scale dynamic optimization problem into a set of lower-dimensional components. A novel aspect of the framework is its efficient bi-level resource allocation mechanism which controls the budget assignment to components and the populations responsible for tracking multiple moving optima. Based on a comprehensive empirical study on a wide range of large-scale dynamic optimization problems with up to 200-D, we show the crucial role of problem decomposition and resource allocation in dealing with these problems. The experimental results clearly show the superiority of the proposed framework over three other approaches in solving large-scale dynamic optimization problems.

A Novel Evolutionary Algorithm for Dynamic Constrained Multiobjective Optimization Problems

Abstract: To promote research on dynamic constrained multiobjective optimization, we first propose a group of generic test problems with challenging characteristics, including different modes of the true Pareto front (e.g., convexity–concavity and connectedness–disconnectedness) and the changing feasible region. Subsequently, motivated by the challenges presented by dynamism and constraints, we design a dynamic constrained multiobjective optimization algorithm with a nondominated solution selection operator, a mating selection strategy, a population selection operator, a change detection method, and a change response strategy. The designed nondominated solution selection operator can obtain a nondominated population with diversity when the environment changes. The mating selection strategy and population selection operator can adaptively handle infeasible solutions. If a change is detected, the proposed change response strategy reuses some portion of the old solutions in combination with randomly generated solutions to reinitialize the population, and a steady-state update method is designed to improve the retained previous solutions. The experimental results show that the proposed test problems can be used to clearly distinguish the performance of algorithms, and that the proposed algorithm is very competitive for solving dynamic constrained multiobjective optimization problems in comparison with state-of-the-art algorithms.