Probability Distributions.- Linear Models for Regression.- Linear Models for Classification.- Neural Networks.- Kernel Methods.- Sparse Kernel Machines.- Graphical Models.- Mixture Models and EM.- Approximate Inference.- Sampling Methods.- Continuous Latent Variables.- Sequential Data.- Combining Models.

Pattern Recognition and Machine Learning

Evolutionary algorithms (EAs) are often well-suited for optimization problems involving several, often conflicting objectives. Since 1985, various evolutionary approaches to multiobjective optimization have been developed that are capable of searching for multiple solutions concurrently in a single run. However, the few comparative studies of different methods presented up to now remain mostly qualitative and are often restricted to a few approaches. In this paper, four multiobjective EAs are compared quantitatively where an extended 0/1 knapsack problem is taken as a basis. Furthermore, we introduce a new evolutionary approach to multicriteria optimization, the strength Pareto EA (SPEA), that combines several features of previous multiobjective EAs in a unique manner. It is characterized by (a) storing nondominated solutions externally in a second, continuously updated population, (b) evaluating an individual's fitness dependent on the number of external nondominated points that dominate it, (c) preserving population diversity using the Pareto dominance relationship, and (d) incorporating a clustering procedure in order to reduce the nondominated set without destroying its characteristics. The proof-of-principle results obtained on two artificial problems as well as a larger problem, the synthesis of a digital hardware-software multiprocessor system, suggest that SPEA can be very effective in sampling from along the entire Pareto-optimal front and distributing the generated solutions over the tradeoff surface. Moreover, SPEA clearly outperforms the other four multiobjective EAs on the 0/1 knapsack problem.

Multiobjective evolutionary algorithms: a comparative case study and the strength Pareto approach

List of Figures. List of Tables. Preface. Foreword. 1. Basic Concepts. 2. Evolutionary Algorithm MOP Approaches. 3. MOEA Test Suites. 4. MOEA Testing and Analysis. 5. MOEA Theory and Issues. 3. MOEA Theoretical Issues. 6. Applications. 7. MOEA Parallelization. 8. Multi-Criteria Decision Making. 9. Special Topics. 10. Epilog. Appendix A: MOEA Classification and Technique Analysis. Appendix B: MOPs in the Literature. Appendix C: Ptrue & PFtrue for Selected Numeric MOPs. Appendix D: Ptrue & PFtrue for Side-Constrained MOPs. Appendix E: MOEA Software Availability. Appendix F: MOEA-Related Information. Index. References.

/pdf/evolutionary-algorithms-for-solving-multi-objective-problems-2txxk4mqci.pdf

Evolutionary algorithms for solving multi-objective problems

In this paper, we provide a systematic comparison of various evolutionary approaches to multiobjective optimization using six carefully chosen test functions. Each test function involves a particular feature that is known to cause difficulty in the evolutionary optimization process, mainly in converging to the Pareto-optimal front (e.g., multimodality and deception). By investigating these different problem features separately, it is possible to predict the kind of problems to which a certain technique is or is not well suited. However, in contrast to what was suspected beforehand, the experimental results indicate a hierarchy of the algorithms under consideration. Furthermore, the emerging effects are evidence that the suggested test functions provide sufficient complexity to compare multiobjective optimizers. Finally, elitism is shown to be an important factor for improving evolutionary multiobjective search.

/pdf/comparison-of-multiobjective-evolutionary-algorithms-2a3t2yk6ku.pdf

Comparison of Multiobjective Evolutionary Algorithms: Empirical Results

This paper presents an approach in which Pareto dominance is incorporated into particle swarm optimization (PSO) in order to allow this heuristic to handle problems with several objective functions. Unlike other current proposals to extend PSO to solve multiobjective optimization problems, our algorithm uses a secondary (i.e., external) repository of particles that is later used by other particles to guide their own flight. We also incorporate a special mutation operator that enriches the exploratory capabilities of our algorithm. The proposed approach is validated using several test functions and metrics taken from the standard literature on evolutionary multiobjective optimization. Results indicate that the approach is highly competitive and that can be considered a viable alternative to solve multiobjective optimization problems.

Handling multiple objectives with particle swarm optimization

Many multi-objective evolutionary algorithms (MOEAs) have been proposed over the years. The main part of the most successful algorithms such as PESA, or NSGA-II, are the Pareto based selection strategy that decide survivors using dominance among individuals. However, does the Pareto based selection strategy always succeed in finding the Pareto optimal solutions? This paper shows a very simple example that can cause serious trouble for the Pareto based MOEAs. In such an instance, various solutions, which are apart from the true Pareto-optimums, are left as hardly-dominated solutions. We define such solutions as dominance resistant solutions (DRSs), and show a class of problems which produces DRSs easily. To cope with this difficulty we propose the /spl alpha/-domination strategy that relaxes the domination introducing a weak trade-off among objectives. With the /spl alpha/-domination strategy, the DRSs are effectively purged from the population.

Failure of Pareto-based MOEAs: does non-dominated really mean near to optimal?

The paper reviews several genetic algorithm (GA) approaches to multi objective optimization problems (MOPs). The keynote point of GAs to MOPs is designing efficient selection/reproduction operators so that a variety of Pareto optimal solutions are generated. From this viewpoint, the paper reviews several devices proposed for multi objective optimization by GAs such as the parallel selection method, the Pareto based ranking, and the fitness sharing. Characteristics of these approaches have been confirmed through computational experiments with a simple example. Moreover, two practical applications of the GA approaches to MOPs are introduced briefly.

Multi-objective optimization by genetic algorithms: a review

Recently, multi-objective optimization by use of the genetic algorithms (GAs) is getting a growing interest as a novel approach to this problem. Population based search of GA is expected to find the Pareto optimal solutions of the multi-objective optimization problems in parallel. To achieve this goal, it is an intrinsic requirement that the evolution process of GA maintains well the diversity of the population in the Pareto optimality set. In this paper, the authors propose to utilize the Thermodynamical Genetic Algorithm (TDGA), a genetic algorithm that uses the concepts of the entropy and the temperature in the selection operation, for multi-objective optimization. Combined with the Pareto-based ranking technique, the computer simulation shows that TDGA can find a variety of Pareto optimal solutions.

Multi-Objective Optimization by Means of the Thermodynamical Genetic Algorithm

The unimodal normal distribution crossover (UNDX) for the real-coded genetic algorithms (RCGA) proposed by Ono et al. (1997, 1998) shows an excellent performance in optimization problems of multi-modal and highly epistatic fitness functions in continuous search space. Further, theoretical analysis of the UNDX shows that the UNDX is a crossover operator that preserves the statistics such as the mean vector and the covariance matrix of the population well. The present paper proposes some design guidelines for crossover operators for RCGA. Then, based on these guidelines, a multi-parental extension of the UNDX is proposed so as to enhance its exploration ability. Performance of the extended UNDX is evaluated by numerical experiments.

Multi-parental extension of the unimodal normal distribution crossover for real-coded genetic algorithms

In applications of the genetic algorithms (GA) to problems of adaptation to changing environments, maintenance of the diversity of the population is an essential requirement. Taking this point into consideration, the authors have proposed to utilize the thermodynamical genetic algorithm (TDGA) for the problems of adaptation to changing environments. The TDGA is a genetic algorithm that uses a selection rule inspired by the principle of the minimal free energy in thermodynamical systems. In the present paper, the authors propose a control method of the temperature, an adjustable parameter in the TDGA. The temperature is controlled by a feedback technique so as to regulate the level of the diversity of the population measured by entropy. The adaptation ability of the proposed method is confirmed by computer simulation taking time-varying knapsack problems as examples.

Hajime Kita

Papers

Failure of Pareto-based MOEAs: does non-dominated really mean near to optimal?

Multi-objective optimization by genetic algorithms: a review

Multi-Objective Optimization by Means of the Thermodynamical Genetic Algorithm

Multi-parental extension of the unimodal normal distribution crossover for real-coded genetic algorithms

Adaption to a Changing Environment by Means of the Thermodynamical Genetic Algorithm