There is, I think, something ethereal about i —the square root of minus one. I remember first hearing about it at school. It seemed an odd beast at that time—an intruder hovering on the edge of reality.

Usually familiarity dulls this sense of the bizarre, but in the case of i it was the reverse: over the years the sense of its surreal nature intensified. It seemed that it was impossible to write mathematics that described the real world in …

I and i

https://www2.econ.iastate.edu/tesfatsi/DeepLearningInNeuralNetworksOverview.JSchmidhuber2015.pdf

Deep learning in neural networks

Differential evolution (DE) is arguably one of the most powerful stochastic real-parameter optimization algorithms in current use. DE operates through similar computational steps as employed by a standard evolutionary algorithm (EA). However, unlike traditional EAs, the DE-variants perturb the current-generation population members with the scaled differences of randomly selected and distinct population members. Therefore, no separate probability distribution has to be used for generating the offspring. Since its inception in 1995, DE has drawn the attention of many researchers all over the world resulting in a lot of variants of the basic algorithm with improved performance. This paper presents a detailed review of the basic concepts of DE and a survey of its major variants, its application to multiobjective, constrained, large scale, and uncertain optimization problems, and the theoretical studies conducted on DE so far. Also, it provides an overview of the significant engineering applications that have benefited from the powerful nature of DE.

/pdf/differential-evolution-a-survey-of-the-state-of-the-art-3lz840pz2t.pdf

Differential Evolution: A Survey of the State-of-the-Art

We empirically study the performance of the particle swarm optimizer (PSO). Four different benchmark functions with asymmetric initial range settings are selected as testing functions. The experimental results illustrate the advantages and disadvantages of the PSO. Under all the testing cases, the PSO always converges very quickly towards the optimal positions but may slow its convergence speed when it is near a minimum. Nevertheless, the experimental results show that the PSO is a promising optimization method and a new approach is suggested to improve PSO's performance near the optima, such as using an adaptive inertia weight.

Empirical study of particle swarm optimization

This paper puts forward two useful methods for self-adaptation of the mutation distribution - the concepts of derandomization and cumulation. Principle shortcomings of the concept of mutative strategy parameter control and two levels of derandomization are reviewed. Basic demands on the self-adaptation of arbitrary (normal) mutation distributions are developed. Applying arbitrary, normal mutation distributions is equivalent to applying a general, linear problem encoding. The underlying objective of mutative strategy parameter control is roughly to favor previously selected mutation steps in the future. If this objective is pursued rigorously, a completely derandomized self-adaptation scheme results, which adapts arbitrary normal mutation distributions. This scheme, called covariance matrix adaptation (CMA), meets the previously stated demands. It can still be considerably improved by cumulation - utilizing an evolution path rather than single search steps. Simulations on various test functions reveal local and global search properties of the evolution strategy with and without covariance matrix adaptation. Their performances are comparable only on perfectly scaled functions. On badly scaled, non-separable functions usually a speed up factor of several orders of magnitude is observed. On moderately mis-scaled functions a speed up factor of three to ten can be expected.

/pdf/completely-derandomized-self-adaptation-in-evolution-4berz8v4fr.pdf

Completely Derandomized Self-Adaptation in Evolution Strategies

This article gives a comprehensive introduction into one of the main branches of evolutionary computation – the evolution strategies (ES) the history of which dates back to the 1960s in Germany Starting from a survey of history the philosophical background is explained in order to make understandable why ES are realized in the way they are Basic ES algorithms and design principles for variation and selection operators as well as theoretical issues are presented, and future branches of ES research are discussed

/pdf/evolution-strategies-a-comprehensive-introduction-ciqnmqxb8a.pdf

Evolution strategies –A comprehensive introduction

https://homepages.fhv.at/hgb/New-Papers/CMAME07_BS07b.pdf

Robust Optimization - A Comprehensive Survey

Charged particle swarm optimization (CPSO) is well suited to the dynamic search problem since inter-particle repulsion maintains population diversity and good tracking can be achieved with a simple algorithm. This work extends the application of CPSO to the dynamic problem by considering a bi-modal parabolic environment of high spatial and temporal severity. Two types of charged swarms and an adapted neutral swarm are compared for a number of different dynamic environments which include extreme ‘needle-inthe-haystack’ cases. The results suggest that charged swarms perform best in the extreme cases, but neutral swarms are better optimizers in milder environments.

Genetic and Evolutionary Computation -- GECCO-2003

1. Introduction.- 2. Concepts for the Analysis of the ES.- 3. The Progress Rate of the (1 % MathType!MTEF!2!1!+- % feaagCart1ev2aaatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbuLwBLn % hiov2DGi1BTfMBaeXatLxBI9gBaerbd9wDYLwzYbItLDharqqtubsr % 4rNCHbGeaGqiVu0Je9sqqrpepC0xbbL8F4rqqrFfpeea0xe9Lq-Jc9 % vqaqpepm0xbba9pwe9Q8fs0-yqaqpepae9pg0FirpepeKkFr0xfr-x % fr-xb9adbaqaaeGaciGaaiaabeqaamaabaabaaGcbaaeaaaaaaaaa8 % qadaqadaWdaeaapeGaaGyma8aadaWfGaqaa8qacaGGSaaal8aabeqa % a8qacqGHRaWkaaGccqaH7oaBaiaawIcacaGLPaaaaaa!3C4E! $$\left( {1\mathop ,\limits^ + \lambda } \right)$$ ?)-ES on the Sphere Model.- 5. The Analysis of the (?, ?)-ES.- 6. The (?/?, ?) Strategies - or Why "Sex" May be Good.- 7. The (1, ?)-?-Self-Adaptation.- Appendices.- A. Integrals.- A.1 Definite Integrals of the Normal Distribution.- A.2 Indefinite Integrals of the Normal Distribution.- A.3 Some Integral Identities.- B. Approximations.- B.1 Frequently Used Taylor Expansions.- B.3 Cumulants, Moments, and Approximations.- B.3.1 Fundamental Relations.- B.3.2 The Weight Coefficients for the Density Approximation of a Standardized Random Variable.- B.4 Approximation of the Quantile Function.- C. The Normal Distribution.- C.3 Product Moments of Correlated Gaussian Mutations.- C.3.1 Fundamental Relations.- C.3.2 Derivation of the Product Moments.- D. (1, ?)-Progress Coefficients.- D.2 Table of Progress Coefficients of the (1, ?)-ES.- References.

The Theory of Evolution Strategies

Self-adaptation is an essential feature of natural evolution. However, in the context of function optimization, self-adaptation features of evolutionary search algorithms have been explored mainly with evolution strategy (ES) and evolutionary programming (EP). In this paper, we demonstrate the self-adaptive feature of real-parameter genetic algorithms (GAs) using a simulated binary crossover (SBX) operator and without any mutation operator. The connection between the working of self-adaptive ESs and real-parameter GAs with the SBX operator is also discussed. Thereafter, the self-adaptive behavior of real-parameter GAs is demonstrated on a number of test problems commonly used in the ES literature. The remarkable similarity in the working principle of real-parameter GAs and self-adaptive ESs shown in this study suggests the need for emphasizing further studies on self-adaptive GAs.

http://repository.ias.ac.in/9415/1/318.pdf

Hans-Georg Beyer

Papers

Evolution strategies –A comprehensive introduction

Robust Optimization - A Comprehensive Survey

Genetic and Evolutionary Computation -- GECCO-2003

The Theory of Evolution Strategies

Self-Adaptive Genetic Algorithms with Simulated Binary Crossover