Showing papers on "Population-based incremental learning published in 2017"

A novel collaborative optimization algorithm in solving complex optimization problems

Abstract: To overcome the deficiencies of weak local search ability in genetic algorithms (GA) and slow global convergence speed in ant colony optimization (ACO) algorithm in solving complex optimization problems, the chaotic optimization method, multi-population collaborative strategy and adaptive control parameters are introduced into the GA and ACO algorithm to propose a genetic and ant colony adaptive collaborative optimization (MGACACO) algorithm for solving complex optimization problems. The proposed MGACACO algorithm makes use of the exploration capability of GA and stochastic capability of ACO algorithm. In the proposed MGACACO algorithm, the multi-population strategy is used to realize the information exchange and cooperation among the various populations. The chaotic optimization method is used to overcome long search time, avoid falling into the local extremum and improve the search accuracy. The adaptive control parameters is used to make relatively uniform pheromone distribution, effectively solve the contradiction between expanding search and finding optimal solution. The collaborative strategy is used to dynamically balance the global ability and local search ability, and improve the convergence speed. Finally, various scale TSP are selected to verify the effectiveness of the proposed MGACACO algorithm. The experiment results show that the proposed MGACACO algorithm can avoid falling into the local extremum, and takes on better search precision and faster convergence speed.

LSHADE with semi-parameter adaptation hybrid with CMA-ES for solving CEC 2017 benchmark problems

Abstract: To improve the optimization performance of LSHADE algorithm, an alternative adaptation approach for the selection of control parameters is proposed. The proposed algorithm, named LSHADE-SPA, uses a new semi-parameter adaptation approach to effectively adapt the values of the scaling factor of the Differential evolution algorithm. The proposed approach consists of two different settings for two control parameters F and Cr. The benefit of this approach is to prove that the semi-adaptive algorithm is better than pure random algorithm or fully adaptive or self-adaptive algorithm. To enhance the performance of our algorithm, we also introduced a hybridization framework named LSHADE-SPACMA between LSHADE-SPA and a modified version of CMA-ES. The modified version of CMA-ES undergoes the crossover operation to improve the exploration capability of the proposed framework. In LSHADE-SPACMA both algorithms will work simultaneously on the same population, but more populations will be assigned gradually to the better performance algorithm. In order to verify and analyze the performance of both LSHADE-SPA and LSHADE-SPACMA, Numerical experiments on a set of 30 test problems from the CEC2017 benchmark for 10, 30, 50 and 100 dimensions, including a comparison with LSHADE algorithm are executed. Experimental results indicate that in terms of robustness, stability, and quality of the solution obtained, of both LSHADE-SPA and LSHADE-SPACMA are better than LSHADE algorithm, especially as the dimension increases.

Research on Traffic Flow Prediction in the Big Data Environment Based on the Improved RBF Neural Network

Abstract: This paper proposes an optimized prediction algorithm of radial basis function neural network based on an improved artificial bee colony (ABC) algorithm in the big data environment. The algorithm first uses crossover and mutation operators of the differential evolution algorithm to replace the search strategy of employed bees in the ABC algorithm, then improves the search strategy of onlookers in the ABC algorithm to produce an optimal candidate food source near the population. The algorithm can better balance local and global searching capability. To verify the efficiency of this algorithm in the big data environment, apply it to Lozi and Tent chaotic time series and measured traffic flow time series, and then compare it with K-nearest neighbor model, radial basis function (RBF) neural network model, improved back propagation neural network model, and RBF neural network based on a cloud genetic algorithm model. The experimental results indicate that the accuracy of prediction for Lozi and Tent chaotic time series and the measured traffic flow improves greatly in the big data environment using the proposed algorithm, which proves the effectiveness of the proposed algorithm in predicting traffic flow time series.

A self-adaptive multi-population based Jaya algorithm for engineering optimization

Abstract: Multi-population algorithms have been widely used for solving the real-world problems. However, it is not easy to get the number of sub-populations to be used for a given problem. This work proposes a self-adaptive multi-population based Jaya (SAMP-Jaya) algorithm for solving the constrained and unconstrained numerical and engineering optimization problems. The Jaya algorithm is a recently proposed advanced optimization algorithm and is not having any algorithmic-specific parameters to be tuned except the common control parameters of population size and the number of iterations. The search mechanism of the Jaya algorithm is upgraded in this paper by using the multi-population search scheme. It uses an adaptive scheme for dividing the population into sub-populations which control the exploration and exploitation rates of the search process based on the problem landscape. The robustness of the proposed SAMP-Jaya algorithm is tested on 15 CEC 2015 unconstrained benchmark problems in addition to 15 unconstrained and 10 constrained standard benchmark problems taken from the literature. The Friedman rank test is conducted in order to compare the performance of the algorithms. It has obtained first rank among six algorithms for 15 CEC 2015 unconstrained problems with the average scores of 1.4 and 1.9 for 10-dimension and 30-dimension problems respectively. Also, the proposed algorithm has obtained first rank for 15 unimodal and multimodal unconstrained benchmark problems with the average scores of 1.7667 and 2.2667 with 50000 and 200000 function evaluations respectively. The performance of the proposed algorithm is further compared with the other latest algorithms such as across neighborhood search (ANS) optimization algorithm, multi-population ensemble of mutation differential evolution (MEMDE), social learning particle swarm optimization algorithm (SL-PSO), competitive swarm optimizer (CSO) and it is found that the performance of the proposed algorithm is better in more than 65% cases. Furthermore, the proposed algorithm is used for solving a case study of the entropy generation minimization of a plate-fin heat exchanger (PFHE). It is found that the number of entropy generation units is reduced by 12.73%, 3.5% and 9.6% using the proposed algorithm as compared to the designs given by genetic algorithm (GA), particle swarm optimization (PSO) and cuckoo search algorithm (CSA) respectively. Thus the computational experiments have proved the effectiveness of the proposed algorithm for solving engineering optimization problems.

Learning to Optimize Neural Nets

Abstract: Learning to Optimize is a recently proposed framework for learning optimization algorithms using reinforcement learning. In this paper, we explore learning an optimization algorithm for training shallow neural nets. Such high-dimensional stochastic optimization problems present interesting challenges for existing reinforcement learning algorithms. We develop an extension that is suited to learning optimization algorithms in this setting and demonstrate that the learned optimization algorithm consistently outperforms other known optimization algorithms even on unseen tasks and is robust to changes in stochasticity of gradients and the neural net architecture. More specifically, we show that an optimization algorithm trained with the proposed method on the problem of training a neural net on MNIST generalizes to the problems of training neural nets on the Toronto Faces Dataset, CIFAR-10 and CIFAR-100.

A Hybrid grey wolf optimizer and genetic algorithm for minimizing potential energy function

Abstract: In this paper, we propose a new hybrid algorithm between the grey wolf optimizer algorithm and the genetic algorithm in order to minimize a simplified model of the energy function of the molecule. We call the proposed algorithm by Hybrid Grey Wolf Optimizer and Genetic Algorithm (HGWOGA). We employ three procedures in the HGWOGA. In the first procedure, we apply the grey wolf optimizer algorithm to balance between the exploration and the exploitation process in the proposed algorithm. In the second procedure, we utilize the dimensionality reduction and the population partitioning processes by dividing the population into sub-populations and using the arithmetical crossover operator in each sub-population in order to increase the diversity of the search in the algorithm. In the last procedure, we apply the genetic mutation operator in the whole population in order to refrain from the premature convergence and trapping in local minima. We implement the proposed algorithm with various molecule size with up to 200 dimensions and compare the proposed algorithm with 8 benchmark algorithms in order to validate its efficiency for solving molecular potential energy function. The numerical experiment results show that the proposed algorithm is a promising, competent, and capable of finding the global minimum or near global minimum of the molecular energy function faster than the other comparative algorithms.

Partitioned-cooperative quantum-behaved particle swarm optimization based on multilevel thresholding applied to medical image segmentation

Abstract: Display Omitted We introduce a novel quantum-behaved particle swarm optimization (SCQPSO) algorithm- SCQPSO.In SCQPSO, the auxiliary swarms and partitioned search space are introduced to increase the population diversity.In SCQPSO, the cooperative theory is introduced into QPSO algorithm to change the updating mode of the particles.SCQPSO is tested on five benchmark test functions and five shift complex functions.We use SCQPSO to optimize the parameters for OTSU image segmentation of four stomach CT images. In this paper, in order to search the global optimum solution with a very fast convergence speed across the whole search space, we propose a partitioned and cooperative quantum-behaved particle swarm optimization (SCQPSO) algorithm. The auxiliary swarms and partitioned search space are introduced to increase the population diversity. The cooperative theory is introduced into QPSO algorithm to change the updating mode of the particles in order to guarantee that this algorithm well balances the effectiveness and simplification. Firstly, we explain how this method leads to enhanced population diversity and improved algorithm over previous strategies, and emphasize this algorithm with comparative experiments using five benchmark test functions and five shift complex functions. After that we demonstrate a reasonable application of the proposed algorithm, by showing how it can be used to optimize the parameters for OTSU image segmentation for processing medical images. The results show that the proposed SCQPSO algorithm outperforms than the other improved QPSO in terms of the quality of the solution, and performs better for solving the image segmentation than the QPSO algorithm, the sunCQPSO algorithm, the CCQPSO algorithm.

Optimizing dam and reservoirs operation based model utilizing shark algorithm approach

Abstract: Computational intelligence (CI) is a fast evolving area in which many novel algorithms, stemmed from various inspiring sources, were developed during the past decade. Nevertheless, many of them are dispersed in different research directions, and their true potential is thus not fully utilized yet. Therefore, there is a need to investigate the potential of these methods in different engineering optimization problems. In fact, shark algorithm is a stochastic search optimization algorithm which is started first in a set of random generated potential solutions, and then performs the search for the optimum one interactively. Such procedure is appropriate to the system features of the reservoir system as it is a stochastic system in nature. In this article, investigation of the potential of shark algorithm is examined as an optimization algorithm for reservoir operation. To achieve that real single reservoir and multi-reservoir optimal operations have been performed utilizing shark algorithm. Many performances indexes have been measured for each case study utilizing the proposed shark algorithm and another existing optimization algorithms namely, Genetic Algorithm (GA) and Particle Swarm Optimization (PSO). The results showed that the proposed shark algorithm outperformed the other algorithms and achieved higher reliability index and lesser vulnerability index. Moreover, standard deviation and coefficient of variation in Shark Algorithm were less than the other two algorithms, which indicates its superiority.

Hybrid real-code population-based incremental learning and differential evolution for many-objective optimisation of an automotive floor-frame

Abstract: In this paper, a many-objective hybrid real-code population-based incremental learning and differential evolution algorithm (MnRPBILDE) is proposed based on the concept of objective function space reduction. The method is then implemented on real engineering design problems. The topology, shape and sizing design of a simplified automotive floor-frame structure are formulated and used as test problems. A variety of well-established multi-objective evolutionary algorithms (MOEAs) including the original version of MnRPBILDE are employed to solve the test problems while the results are compared based on hypervolume and C indicators. The results indicate that our proposed algorithm outperforms the other MOEAs. The proposed algorithm is effective and efficient for many-objective optimisations of a car floor-frame structure.

A Many-Objective Evolutionary Algorithm with Angle-Based Selection and Shift-Based Density Estimation

Abstract: Evolutionary many-objective optimization has been gaining increasing attention from the evolutionary computation research community. Much effort has been devoted to addressing this issue by improving the scalability of multiobjective evolutionary algorithms, such as Pareto-based, decomposition-based, and indicator-based approaches. Different from current work, we propose a novel algorithm in this paper called AnD, which consists of an angle-based selection strategy and a shift-based density estimation strategy. These two strategies are employed in the environmental selection to delete the poor individuals one by one. Specifically, the former is devised to find a pair of individuals with the minimum vector angle, which means that these two individuals share the most similar search direction. The latter, which takes both the diversity and convergence into account, is adopted to compare these two individuals and to delete the worse one. AnD has a simple structure, few parameters, and no complicated operators. The performance of AnD is compared with that of seven state-of-the-art many-objective evolutionary algorithms on a variety of benchmark test problems with up to 15 objectives. The experimental results suggest that AnD can achieve highly competitive performance. In addition, we also verify that AnD can be readily extended to solve constrained many-objective optimization problems.

A parallel algorithm for Bayesian network structure learning from large data sets

Abstract: This paper considers a parallel algorithm for Bayesian network structure learning from large data sets. The parallel algorithm is a variant of the well known PC algorithm. The PC algorithm is a constraint-based algorithm consisting of five steps where the first step is to perform a set of (conditional) independence tests while the remaining four steps relate to identifying the structure of the Bayesian network using the results of the (conditional) independence tests. In this paper, we describe a new approach to parallelization of the (conditional) independence testing as experiments illustrate that this is by far the most time consuming step. The proposed parallel PC algorithm is evaluated on data sets generated at random from five different real-world Bayesian networks. The algorithm is also compared empirically with a process-based approach where each process manages a subset of the data over all the variables on the Bayesian network. The results demonstrate that significant time performance improvements are possible using both approaches.

Unsupervised extreme learning machine with representational features

Abstract: Extreme learning machine (ELM) is not only an effective classifier but also a useful cluster. Unsupervised extreme learning machine (US-ELM) gives favorable performance compared to state-of-the-art clustering algorithms. Extreme learning machine as an auto encoder (ELM-AE) can obtain principal components which represent original samples. The proposed unsupervised extreme learning machine based on embedded features of ELM-AE (US-EF-ELM) algorithm applies ELM-AE to US-ELM. US-EF-ELM regards embedded features of ELM-AE as the outputs of US-ELM hidden layer, and uses US-ELM to obtain the embedded matrix of US-ELM. US-EF-ELM can handle the multi-cluster clustering. The learning capability and computational efficiency of US-EF-ELM are as same as US-ELM. By experiments on UCI data sets, we compared US-EF-ELM k-means algorithm with k-means algorithm, spectral clustering algorithm, and US-ELM k-means algorithm in accuracy and efficiency.

Maximizing influence in a social network: Improved results using a genetic algorithm

Abstract: The influence maximization problem focuses on finding a small subset of nodes in a social network that maximizes the spread of influence. While the greedy algorithm and some improvements to it have been applied to solve this problem, the long solution time remains a problem. Stochastic optimization algorithms, such as simulated annealing, are other choices for solving this problem, but they often become trapped in local optima. We propose a genetic algorithm to solve the influence maximization problem. Through multi-population competition, using this algorithm we achieve an optimal result while maintaining diversity of the solution. We tested our method with actual networks, and our genetic algorithm performed slightly worse than the greedy algorithm but better than other algorithms.

Modified genetic algorithm for simple straight and U-shaped assembly line balancing with fuzzy processing times

Abstract: This paper aims at the straight and U-shaped assembly line balancing. Due to the uncertainty, variability and imprecision in actual production systems, the processing time of tasks are presented in triangular fuzzy numbers. In this case, it is intended to optimize the efficiency and idleness percentage of the assembly line as well as and concurrently with minimizing the number of workstations. To solve the problem, a modified genetic algorithm is proposed. One-fifth success rule in selection operator to improve the genetic algorithm performance. This leads genetic algorithm being controlled in convergence and diversity simultaneously by the means of controlling the selective pressure. Also a fuzzy controller in selective pressure employed for one-fifth success rule better implementation in genetic algorithm. In addition, Taguchi design of experiments used for parameter control and calibration. Finally, numerical examples are presented to compare the performance of proposed method with existing ones. Results show the high performance of the proposed algorithm.

An improved teaching-learning-based optimization algorithm and its application to a combinatorial optimization problem in foundry industry

Abstract: Display Omitted We propose a novel improved teaching-learning-based optimization algorithm with the concept of historical population.Two new operators are designed in the proposed algorithm to achieve the balance of exploration and exploitation ability.24 benchmark functions are tested with other algorithms to verify the good exploration and exploitation ability of proposed algorithm.The proposed algorithm is applied to address a combinatorial optimization problem in foundry industry with the design of coding and decoding mechanism. Teaching-learning-based optimization (TLBO) algorithm is a novel nature-inspired algorithm that mimics the teaching and learning process. In this paper, an improved version of TLBO algorithm (I-TLBO) is investigated to enhance the performance of original TLBO by achieving a balance between exploitation and exploration ability. Inspired by the concept of historical population, two new phases, namely self-feedback learning phase as well as mutation and crossover phase, are introduced in I-TLBO algorithm. In self-feedback learning phase, a learner can improve his result based on the historical experience if his present state is better than the historical state. In mutation and crossover phase, the learners update their positions with probability based on the new population obtained by the crossover and mutation operations between present population and historical population. The design of self-feedback learning phase seeks the maintaining of good exploitation ability while the introduction of the mutation and crossover phase aims at the improvement of exploration ability in original TLBO. The effectiveness of proposed I-TLBO algorithm is tested on some benchmark functions and a combinatorial optimization problem of heat treating in foundry industry. The comparative results with some other improved TLBO algorithms and classic algorithms show that I-TLBO algorithm has significant advantages due to the balance between exploitation and exploration ability.

A Nonhomogeneous Cuckoo Search Algorithm Based on Quantum Mechanism for Real Parameter Optimization

Abstract: Cuckoo search (CS) algorithm is a nature-inspired search algorithm, in which all the individuals have identical search behaviors. However, this simple homogeneous search behavior is not always optimal to find the potential solution to a special problem, and it may trap the individuals into local regions leading to premature convergence. To overcome the drawback, this paper presents a new variant of CS algorithm with nonhomogeneous search strategies based on quantum mechanism to enhance search ability of the classical CS algorithm. Featured contributions in this paper include: 1) quantum-based strategy is developed for nonhomogeneous update laws and 2) we, for the first time, present a set of theoretical analyses on CS algorithm as well as the proposed algorithm, respectively, and conclude a set of parameter boundaries guaranteeing the convergence of the CS algorithm and the proposed algorithm. On 24 benchmark functions, we compare our method with five existing CS-based methods and other ten state-of-the-art algorithms. The numerical results demonstrate that the proposed algorithm is significantly better than the original CS algorithm and the rest of compared methods according to two nonparametric tests.

It's not the algorithm, it's the data

Abstract: In risk assessment and predictive policing, biased data can yield biased results.

Learning Graphical Models Using Multiplicative Weights

Abstract: We give a simple, multiplicative-weight update algorithm for learning undirected graphical models or Markov random fields (MRFs). The approach is new, and for the well-studied case of Ising models or Boltzmann machines, we obtain an algorithm that uses a nearly optimal number of samples and has quadratic running time (up to logarithmic factors), subsuming and improving on all prior work. Additionally, we give the first efficient algorithm for learning Ising models over general alphabets. Our main application is an algorithm for learning the structure of t-wise MRFs with nearly-optimal sample complexity (up to polynomial losses in necessary terms that depend on the weights) and running time that is $n^{O(t)}$. In addition, given $n^{O(t)}$ samples, we can also learn the parameters of the model and generate a hypothesis that is close in statistical distance to the true MRF. All prior work runs in time $n^{\Omega(d)}$ for graphs of bounded degree d and does not generate a hypothesis close in statistical distance even for t=3. We observe that our runtime has the correct dependence on n and t assuming the hardness of learning sparse parities with noise. Our algorithm--the Sparsitron-- is easy to implement (has only one parameter) and holds in the on-line setting. Its analysis applies a regret bound from Freund and Schapire's classic Hedge algorithm. It also gives the first solution to the problem of learning sparse Generalized Linear Models (GLMs).

A PAC Approach to Application-Specific Algorithm Selection

Abstract: The best algorithm for a computational problem generally depends on the “relevant inputs,” a concept that depends on the application domain and often defies formal articulation. While there is a large body of literature on empirical approaches to selecting the best algorithm for a given application domain, there has been surprisingly little theoretical analysis of the problem. This paper adapts concepts from statistical and online learning theory to reason about application-specific algorithm selection. Our models capture several state-of-the-art empirical and theoretical approaches to the problem, ranging from self-improving algorithms to empirical performance models, and our results identify conditions under which these approaches are guaranteed to perform well. We present one framework that models algorithm selection as a statistical learning problem, and our work here shows that dimension notions from statistical learning theory, historically used to measure the complexity of classes of binary- and re...