Showing papers on "Particle swarm optimization published in 2004"
TL;DR: An approach in which Pareto dominance is incorporated into particle swarm optimization (PSO) in order to allow this heuristic to handle problems with several objective functions and indicates that the approach is highly competitive and that can be considered a viable alternative to solve multiobjective optimization problems.
Abstract: This paper presents an approach in which Pareto dominance is incorporated into particle swarm optimization (PSO) in order to allow this heuristic to handle problems with several objective functions. Unlike other current proposals to extend PSO to solve multiobjective optimization problems, our algorithm uses a secondary (i.e., external) repository of particles that is later used by other particles to guide their own flight. We also incorporate a special mutation operator that enriches the exploratory capabilities of our algorithm. The proposed approach is validated using several test functions and metrics taken from the standard literature on evolutionary multiobjective optimization. Results indicate that the approach is highly competitive and that can be considered a viable alternative to solve multiobjective optimization problems.
3,474 citations
TL;DR: A novel parameter automation strategy for the particle swarm algorithm and two further extensions to improve its performance after a predefined number of generations to overcome the difficulties of selecting an appropriate mutation step size for different problems.
Abstract: This paper introduces a novel parameter automation strategy for the particle swarm algorithm and two further extensions to improve its performance after a predefined number of generations. Initially, to efficiently control the local search and convergence to the global optimum solution, time-varying acceleration coefficients (TVAC) are introduced in addition to the time-varying inertia weight factor in particle swarm optimization (PSO). From the basis of TVAC, two new strategies are discussed to improve the performance of the PSO. First, the concept of "mutation" is introduced to the particle swarm optimization along with TVAC (MPSO-TVAC), by adding a small perturbation to a randomly selected modulus of the velocity vector of a random particle by predefined probability. Second, we introduce a novel particle swarm concept "self-organizing hierarchical particle swarm optimizer with TVAC (HPSO-TVAC)". Under this method, only the "social" part and the "cognitive" part of the particle swarm strategy are considered to estimate the new velocity of each particle and particles are reinitialized whenever they are stagnated in the search space. In addition, to overcome the difficulties of selecting an appropriate mutation step size for different problems, a time-varying mutation step size was introduced. Further, for most of the benchmarks, mutation probability is found to be insensitive to the performance of MPSO-TVAC method. On the other hand, the effect of reinitialization velocity on the performance of HPSO-TVAC method is also observed. Time-varying reinitialization step size is found to be an efficient parameter optimization strategy for HPSO-TVAC method. The HPSO-TVAC strategy outperformed all the methods considered in this investigation for most of the functions. Furthermore, it has also been observed that both the MPSO and HPSO strategies perform poorly when the acceleration coefficients are fixed at two.
2,753 citations
TL;DR: A study of boundary conditions is presented indicating the invisible wall technique outperforms absorbing and reflecting wall techniques and is integrated into a representative example of optimization of a profiled corrugated horn antenna.
Abstract: The particle swarm optimization (PSO), new to the electromagnetics community, is a robust stochastic evolutionary computation technique based on the movement and intelligence of swarms. This paper introduces a conceptual overview and detailed explanation of the PSO algorithm, as well as how it can be used for electromagnetic optimizations. This paper also presents several results illustrating the swarm behavior in a PSO algorithm developed by the authors at UCLA specifically for engineering optimizations (UCLA-PSO). Also discussed is recent progress in the development of the PSO and the special considerations needed for engineering implementation including suggestions for the selection of parameter values. Additionally, a study of boundary conditions is presented indicating the invisible wall technique outperforms absorbing and reflecting wall techniques. These concepts are then integrated into a representative example of optimization of a profiled corrugated horn antenna.
2,165 citations
TL;DR: A variation on the traditional PSO algorithm, called the cooperative particle swarm optimizer, or CPSO, employing cooperative behavior to significantly improve the performance of the original algorithm.
Abstract: The particle swarm optimizer (PSO) is a stochastic, population-based optimization technique that can be applied to a wide range of problems, including neural network training. This paper presents a variation on the traditional PSO algorithm, called the cooperative particle swarm optimizer, or CPSO, employing cooperative behavior to significantly improve the performance of the original algorithm. This is achieved by using multiple swarms to optimize different components of the solution vector cooperatively. Application of the new PSO algorithm on several benchmark optimization problems shows a marked improvement in performance over the traditional PSO.
2,038 citations
TL;DR: The canonical particle swarm algorithm is a new approach to optimization, drawing inspiration from group behavior and the establishment of social norms, but each individual is not simply influenced by the best performer among his neighbors.
Abstract: The canonical particle swarm algorithm is a new approach to optimization, drawing inspiration from group behavior and the establishment of social norms. It is gaining popularity, especially because of the speed of convergence and the fact that it is easy to use. However, we feel that each individual is not simply influenced by the best performer among his neighbors. We, thus, decided to make the individuals "fully informed." The results are very promising, as informed individuals seem to find better solutions in all the benchmark functions.
1,682 citations
TL;DR: The proposed PSO method was indeed more efficient and robust in improving the step response of an AVR system and had superior features, including easy implementation, stable convergence characteristic, and good computational efficiency.
Abstract: In this paper, a novel design method for determining the optimal proportional-integral-derivative (PID) controller parameters of an AVR system using the particle swarm optimization (PSO) algorithm is presented. This paper demonstrated in detail how to employ the PSO method to search efficiently the optimal PID controller parameters of an AVR system. The proposed approach had superior features, including easy implementation, stable convergence characteristic, and good computational efficiency. Fast tuning of optimum PID controller parameters yields high-quality solution. In order to assist estimating the performance of the proposed PSO-PID controller, a new time-domain performance criterion function was also defined. Compared with the genetic algorithm (GA), the proposed method was indeed more efficient and robust in improving the step response of an AVR system.
1,485 citations
19 Jun 2004
TL;DR: The results from this study show that DE generally outperforms the other algorithms, however, on two noisy functions, both DE and PSO were outperformed by the EA.
Abstract: Several extensions to evolutionary algorithms (EAs) and particle swarm optimization (PSO) have been suggested during the last decades offering improved performance on selected benchmark problems. Recently, another search heuristic termed differential evolution (DE) has shown superior performance in several real-world applications. In this paper, we evaluate the performance of DE, PSO, and EAs regarding their general applicability as numerical optimization techniques. The comparison is performed on a suite of 34 widely used benchmark problems. The results from our study show that DE generally outperforms the other algorithms. However, on two noisy functions, both DE and PSO were outperformed by the EA.
1,252 citations
19 Jun 2004
TL;DR: The individual particle of a PSO system moving in a quantum multidimensional space is studied and a quantum delta potential well model for PSO is established and a trial method of parameter control and QDPSO is proposed.
Abstract: In this paper, inspired by the analysis of convergence of PSO, we study the individual particle of a PSO system moving in a quantum multidimensional space and establish a quantum delta potential well model for PSO. After that, a trial method of parameter control and QDPSO is proposed. The experiment result shows much advantage of QDPSO to the traditional PSO.
1,202 citations
01 Apr 2004
TL;DR: An evolutionary recurrent network which automates the design of recurrent neural/fuzzy networks using a new evolutionary learning algorithm based on a hybrid of genetic algorithm (GA) and particle swarm optimization (PSO), and is thus called HGAPSO.
Abstract: An evolutionary recurrent network which automates the design of recurrent neural/fuzzy networks using a new evolutionary learning algorithm is proposed in this paper. This new evolutionary learning algorithm is based on a hybrid of genetic algorithm (GA) and particle swarm optimization (PSO), and is thus called HGAPSO. In HGAPSO, individuals in a new generation are created, not only by crossover and mutation operation as in GA, but also by PSO. The concept of elite strategy is adopted in HGAPSO, where the upper-half of the best-performing individuals in a population are regarded as elites. However, instead of being reproduced directly to the next generation, these elites are first enhanced. The group constituted by the elites is regarded as a swarm, and each elite corresponds to a particle within it. In this regard, the elites are enhanced by PSO, an operation which mimics the maturing phenomenon in nature. These enhanced elites constitute half of the population in the new generation, whereas the other half is generated by performing crossover and mutation operation on these enhanced elites. HGAPSO is applied to recurrent neural/fuzzy network design as follows. For recurrent neural network, a fully connected recurrent neural network is designed and applied to a temporal sequence production problem. For recurrent fuzzy network design, a Takagi-Sugeno-Kang-type recurrent fuzzy network is designed and applied to dynamic plant control. The performance of HGAPSO is compared to both GA and PSO in these recurrent networks design problems, demonstrating its superiority.
961 citations
TL;DR: The particle swarm optimizer shares the ability of the genetic algorithm to handle arbitrary nonlinear cost functions, but with a much simpler implementation it clearly demonstrates good possibilities for widespread use in electromagnetic optimization.
Abstract: Particle swarm optimization is a recently invented high-performance optimizer that is very easy to understand and implement. It is similar in some ways to genetic algorithms or evolutionary algorithms, but requires less computational bookkeeping and generally only a few lines of code. In this paper, a particle swarm optimizer is implemented and compared to a genetic algorithm for phased array synthesis of a far-field sidelobe notch, using amplitude-only, phase-only, and complex tapering. The results show that some optimization scenarios are better suited to one method versus the other (i.e., particle swarm optimization performs better in some cases while genetic algorithms perform better in others), which implies that the two methods traverse the problem hyperspace differently. The particle swarm optimizer shares the ability of the genetic algorithm to handle arbitrary nonlinear cost functions, but with a much simpler implementation it clearly demonstrates good possibilities for widespread use in electromagnetic optimization.
877 citations
TL;DR: The approaches include transformations of the objective function through the recently proposed deflection and stretching techniques, as well as a repulsion source at each detected minimizer, resulting in an efficient algorithm which has the ability to avoid previously detected solutions and, thus, detect all global minimizers of a function.
Abstract: This paper presents approaches for effectively computing all global minimizers of an objective function. The approaches include transformations of the objective function through the recently proposed deflection and stretching techniques, as well as a repulsion source at each detected minimizer. The aforementioned techniques are incorporated in the context of the particle swarm optimization (PSO) method, resulting in an efficient algorithm which has the ability to avoid previously detected solutions and, thus, detect all global minimizers of a function. Experimental results on benchmark problems originating from the fields of global optimization, dynamical systems, and game theory, are reported, and conclusions are derived.
01 Dec 2004
TL;DR: A so-called mainstream thought of the population is introduced to evaluate the search scope of a particle and thus a novel parameter control method of QPSO is proposed.
Abstract: Based on the quantum-behaved particle swarm optimization (QPSO) algorithm, we formulate the philosophy of QPSO and introduce a so-called mainstream thought of the population to evaluate the search scope of a particle and thus propose a novel parameter control method of QPSO. After that, we test the revised QPSO algorithm on several benchmark functions and the experiment results show its superiority.
TL;DR: The proposed method out-performs and provides quality solutions compared to other existing techniques for EDP considering valve-point effects are shown in general.
19 Jun 2004
TL;DR: This paper reviews the development of the particle swarm optimization method in recent years and modifications to adapt to different and complex environments are reviewed, and real world applications are listed.
Abstract: This paper reviews the development of the particle swarm optimization method in recent years. Included are brief discussions of various parameters. Modifications to adapt to different and complex environments are reviewed, and real world applications are listed.
TL;DR: The computational experiments show that the proposed HTGA not only can find optimal or close-to-optimal solutions but also can obtain both better and more robust results than the existing algorithm reported recently in the literature.
Abstract: In this paper, a hybrid Taguchi-genetic algorithm (HTGA) is proposed to solve global numerical optimization problems with continuous variables. The HTGA combines the traditional genetic algorithm (TGA), which has a powerful global exploration capability, with the Taguchi method, which can exploit the optimum offspring. The Taguchi method is inserted between crossover and mutation operations of a TGA. Then, the systematic reasoning ability of the Taguchi method is incorporated in the crossover operations to select the better genes to achieve crossover, and consequently, enhance the genetic algorithm. Therefore, the HTGA can be more robust, statistically sound, and quickly convergent. The proposed HTGA is effectively applied to solve 15 benchmark problems of global optimization with 30 or 100 dimensions and very large numbers of local minima. The computational experiments show that the proposed HTGA not only can find optimal or close-to-optimal solutions but also can obtain both better and more robust results than the existing algorithm reported recently in the literature.
01 Jan 2004
TL;DR: Results show Discrete PSO is certainly not as powerful as some specific algorithms, but, on the other hand, it can easily be modified for any discrete/combinatorial problem for which the authors have no good specialized algorithm.
Abstract: The classical Particle Swarm Optimization is a powerful method to find the minimum of a numerical function, on a continuous definition domain. As some binary versions have already successfully been used, it seems quite natural to try to define a framework for a discrete PSO. In order to better understand both the power and the limits of this approach, we examine in detail how it can be used to solve the well known Traveling Salesman Problem, which is in principle very “bad” for this kind of optimization heuristic. Results show Discrete PSO is certainly not as powerful as some specific algorithms, but, on the other hand, it can easily be modified for any discrete/combinatorial problem for which we have no good specialized algorithm.
TL;DR: A new evolutionary approach, particle swarm optimization, is adapted for single-slice 3D-to-3D biomedical image registration and a new hybrid particle swarm technique is proposed that incorporates initial user guidance.
Abstract: Biomedical image registration, or geometric alignment of two-dimensional and/or three-dimensional (3D) image data, is becoming increasingly important in diagnosis, treatment planning, functional studies, computer-guided therapies, and in biomedical research. Registration based on intensity values usually requires optimization of some similarity metric between the images. Local optimization techniques frequently fail because functions of these metrics with respect to transformation parameters are generally nonconvex and irregular and, therefore, global methods are often required. In this paper, a new evolutionary approach, particle swarm optimization, is adapted for single-slice 3D-to-3D biomedical image registration. A new hybrid particle swarm technique is proposed that incorporates initial user guidance. Multimodal registrations with initial orientations far from the ground truth were performed on three volumes from different modalities. Results of optimizing the normalized mutual information similarity metric were compared with various evolutionary strategies. The hybrid particle swarm technique produced more accurate registrations than the evolutionary strategies in many cases, with comparable convergence. These results demonstrate that particle swarm approaches, along with evolutionary techniques and local methods, are useful in image registration, and emphasize the need for hybrid approaches for difficult registration problems.
TL;DR: It is shown that the multi-swarm optimizer significantly outperforms single population PSO on this problem, and that multi-quantum swarms are superior to multi-charged swarms and SOS.
Abstract: Many real-world problems are dynamic, requiring an optimization algorithm which is able to continuously track a changing optimum over time. In this paper, we present new variants of Particle Swarm Optimization (PSO) specifically designed to work well in dynamic environments. The main idea is to extend the single population PSO and Charged Particle Swarm Optimization (CPSO) methods by constructing interacting multi-swarms. In addition, a new algorithmic variant, which broadens the implicit atomic analogy of CPSO to a quantum model, is introduced. The multi-swarm algorithms are tested on a multi-modal dynamic function – the moving peaks benchmark – and results are compared to the single population approach of PSO and CPSO, and to results obtained by a state-of-the-art evolutionary algorithm, namely self-organizing scouts (SOS). We show that our multi-swarm optimizer significantly outperforms single population PSO on this problem, and that multi-quantum swarms are superior to multi-charged swarms and SOS.
17 Jul 2004
TL;DR: This paper reflects on the “swarm” terminology to help clarify its association with various robotic concepts.
Abstract: The term “swarm” has been applied to many systems (in biology, engineering, computation, etc.) as they have some of the qualities that the English-language term “swarm” denotes. With the growth of the various area of “swarm” research, the “swarm” terminology has become somewhat confusing. In this paper, we reflect on this terminology to help clarify its association with various robotic concepts.
TL;DR: In this article, a coarse-grained parallelization of the particle swarm optimization (PSO) algorithm is presented. But the performance of the parallel PSO algorithm is limited by the synchronization requirement of each iteration.
Abstract: Present day engineering optimization problems often impose large computational demands, resulting in long solution times even on a modern high-end processor. To obtain enhanced computational throughput and global search capability, we detail the coarse-grained parallelization of an increasingly popular global search method, the particle swarm optimization (PSO) algorithm. Parallel PSO performance was evaluated using two categories of optimization problems possessing multiple local minima-large-scale analytical test problems with computationally cheap function evaluations and medium-scale biomechanical system identification problems with computationally expensive function evaluations. For load-balanced analytical test problems formulated using 128 design variables, speedup was close to ideal and parallel efficiency above 95% for up to 32 nodes on a Beowulf cluster. In contrast, for load-imbalanced biomechanical system identification problems with 12 design variables, speedup plateaued and parallel efficiency decreased almost linearly with increasing number of nodes. The primary factor affecting parallel performance was the synchronization requirement of the parallel algorithm, which dictated that each iteration must wait for completion of the slowest fitness evaluation. When the analytical problems were solved using a fixed number of swarm iterations, a single population of 128 particles produced a better convergence rate than did multiple independent runs performed using sub-populations (8 runs with 16 particles, 4 runs with 32 particles, or 2 runs with 64 particles). These results suggest that (1) parallel PSO exhibits excellent parallel performance under load-balanced conditions, (2) an asynchronous implementation would be valuable for real-life problems subject to load imbalance, and (3) larger population sizes should be considered when multiple processors are available.
01 Jan 2004
TL;DR: The Unified Particle Swarm Optimization (UPSO) as mentioned in this paper is a new scheme that harnesses the local and global variants of the standard PSO algorithm, combining their exploration and exploitation abilities.
Abstract: We introduce Unified Particle Swarm Optimization, a new scheme that harnesses the local and global variants of the standard Particle Swarm Optimization algorithm, combining their exploration and exploitation abilities. Convergence in probability can be proved for the new approach in unimodal cases and preliminary results justify its superiority against the standard Particle Swarm Optimization.
01 Dec 2004
TL;DR: The results suggest that (1) parallel PSO exhibits excellent parallel performance under load‐balanced conditions, (2) an asynchronous implementation would be valuable for real‐life problems subject to load imbalance, and (3) larger population sizes should be considered when multiple processors are available.
Abstract: : Present day engineering optimization problems often impose large computational demands, resulting in long solution times even on a modern high-end processor. To obtain enhanced computational throughput and global search capability, we detail the coarse-grained parallelization of an increasingly popular global search method, the Particle Swarm Optimization (PSO) algorithm. Parallel PSO performance was evaluated using two categories of optimization problems possessing multiple local minima - large-scale analytical test problems with computationally cheap function evaluations and medium-scale biomechanical system identification problems with computationally expensive function evaluations. For load-balanced analytical test problems formulated using 128 design variables, speedup was close to ideal and parallel efficiency above 95% for up to 32 nodes on a Beowulf cluster. In contrast, for load-imbalanced biomechanical system identification problems with 12 design variables, speedup plateaued and parallel efficiency decreased almost linearly with increasing number of nodes. The primary factor affecting parallel performance was the synchronization requirement of the parallel algorithm, which dictated that each iteration must wait for completion of the slowest fitness evaluation. When the analytical problems were solved using a fixed number of swarm iterations, a single population of 128 particles produced a better convergence rate than did multiple independent runs performed using sub-populations (8 runs with 16 particles, 4 runs with 32 particles, or 2 runs with 64 particles). These results suggest that 1) parallel PSO exhibits excellent parallel performance under load-balanced conditions, 2) an asynchronous implementation would be valuable for real-life problems subject to load imbalance, and 3) larger population sizes should be considered when multiple processors are available.
01 May 2004
TL;DR: The results obtained seem to indicate that Particle Swarm Data Mining Algorithms are competitive, not only with other evolutionary techniques, but also with industry standard algorithms such as the J48 algorithm, and can be successfully applied to more demanding problem domains.
Abstract: Particle Swarm Optimisers are inherently distributed algorithms where the solution for a problem emerges from the interactions between many simple individual agents called particles. This article proposes the use of the Particle Swarm Optimiser as a new tool for Data Mining. In the first phase of our research, three different Particle Swarm Data Mining Algorithms were implemented and tested against a Genetic Algorithm and a Tree Induction Algorithm (J48). From the obtained results, Particle Swarm Optimisers proved to be a suitable candidate for classification tasks. The second phase was dedicated to improving one of the Particle Swarm optimiser variants in terms of attribute type support and temporal complexity. The data sources here used for experimental testing are commonly used and considered as a de facto standard for rule discovery algorithms reliability ranking. The results obtained in these domains seem to indicate that Particle Swarm Data Mining Algorithms are competitive, not only with other evolutionary techniques, but also with industry standard algorithms such as the J48 algorithm, and can be successfully applied to more demanding problem domains.
TL;DR: Various novel heuristic stochastic search techniques have been proposed for optimization of proportional–integral–derivative gains used in Sugeno fuzzy logic based automatic generation control of multi-area thermal generating plants.
19 Jun 2004
TL;DR: A new discrete particle swarm optimization algorithm based on quantum individual is proposed, which is simpler and more powerful than the algorithms available.
Abstract: The particle swarm optimization algorithm is a new methodology in evolutionary computation. It has been found to be extremely effective is solving a wide range of engineering problems, however, it is of low efficiency in dealing with the discrete problems. In this paper, a new discrete particle swarm optimization algorithm based on quantum individual is proposed. It is simpler and more powerful than the algorithms available. The simulation experiments and its application in the CDMA also prove its high efficiency.
19 Jun 2004
TL;DR: A heuristic rule, the smallest position value (SPV) rule, is developed to enable the continuous particle swarm optimization algorithm to be applied to all classes of sequencing problems, which are NP-hard in the literature.
Abstract: In This work we present a particle swarm optimization algorithm to solve the single machine total weighted tardiness problem. A heuristic rule, the smallest position value (SPV) rule, is developed to enable the continuous particle swarm optimization algorithm to be applied to all classes of sequencing problems, which are NP-hard in the literature. A simple but very efficient local search method is embedded in the particle swarm optimization algorithm. The computational results show that the particle swarm algorithm is able to find the optimal and best-known solutions on all instances of widely used benchmarks from the OR library.
TL;DR: The PSO and its variants are applied to a synthetic test system of five types of candidate units with 6- and 14-year planning horizon and the results obtained are compared with dynamic programming in terms of speed and efficiency.
19 Jun 2004
TL;DR: This work presents a simple mechanism to handle constraints with a particle swarm optimization algorithm that uses a simple criterion based on closeness of a particle to the feasible region in order to select a leader.
Abstract: This work presents a simple mechanism to handle constraints with a particle swarm optimization algorithm. Our proposal uses a simple criterion based on closeness of a particle to the feasible region in order to select a leader. Additionally, our algorithm incorporates a turbulence operator that improves the exploratory capabilities of our particle swarm optimization algorithm. Despite its relative simplicity, our comparison of results indicates that the proposed approach is highly competitive with respect to three constraint-handling techniques representative of the state-of-the-art in the area.
27 Sep 2004
TL;DR: In this paper, a particle swarm optimization algorithm-based technique, called PSO-clustering, is proposed to search the cluster center in the arbitrary data set automatically, which can help the user to distinguish the structure of data and simplify the complexity of data from mass information.
Abstract: Clustering analysis is applied generally to pattern recognition, color quantization and image classification. It can help the user to distinguish the structure of data and simplify the complexity of data from mass information. The user can understand the implied information behind extracting these data. In real case, the distribution of information can be any size and shape. A particle swarm optimization algorithm-based technique, called PSO-clustering, is proposed in this article. We adopt the particle swarm optimization to search the cluster center in the arbitrary data set automatically. PSO can search the best solution from the probability option of the social-only model and cognition-only model. This method is quite simple and valid, and it can avoid the minimum local value. Finally, the effectiveness of the PSO-clustering is demonstrated on four artificial data sets.