scispace - formally typeset
Search or ask a question
Journal ArticleDOI

Particle swarm optimisation of memory usage in embedded systems

TL;DR: A dynamic, non-dominated sorting, multi-objective particle-swarm-based optimiser, named H-NSPSO, for memory usage optimisation in embedded systems, which significantly reduces the computational complexity of others multi- objective particle swarm optimisation algorithms.
Abstract: In this paper, we propose a dynamic, non-dominated sorting, multi-objective particle-swarm-based optimiser, named hierarchical non-dominated sorting particle swarm optimiser (H-NSPSO), for memory usage optimisation in embedded systems. It significantly reduces the computational complexity of others multi- objective particle swarm optimisation (MOPSO) algorithms. Concretely, it first uses a fast non-dominated sorting approach with O(mN 2) computational complexity. Second, it maintains an external archive to store a fixed number of non-dominated particles, which is used to drive the particle population towards the best non-dominated set over many iteration steps. Finally, the proposed algorithm separates particles into multi sub-swarms, building several tree networks as the neighbourhood topology. H-NSPSO has been made adaptive in nature by allowing its vital parameters (inertia weight and learning factors) to change within iterations. The method is evaluated using two real world examples in embedded applications and compared with existing covering methods.

Summary (4 min read)

1 INTRODUCTION

  • Optimizations with multiple objectives are needed in a great variety of real-life optimization problems.
  • Nevertheless, since the decision makers only require a restricted amount of well-distributed solutions along the ParetoOptimal Front (POF), the task of multi-objective optimization methods can be simplified to find a relatively small set of solutions.
  • On the other hand, the multi-objective methods can reach a large set of non-dominated solutions, if they are executed for a large number of generations.
  • The particles in the population of the H-NSPSO are divided into sub-swarms after each generation by using the fast non-dominated sorting method in Deb et al. (2002), and subsequently these sub-swarms take the responsibility to recover the POF.
  • Section 4 explains the experimental results when applying it to a real world problem of embedded systems design.

2.1 Multi-objective optimization

  • Multi-objective optimization aims at simultaneously optimizing several contradictory objectives.
  • For such kind of problems, a single optimal solution does not exist, and compromises have to be made.
  • The set of all elements of the search space that are not dominated by any other element is called the Pareto Optimal Front (POF) of the multi-objective problem: it represents the best possible solution with respect to the contradictory objectives.
  • A multi-objective optimization problem is solved, when its complete POS is found.

2.2 Particle swarm optimization

  • Particle Swarm Optimization (PSO) is a heuristic search technique that simulates the movements of a flock of birds that aim at finding food Eberhart and Shi (1998).
  • Moore and Chapman proposed the first extension of the PSO strategy for solving multi-objective problems in an unpublished manuscript from 1991 Moore and Chapman (1999).
  • The position of each particle is changed according to its own experience and its neighbors.
  • All particles are arranged in a tree and each node of the tree contains exactly one particle Janson and Middendorf (2005), and ~xleader is the first particle in the tree.
  • A particle is influenced by its own best position so far (~xpbest) and by the best position of the particle that is directly above in the tree .

3 HIERARCHICAL NON-DOMINATED SORTING PSO

  • The hierarchical version of the Non-dominated PSO, i.e., H-NSPSO is introduced in this section.
  • H-NSPSO applies the main mechanisms of the NSGAII Deb et al. (2002).
  • Similarly, Xiaodong Li proposed a Non-dominated PSO algorithm Li (2003).
  • In the NSPSO algorithm, once a particle has updated its position, instead of comparing the new position only against the ~xpbest position of the particle, all the ~xpbest positions of the swarm and all the new positions recently obtained are combined in just one set (given a total of 2N solutions, where N is the size of the swarm).
  • This approach also selects the leaders randomly from the leaders set (stored in an external archive) among the best of them, based on two different mechanisms: a niche count and a nearest neighbor density estimator.

3.1 Hierarchical topology

  • In H-NSPSO all particles are arranged in several tree networks that define the neighborhood structure.
  • Each particle is neighbored to itself and the parent in the tree.
  • To this end, the authors apply the fast non-dominated sorting algorithm proposed in Deb et al. (2002), obtaining three fronts in Figure 2.
  • Front 1 is the best non-dominated set, since all particles in Front 1 are not dominated by any other particles in the entire population.
  • The design of multi-objective optimization algorithm not only requires good convergence quality, but also demands the appropriate distribution quality of the founded solutions in the whole objective space.

3.2 Dynamic setting of inertia weight

  • The inertia weight (W ) value plays a crucial role in the convergence quality of particle swarm optimization algorithms.
  • It controls the effect of the historic speed on the present one, and balances the use of the global research and the partial one.
  • Thus, the crowding distance serves in their case as an estimate of the size of the largest cuboid shape enclosing the particle i without including any other particle in the population.
  • Such behavior promotes diversity, since a small crowding distance results into a large density of particles.

3.3 Learning factors

  • In the velocity update equation (5), higher values of C1 ensure larger deviation of the particle in the search space, while the higher values of C2 imply the convergence to the leader.
  • To incorporate better compromise between the exploration and exploitation of the search space in PSO, time variant acceleration coefficients have been introduced in Ratnaweera et al. (2004).

3.4 Mutation operator

  • In general, when the velocities of the particles are almost zero, it is not possible to generate new solutions which might lead the swarm out of this state.
  • Since the leader attracts all members of its sub-swarm, it is possible to move the sub-swarm away from a current location by mutating a single particle, if the mutated particle becomes the new leader.
  • This mechanism potentially provides a means both of escaping local optima and of speeding up the search Stacey et al. (2003).
  • The use of a mutation operator is very important to avoid local optima and to improve the exploratory capabilities of PSO.
  • Moreover, different mutation operators have been proposed in the literature, which mutate components of either the position or the velocity of a particle.

3.5 H-NSPSO algorithm

  • As the previous algorithm shows, initially, a random swarm pop is created.
  • Next, the authors iterate the following procedure until the termination condition is satisfied: First, they create a copy of pop, called childPop.
  • Then, the authors assign the crowding distance to each particle in the swarm.
  • The fast non-dominated sorting algorithm is employed, dividing particles into non-dominated fronts.
  • By means of equation (6) the authors calculate the learning factors (C1 and C2).

4 MEMORY OPTIMIZATION

  • For having a comparison with the previous proposed HNSPSO, a real world example on embedded applications design is studied here.
  • The authors compare their algorithm with other state-of-the art results.
  • Latest multimedia embedded devices are enhancing their capabilities and are able to run applications reserved to powerful desktop computers (e.g., 3D games, video players).
  • DDT library DDT Description AR Array AR(P), also known as One major Table 1.
  • Array of pointers SLL Singly-linked list DLL Doubly-linked list SLL(O).

4.1 The Dynamic Data Types exploration problem

  • The implementation of a DDT has two main components.
  • In the second phase, using this detailed report of the accesses, the authors extract all the information needed by the optimization phase.
  • The authors also obtain the gain on memory accesses, memory usage and energy consumption.
  • The first benchmark is VDrift, which is a driving simulation game.
  • It includes 3128 dynamic variables in its source code for which the authors select the optimal DDT implementation.

4.2 Optimization model

  • The first term (Nr +Nw) × (1−Npa) × CaccT is for calculating the amount of time taken for the processor to access the cache.
  • The bus communication time cost is supposed to be constant (Tbus).
  • Table 2 shows the representation of a candidate solution (gray shaded cells).

4.3 Experimental methodology

  • The model of the embedded system architecture consisted of a processor with an instruction cache, a data cache, and embedded DRAM as main memory.
  • The data cache uses a write-through strategy.
  • Since this metric is not free from arbitrary scaling of objectives, the authors have evaluated the metric by using normalized objective function values using PISA Bleuler et al. (2003).
  • Finally, to compare the performance of five algorithms, all parameters are set as follows: Population/Swarm size: 100 in the case of VDrift and 200 in the case of Physics.
  • Polynomial mutation operator (ηm = 20), with probability inversely proportional to the chromosome length (as suggested in Deb et al. (2002)) for NSGA-II and SPEA2, also known as Mutation.

4.4 Results

  • The authors have explored DDTs for VDrift and Physics with each of the five algorithms proposed (H-NSPSO, NSGA-II, NSPSO, OMOPSO and SPEA2).
  • The hypervolume values are calculated by averaging results of 30 trials.
  • With respect to VDrift, Figure 4 shows that the surfaces for H-NSPSO, OMOPSO and SPEA2 outperform the surfaces offered by NSPSO and NSGA-II.
  • For comparison reasons the authors present Figure 5 to illustrate the optimization process that their methodology performs.
  • In both cases, H-NSPSO algorithm reaches better values compared to the other MOEAs.

5 CONCLUSIONS

  • In the present article, a novel multi-objective PSO algorithm, called H-NSPSO, has been presented.
  • This adaptation enables it to attain a good balance between the exploration and the exploitation of the search space.
  • The selection of the leader is done from this archive, where all particles are arranged in several tree networks that define the neighborhood structure.
  • Results show that H-NSPSO offers better results with respect to the other four algorithms tested.
  • Hypervolume metric for VDrift/Physics, also known as Table 4.

Did you find this useful? Give us your feedback

Content maybe subject to copyright    Report

Particle Swarm Optimization
of Memory usage in
Embedded Systems
Jos´e L. Risco-Mart´ın*
Department of Computer Architecture and Automation,
Complutense University of Madrid, Madrid, Spain
E-mail: jlrisco@dacya.ucm.es
*Corresponding author
Oscar Garnica
Department of Computer Architecture and Automation,
Complutense University of Madrid, Madrid, Spain
E-mail: ogarnica@dacya.ucm.es
Juan Lanchares
Department of Computer Architecture and Automation,
Complutense University of Madrid, Madrid, Spain
E-mail: julandan@dacya.ucm.es
David Atienza
Embedded Systems Laboratory (ESL)
Ecole Polytechnique F´ed´erale de Lausanne (EPFL), Switzerland
E-mail: david.atienza@epfl.ch,
Department of Computer Architecture and Automation,
Complutense University of Madrid, Madrid, Spain
E-mail: datienza@dacya.ucm.es
J. Ignacio Hidalgo
Department of Computer Architecture and Automation,
Complutense University of Madrid, Madrid, Spain
E-mail: hidalgo@dacya.ucm.es
Abstract: In this paper, we propose a dynamic, non-dominated sorting, multi-
objective particle-swarm-based optimizer, named Hierarchical Non-dominated Sort-
ing Particle Swarm Optimizer (H-NSPSO), for memory usage optimization in embed-
ded systems. It significantly reduces the computational complexity of others Multi-
Objective Particle Swarm Optimization (MOPSO) algorithms. Concretely, it first uses
a fast non-dominated sorting approach with O(mN
2
) computational complexity. Sec-
ond, it maintains an external archive to store a fixed number of non-dominated par-
ticles, which is used to drive the particle population towards the best non-dominated
set over many iteration steps. Finally, the proposed algorithm separates particles into
multi sub-swarms, building several tree networks as the neighborhood topology. H-
NSPSO has been made adaptive in nature by allowing its vital parameters (inertia
weight and learning factors) to change within iterations. The method is evaluated
using two real world examples in embedded applications and compared with existing
covering methods.
Keywords: Embedded Systems; Dynamic Memory Optimization; Particle Swarm Op-
timization; Multi-Objective Optimization; Evolutionary Computation.
Biographical notes: Jos´e L. Risco-Mart´ın is is an Assistant Professor in Complutense
University of Madrid, Spain. He received his Ph.D. from Complutense University of
Madrid in 2004. His research interests are computational theory of modeling and simu-
lation, with emphasis on DEVS, dynamic memory management of embedded systems,
and net-centric computing.

1 INTRODUCTION
Optimizations with multiple objectives are needed in a
great variety of real-life optimization problems. In these
problems there are several conflicting objectives to be op-
timized and it is difficult to achieve the best solution. A
multi-objective optimization problem is solved, when all
its Pareto-optimal solutions are found. Indeed, the goal
of multi-objective optimization is to find a set of opti-
mal solutions in one simulation run, in contrast to classi-
cal optimization methods, which generally find one of the
Pareto optimal solutions by converting the initial multi-
objective optimization problem into a single-objective one.
Unfortunately, it is impossible to find the whole set of
Pareto-Optimal Solutions of a continuous front. Never-
theless, since the decision makers only require a restricted
amount of well-distributed solutions along the Pareto-
Optimal Front (POF), the task of multi-objective opti-
mization methods can be simplified to find a relatively
small set of solutions.
In elitist Multi-Objective Evolutionary Algorithm
(MOEA) and Multi-Objective Particle Swarm Optimiza-
tion (MOPSO) methods, the elite solutions are transferred
by an archive to the next generation, and the archive of the
last generation is the output of the method. Consequently,
restricting the size of the archive affects the diversity of
solutions and the computational time. Therefore, most
MOEA and MOPSO methods try to restrict the amount
of solutions in the output, while keeping a good diversity
along the POF.
Diversity of output solutions is studied by applying
methods like niching, clustering or truncation by several re-
searchers Deb (2001), Zitzler (1999). However, these tech-
niques often need a high computational time and at last
we only have a restricted number of solutions in the out-
put Zitzler (1999). On the other hand, the multi-objective
methods can reach a large set of non-dominated solutions,
if they are executed for a large number of generations.
Thus, although they also imply a high computational time,
the decision maker has the flexibility to choose among sev-
eral solutions from the whole POF.
In this paper, we address the problem of finding the
POF by applying MOPSO. MOPSO methods have the
property that the particles move towards the POF dur-
ing generations. Consequently, by running a MOPSO
with a restricted archive size, it is possible to find a well-
distributed set of non-dominated solutions very close to
the POF Mostaghim and Teich (2003). In this work, we
exploit this knowledge to propose another MOPSO, called
Hierarchical Non-dominated Sorting Particle Swarm Opti-
mizer (H-NSPSO), which covers the gaps between the non-
dominated solutions. The particles in the population of
the H-NSPSO are divided into sub-swarms after each gen-
eration by using the fast non-dominated sorting method
in Deb et al. (2002), and subsequently these sub-swarms
take the responsibility to recover the POF. To this aim, we
maintain in H-NSPSO an external archive to store a fixed
Copyright
c
200x Inderscience Enterprises Ltd.
number of non-dominated particles, the inertia weight and
learning factors are modified between iterations, and a mu-
tation operator is applied to each particle. This method
is validated on different test functions and compared with
other state-of-the-art approaches for a real world applica-
tion in the domain of embedded systems design. Results
show that H-NSPSO outperforms other evolutionary algo-
rithms in terms of quality of the generated aproximation
set, under the assumption that the hypervolume metric
reflects the decision maker’s preferences.
The remainder of the paper has the following structure.
Definitions of PSO and a brief background are given in
Section 2. In Section 3, the proposed H-NSPSO method is
studied. Section 4 explains the experimental results when
applying it to a real world problem of embedded systems
design. In Section 5 we summarize the main conclusions
of this work.
2 BACKGROUND
2.1 Multi-objective optimization
Multi-objective optimization aims at simultaneously op-
timizing several contradictory objectives. For such kind
of problems, a single optimal solution does not exist, and
compromises have to be made. Thus, without any loss of
generality, we can assume the following formulation of the
m-objective minimization problem:
Minimize ~z = (f
1
(~x), f
2
(~x), . . . f
m
(~x))
subject to ~x X (1)
where ~x = [x
1
, x
2
, . . . x
n
] is the vector of decision variables,
f
i
: <
n
< , i = 1, 2, . . . , m , are the objective functions
and X <
n
is the feasible region in the decision space.
A solution ~x X is said to dominate another solution
~y X (denoted as ~x ~y) if the following two conditions
are satisfied:
i {1, 2, . . . , m} , f
i
(~x) f
i
(~y)
i {1, 2, . . . , m} , f
i
(~x) < f
i
(~y) (2)
If there is no solution which dominates ~x X, ~x is
said to be a Pareto Optimal Solution (POS). The set of all
elements of the search space that are not dominated by any
other element is called the Pareto Optimal Front (POF) of
the multi-objective problem: it represents the best possible
solution with respect to the contradictory objectives. A
multi-objective optimization problem is solved, when its
complete POS is found.
2.2 Particle swarm optimization
Particle Swarm Optimization (PSO) is a heuristic search
technique that simulates the movements of a flock of birds
that aim at finding food Eberhart and Shi (1998). The
2

relative simplicity of PSO and the fact that is a population-
based technique have made it a natural candidate to be
extended for multi-objective optimization.
Moore and Chapman proposed the first extension of the
PSO strategy for solving multi-objective problems in an
unpublished manuscript from 1991 Moore and Chapman
(1999). There are currently over twenty five different pro-
posals of multi-objective PSOs (or MOPSOs) reported in
the literature Reyes-Sierra and Coello (2006).
In PSO, particles are “flown” through a hyper-
dimensional search space. Changes to the position of the
particles within the search space are based on the social-
psychological tendency of individuals to emulate the suc-
cess of other individuals.
The position of each particle is changed according to its
own experience and its neighbors. Let ~x
i
(t) denote the
position of particle p
i
, at time step t. The position of p
i
is then changed by adding a velocity ~v
i
(t) to the current
position, i.e.:
~x
i
(t) = ~x
i
(t 1) + ~v
i
(t) (3)
The velocity vector reflects the socially exchanged infor-
mation and, in general, is defined in the following way:
~v
i
(t) = W~v
i
(t 1)
+C
1
r
1
(~x
pbest
~x
i
(t 1)) (4)
+C
2
r
2
(~x
leader
~x
i
(t 1))
where:
W is the inertia weight. It is employed to control the
impact of the previous history of velocities.
C
1
and C
2
are the learning factors. C
1
is the cognitive
learning factor and represents the attraction that a
particle has toward its own success. C
2
is the social
learning factor and represents the attraction that a
particle has toward the success of its neighbors.
r
1
, r
1
[0, 1] are random values.
~x
pbest
is the personal best position of particle i,
namely, the position of the particle that has provided
the greatest success.
~x
leader
is the position of the particle that is used to
guide particle i towards better regions of the search
space.
Particles tend to be influenced by the success of any
other element they are connected to. These neighbors are
not necessary particles close to each other in the decision
variable space, but instead are particles that are close to
each other based on a neighborhood topology, which de-
fines the social structure of the swarm.
We can define for instance a fully-connected graph or
star topology Engelbrecht (2002), which connects all the
members of the swarm to one another. In this case, ~x
leader
Figure 1: Tree network topology (each circle represents a
particle). All particles are arranged in a tree, and it is
influenced by its own best position so far and by the best
position of its parent.
in equation (5) is defined by the position of the best particle
of the entire swarm.
We can also define a tree network is shown in Figure 1.
In this topology, all particles are arranged in a tree and
each node of the tree contains exactly one particle Janson
and Middendorf (2005), and ~x
leader
is the first particle in
the tree. A particle is influenced by its own best position so
far (~x
pbest
) and by the best position of the particle that is
directly above in the tree (parent). If a particle has found a
solution that is better than ~x
pbest
parent
at the parent node,
both particles are exchanged. As a result, this topology of-
fers a dynamic neighborhood. This structure is also called
hierarchical topology, where ~x
leader
= ~x
pbest
parent
.
We refer the interested reader to Kennedy (1999) for a
complete survey of other neighborhood topologies.
3 HIERARCHICAL NON-DOMINATED SORTING PSO
Our hierarchical version of the Non-dominated PSO, i.e.,
H-NSPSO is introduced in this section.
H-NSPSO applies the main mechanisms of the NSGA-
II Deb et al. (2002). Similarly, Xiaodong Li proposed a
Non-dominated PSO (NSPSO) algorithm Li (2003). His
approach is based on a fully- connected topology and in-
corporates the behavior of the NSGA-II to the PSO algo-
rithm. In the NSPSO algorithm, once a particle has up-
dated its position, instead of comparing the new position
only against the ~x
pbest
position of the particle, all the ~x
pbest
positions of the swarm and all the new positions recently
obtained are combined in just one set (given a total of 2N
solutions, where N is the size of the swarm). Then, the
approach selects the best solutions among them to define
the next swarm (by means of a non-dominated sorting).
This approach also selects the leaders randomly from the
leaders set (stored in an external archive) among the best
of them, based on two different mechanisms: a niche count
3

Figure 2: In this example particles of a swarm popula-
tion are classified into 3 successive non-dominated fronts.
Particles are arranged in several trees (subswarms).
and a nearest neighbor density estimator.
H-NSPSO introduces three modifications to NSPSO: (1)
instead of a star topology, our algorithm makes use of
a hierarchical topology dividing the population into sub-
swarms, (2) we design a procedure to estimate the value
of the inertia weight based on the crowding distance and
learning factors, and (3) a mutation operator is applied.
3.1 Hierarchical topology
In H-NSPSO all particles are arranged in several tree net-
works that define the neighborhood structure. Each par-
ticle is neighbored to itself and the parent in the tree. We
construct the trees by means of the fast non-dominated
sorting algorithm proposed in Deb et al. (2002).
Figure 2 illustrates the process. First we sort the entire
population according to the level of non-domination. To
this end, we apply the fast non-dominated sorting algo-
rithm proposed in Deb et al. (2002), obtaining three fronts
in Figure 2. Front 1 is the best non-dominated set, since
all particles in Front 1 are not dominated by any other
particles in the entire population. Front 2 is the best non-
dominated set when Front 1 is removed from the popula-
tion, and so forth. Thus, the fast non-dominated sorting
procedure applied to a population returns the list of non-
dominated fronts. The overall complexity of this sorting
algorithm is O(mN
2
), where m is the number of objectives
and N is the size of the population. Next, we construct
the trees according to the distribution of particles over the
three fronts. Starting from Front 3 and Front 2, we assign
to each particle in Front 3 a parent in Front 2. Then, the
process is repeated for Fronts 2 and 1. If there exist just
one front, then the NSPSO topology is applied as in Li
(2003).
The design of multi-objective optimization algorithm
not only requires good convergence quality, but also de-
mands the appropriate distribution quality of the founded
solutions in the whole objective space. Hence, we make
use of a dynamic setting of inertia weight, learning factors
and a mutation operator.
3.2 Dynamic setting of inertia weight
The inertia weight (W ) value plays a crucial role in the
convergence quality of particle swarm optimization algo-
rithms. It controls the effect of the historic speed on the
present one, and balances the use of the global research
and the partial one. In particular, in H-NSPSO we make
use of the crowding distance used in NSGA-II to calculate
the inertia weight of each particle. Thus, the crowding
distance serves in our case as an estimate of the size of
the largest cuboid shape enclosing the particle i without
including any other particle in the population.
Also, we allow the value of W to decrease from W (0) to
W (T ). The value of inertia weight at iteration t is obtained
as:
W (t) = (W (T ) W (0)) ·
t · e
cd
T
+ W (0) · e
cd
(5)
where T is the maximum number of iterations, t is the
iteration number and cd is the crowding distance.
As a result, we can conclude from equation (5) that iner-
tia weight W is [0, W (0)]. Particle’s inertia weight with the
smallest crowding distance tends to be W (0) when t = 0
and W (T ) when t = T , and those with larger crowding
distance tend to be 0. Such behavior promotes diversity,
since a small crowding distance results into a large density
of particles.
3.3 Learning factors
In the velocity update equation (5), higher values of C
1
ensure larger deviation of the particle in the search space,
while the higher values of C
2
imply the convergence to
the leader. To incorporate better compromise between the
exploration and exploitation of the search space in PSO,
time variant acceleration coefficients have been introduced
in Ratnaweera et al. (2004). We also exploit this concept
in the design of H-NSPSO. Thus, we ensure a better search
for the POS in the following way: C
1
is allowed to decrease
from its initial value C
1
(0) to C
1
(T ), whereas C
2
can be
increased from C
2
(0) to C
2
(T ). Using the following equa-
tion as in Ratnaweera et al. (2004), the values of C
1
and
C
2
are evaluated as follows:
C
1
(t) = (C
1
(T ) C
1
(0)) ·
t
T
+ C
1
(0)
C
2
(t) = (C
2
(T ) C
2
(0)) ·
t
T
+ C
2
(0) (6)
3.4 Mutation operator
In general, when the velocities of the particles are almost
zero, it is not possible to generate new solutions which
might lead the swarm out of this state. This behavior can
lead to the whole swarm being trapped in a local optimum
from which it becomes impossible to escape. However,
since the leader attracts all members of its sub-swarm, it
is possible to move the sub-swarm away from a current
4

location by mutating a single particle, if the mutated par-
ticle becomes the new leader. This mechanism potentially
provides a means both of escaping local optima and of
speeding up the search Stacey et al. (2003). Consequently,
the use of a mutation operator is very important to avoid
local optima and to improve the exploratory capabilities of
PSO. To this end, when a solution is chosen to be mutated,
each component is then mutated (randomly changed) or
not with certain probability. Moreover, different mutation
operators have been proposed in the literature, which mu-
tate components of either the position or the velocity of
a particle. In the case of H-NSPSO, we have employed a
mutation operator that randomly changes the position of
a particle in the population.
3.5 H-NSPSO algorithm
Overall, the proposed H-NSPSO can be summarized in the
following algorithm:
Algorithm 1: H-NSPSO::Main()
for k = 1 to popSize do
~x
k
is the position of particle k, ~x
L
its lower bound
and ~x
U
the upper bound. ~v
k
is the velocity;
for i = 1 to n do
~x
k
(i) = rand(~x
L
(i), ~x
U
(i));
~v
k
(i) = 0;
end
end
t = 0;
while t < T do
childpop = pop.clone();
childpop.assignCrowdingDistance();
childPop.assignTopologyAndLeaders();
foreach particle p in childPop do
p.updateParameters(t, T );
p.updateVelocityAndPosition();
p.mutate();
p.evaluate();
p.updatePersonalBest();
end
childPop.add(pop);
pop=childPop.reduceNSGA2();
t = t + 1;
end
As the previous algorithm shows, initially, a random
swarm pop is created. Next, we iterate the following pro-
cedure until the termination condition is satisfied: First,
we create a copy of pop, called childPop. Then, we assign
the crowding distance to each particle in the swarm. When
all the crowding distances are set, we apply the proposed
topology. Consequently, the fast non-dominated sorting al-
gorithm is employed, dividing particles into non-dominated
fronts. After that, we assign leaders according the follow-
ing algorithm:
Algorithm 2: Pop::assignTopologyAndLeaders()
fronts=Pop.fastNonDominatedSort();
if |fronts| = 1 then
assignLeadersAsInNSPSO();
return;
end
i = 2;
while i |fronts| do
frontParent=fronts[i-1];
frontChild=fronts[i];
ratio = |frontChild|/|frontP arent|;
j = 1;
while j |f rontChild| do
pChild=frontChild[j];
p = j/ratio;
pChild.leader=frontParent[p].clone();
j = j + 1;
end
i = i + 1;
end
In this case, we utilize the crowding distance to calculate
the inertia weight W using equation (5). By means of
equation (6) we calculate the learning factors (C
1
and C
2
).
In the following steps we apply the common functions of
the PSO algorithm, including the mutation operator.
Next, we combine the previous swarm pop with the cur-
rent one childPop into the new swarm, which contains 2N
particles. Finally, the solutions of the combined swarm are
sorted according to
n
, as defined in NSGA-II, and the
first N points are selected. Regarding the complexity of
one cycle of the entire algorithm, the basic operations be-
ing performed and the worst case complexities associated
with them are the following ones:
The crowding distance assignment is O(mNlogN)
The non-dominated sort and leaders assignment has
a complexity of O(mN
2
), and
The sorting phase based on
n
is O (2Nlog (2N))
As a result, the overall complexity of the complete algo-
rithm is O(mN
2
).
4 MEMORY OPTIMIZATION
For having a comparison with the previous proposed H-
NSPSO, a real world example on embedded applications
design is studied here. In this section, we compare our
algorithm with other state-of-the art results.
Latest multimedia embedded devices are enhancing their
capabilities and are able to run applications reserved to
powerful desktop computers (e.g., 3D games, video play-
ers). As a result, one of the most important problems de-
signers face nowadays is the integration of a great amount
of applications coming from the general-purpose domain
in a compact and highly-constrained device. One major
5

Citations
More filters
Journal ArticleDOI
TL;DR: A particle re-initialization scheme is also presented in this paper to further improve the execution performance of the PSO and demonstrate that the proposed HW/SW co-design approach to realize PSO is capable of achieving a high-quality solution effectively.

40 citations

Proceedings ArticleDOI
22 Nov 2010
TL;DR: A particle re-initialization scheme is also presented in this paper to further improve the execution performance of the PSO and demonstrate that the proposed HW/SW co-design approach to realize PSO is capable of achieving a high-quality solution effectively.
Abstract: This paper presents a hardware/software (HW/SW) co-design approach using SOPC technique and pipeline design method to improve the performance of particle swarm optimization (PSO) for embedded applications. Based on modular design architecture, a particle updating accelerator module via hardware implementation for updating velocity and position of particles and a fitness evaluation module implemented on a soft-cored processor for evaluating the objective functions are respectively designed and work closely together to accelerate the evolution process. Thanks to a flexible design, the proposed approach can tackle various optimization problems of embedded applications without the need for hardware redesign. To compensate the deficiency in generating truly random numbers by hardware implementation, a particle re-initialization scheme is also presented in this paper to further improve the execution performance of the PSO. Experiment results have demonstrated that the proposed HW/SW co-design approach to realize PSO is capable of achieving a high-quality solution effectively.

27 citations

Proceedings ArticleDOI
01 Dec 2014
TL;DR: Synthesis and simulation results demonstrate that FPGA implementation proposed in this paper converges faster compared to earlier reported work.
Abstract: Attraction towards Particle Swarm Optimization (PSO) algorithm has grown rapidly in the recent times due to being a simple and efficient optimization technique able to solve many continuous multimodal and multidimensional problems. In this paper, we have attempted to realize the PSO algorithm in Xilinx™ Vertex V FPGA (Field Programmable Gate Array). Standard benchmark functions such as Sphere, Rastrigin and Rosenbrock are considered to investigate the efficiency of the implementation. We have studied the effect of diverse swarm size to evaluate the performance of the PSO algorithm. IEEE 754 double precision floating point format is used to implement arithmetic modules. The results obtained in this work are compared with those reported in literature. Synthesis and simulation results demonstrate that FPGA implementation proposed in this paper converges faster compared to earlier reported work.

8 citations

References
More filters
Journal ArticleDOI
TL;DR: This paper suggests a non-dominated sorting-based MOEA, called NSGA-II (Non-dominated Sorting Genetic Algorithm II), which alleviates all of the above three difficulties, and modify the definition of dominance in order to solve constrained multi-objective problems efficiently.
Abstract: Multi-objective evolutionary algorithms (MOEAs) that use non-dominated sorting and sharing have been criticized mainly for: (1) their O(MN/sup 3/) computational complexity (where M is the number of objectives and N is the population size); (2) their non-elitism approach; and (3) the need to specify a sharing parameter. In this paper, we suggest a non-dominated sorting-based MOEA, called NSGA-II (Non-dominated Sorting Genetic Algorithm II), which alleviates all of the above three difficulties. Specifically, a fast non-dominated sorting approach with O(MN/sup 2/) computational complexity is presented. Also, a selection operator is presented that creates a mating pool by combining the parent and offspring populations and selecting the best N solutions (with respect to fitness and spread). Simulation results on difficult test problems show that NSGA-II is able, for most problems, to find a much better spread of solutions and better convergence near the true Pareto-optimal front compared to the Pareto-archived evolution strategy and the strength-Pareto evolutionary algorithm - two other elitist MOEAs that pay special attention to creating a diverse Pareto-optimal front. Moreover, we modify the definition of dominance in order to solve constrained multi-objective problems efficiently. Simulation results of the constrained NSGA-II on a number of test problems, including a five-objective, seven-constraint nonlinear problem, are compared with another constrained multi-objective optimizer, and the much better performance of NSGA-II is observed.

37,111 citations


"Particle swarm optimisation of memo..." refers background or methods in this paper

  • ...The particles in the population of the H-NSPSO are divided into sub-swarms after each generation by using the fast non-dominated sorting method in Deb et al. (2002), and subsequently these sub-swarms take the responsibility to recover the POF....

    [...]

  • ...We construct the trees by means of the fast non-dominated sorting algorithm proposed in Deb et al. (2002)....

    [...]

  • ...To this end, we apply the fast non-dominated sorting algorithm proposed in Deb et al. (2002), obtaining three fronts in Figure 2....

    [...]

  • ...H-NSPSO applies the main mechanisms of the NSGAII Deb et al. (2002)....

    [...]

  • ...Crossover probability of 0.9 (as suggested in Deb et al. (2002)) for NSGA-II and SPEA2....

    [...]

Book
01 Jan 2001
TL;DR: This text provides an excellent introduction to the use of evolutionary algorithms in multi-objective optimization, allowing use as a graduate course text or for self-study.
Abstract: From the Publisher: Evolutionary algorithms are relatively new, but very powerful techniques used to find solutions to many real-world search and optimization problems. Many of these problems have multiple objectives, which leads to the need to obtain a set of optimal solutions, known as effective solutions. It has been found that using evolutionary algorithms is a highly effective way of finding multiple effective solutions in a single simulation run. · Comprehensive coverage of this growing area of research · Carefully introduces each algorithm with examples and in-depth discussion · Includes many applications to real-world problems, including engineering design and scheduling · Includes discussion of advanced topics and future research · Features exercises and solutions, enabling use as a course text or for self-study · Accessible to those with limited knowledge of classical multi-objective optimization and evolutionary algorithms The integrated presentation of theory, algorithms and examples will benefit those working and researching in the areas of optimization, optimal design and evolutionary computing. This text provides an excellent introduction to the use of evolutionary algorithms in multi-objective optimization, allowing use as a graduate course text or for self-study.

12,134 citations

Journal ArticleDOI
TL;DR: This study provides a rigorous analysis of the limitations underlying this type of quality assessment in multiobjective evolutionary algorithms and develops a mathematical framework which allows one to classify and discuss existing techniques.
Abstract: An important issue in multiobjective optimization is the quantitative comparison of the performance of different algorithms. In the case of multiobjective evolutionary algorithms, the outcome is usually an approximation of the Pareto-optimal set, which is denoted as an approximation set, and therefore the question arises of how to evaluate the quality of approximation sets. Most popular are methods that assign each approximation set a vector of real numbers that reflect different aspects of the quality. Sometimes, pairs of approximation sets are also considered. In this study, we provide a rigorous analysis of the limitations underlying this type of quality assessment. To this end, a mathematical framework is developed which allows one to classify and discuss existing techniques.

3,702 citations


"Particle swarm optimisation of memo..." refers methods in this paper

  • ...Since the size of possible DDT implementations is large and it is not possible to cover the exact set of the POF, we compare the obtained Pareto Front (PF) with each other using the hypervolume metric Zitzler et al. (2003)....

    [...]

Journal ArticleDOI
TL;DR: A novel parameter automation strategy for the particle swarm algorithm and two further extensions to improve its performance after a predefined number of generations to overcome the difficulties of selecting an appropriate mutation step size for different problems.
Abstract: This paper introduces a novel parameter automation strategy for the particle swarm algorithm and two further extensions to improve its performance after a predefined number of generations. Initially, to efficiently control the local search and convergence to the global optimum solution, time-varying acceleration coefficients (TVAC) are introduced in addition to the time-varying inertia weight factor in particle swarm optimization (PSO). From the basis of TVAC, two new strategies are discussed to improve the performance of the PSO. First, the concept of "mutation" is introduced to the particle swarm optimization along with TVAC (MPSO-TVAC), by adding a small perturbation to a randomly selected modulus of the velocity vector of a random particle by predefined probability. Second, we introduce a novel particle swarm concept "self-organizing hierarchical particle swarm optimizer with TVAC (HPSO-TVAC)". Under this method, only the "social" part and the "cognitive" part of the particle swarm strategy are considered to estimate the new velocity of each particle and particles are reinitialized whenever they are stagnated in the search space. In addition, to overcome the difficulties of selecting an appropriate mutation step size for different problems, a time-varying mutation step size was introduced. Further, for most of the benchmarks, mutation probability is found to be insensitive to the performance of MPSO-TVAC method. On the other hand, the effect of reinitialization velocity on the performance of HPSO-TVAC method is also observed. Time-varying reinitialization step size is found to be an efficient parameter optimization strategy for HPSO-TVAC method. The HPSO-TVAC strategy outperformed all the methods considered in this investigation for most of the functions. Furthermore, it has also been observed that both the MPSO and HPSO strategies perform poorly when the acceleration coefficients are fixed at two.

2,753 citations


"Particle swarm optimisation of memo..." refers background or methods in this paper

  • ...• C1(0) = 2.5, C1(T ) = 0.5, C2(0) = 0.5, and C2(T ) = 2.5 (as suggested in Ratnaweera et al. (2004)) for HNSPSO....

    [...]

  • ...Using the following equation as in Ratnaweera et al. (2004), the values of C1 and C2 are evaluated as follows: C1(t) = (C1(T )− C1(0)) · t T + C1(0) C2(t) = (C2(T )− C2(0)) · t T + C2(0) (6)...

    [...]

  • ...Moore and Chapman proposed the first extension of the PSO strategy for solving multi-objective problems in an unpublished manuscript from 1991 Moore and Chapman (1999)....

    [...]

  • ...To incorporate better compromise between the exploration and exploitation of the search space in PSO, time variant acceleration coefficients have been introduced in Ratnaweera et al. (2004)....

    [...]

Book
01 Jan 2018
TL;DR: Computational Intelligence: An Introduction, Second Edition offers an in-depth exploration into the adaptive mechanisms that enable intelligent behaviour in complex and changing environments, encompassing swarm intelligence, fuzzy systems, artificial neutral networks, artificial immune systems and evolutionary computation.
Abstract: Computational Intelligence: An Introduction, Second Edition offers an in-depth exploration into the adaptive mechanisms that enable intelligent behaviour in complex and changing environments. The main focus of this text is centred on the computational modelling of biological and natural intelligent systems, encompassing swarm intelligence, fuzzy systems, artificial neutral networks, artificial immune systems and evolutionary computation. Engelbrecht provides readers with a wide knowledge of Computational Intelligence (CI) paradigms and algorithms; inviting readers to implement and problem solve real-world, complex problems within the CI development framework. This implementation framework will enable readers to tackle new problems without any difficulty through a single Java class as part of the CI library. Key features of this second edition include: A tutorial, hands-on based presentation of the material. State-of-the-art coverage of the most recent developments in computational intelligence with more elaborate discussions on intelligence and artificial intelligence (AI). New discussion of Darwinian evolution versus Lamarckian evolution, also including swarm robotics, hybrid systems and artificial immune systems. A section on how to perform empirical studies; topics including statistical analysis of stochastic algorithms, and an open source library of CI algorithms. Tables, illustrations, graphs, examples, assignments, Java code implementing the algorithms, and a complete CI implementation and experimental framework. Computational Intelligence: An Introduction, Second Edition is essential reading for third and fourth year undergraduate and postgraduate students studying CI. The first edition has been prescribed by a number of overseas universities and is thus a valuable teaching tool. In addition, it will also be a useful resource for researchers in Computational Intelligence and Artificial Intelligence, as well as engineers, statisticians, operational researchers, and bioinformaticians with an interest in applying AI or CI to solve problems in their domains. Check out http://www.ci.cs.up.ac.za for examples, assignments and Java code implementing the algorithms.

2,198 citations

Frequently Asked Questions (1)
Q1. What have the authors contributed in "Particle swarm optimization of memory usage in embedded systems" ?

In this paper, the authors propose a dynamic, non-dominated sorting, multiobjective particle-swarm-based optimizer, named Hierarchical Non-dominated Sorting Particle Swarm Optimizer ( H-NSPSO ), for memory usage optimization in embedded systems. It significantly reduces the computational complexity of others MultiObjective Particle Swarm Optimization ( MOPSO ) algorithms. Second, it maintains an external archive to store a fixed number of non-dominated particles, which is used to drive the particle population towards the best non-dominated set over many iteration steps. Finally, the proposed algorithm separates particles into multi sub-swarms, building several tree networks as the neighborhood topology.