# Particle swarm optimisation of memory usage in embedded systems

## Summary (4 min read)

### 1 INTRODUCTION

- Optimizations with multiple objectives are needed in a great variety of real-life optimization problems.
- Nevertheless, since the decision makers only require a restricted amount of well-distributed solutions along the ParetoOptimal Front (POF), the task of multi-objective optimization methods can be simplified to find a relatively small set of solutions.
- On the other hand, the multi-objective methods can reach a large set of non-dominated solutions, if they are executed for a large number of generations.
- The particles in the population of the H-NSPSO are divided into sub-swarms after each generation by using the fast non-dominated sorting method in Deb et al. (2002), and subsequently these sub-swarms take the responsibility to recover the POF.
- Section 4 explains the experimental results when applying it to a real world problem of embedded systems design.

### 2.1 Multi-objective optimization

- Multi-objective optimization aims at simultaneously optimizing several contradictory objectives.
- For such kind of problems, a single optimal solution does not exist, and compromises have to be made.
- The set of all elements of the search space that are not dominated by any other element is called the Pareto Optimal Front (POF) of the multi-objective problem: it represents the best possible solution with respect to the contradictory objectives.
- A multi-objective optimization problem is solved, when its complete POS is found.

### 2.2 Particle swarm optimization

- Particle Swarm Optimization (PSO) is a heuristic search technique that simulates the movements of a flock of birds that aim at finding food Eberhart and Shi (1998).
- Moore and Chapman proposed the first extension of the PSO strategy for solving multi-objective problems in an unpublished manuscript from 1991 Moore and Chapman (1999).
- The position of each particle is changed according to its own experience and its neighbors.
- All particles are arranged in a tree and each node of the tree contains exactly one particle Janson and Middendorf (2005), and ~xleader is the first particle in the tree.
- A particle is influenced by its own best position so far (~xpbest) and by the best position of the particle that is directly above in the tree .

### 3 HIERARCHICAL NON-DOMINATED SORTING PSO

- The hierarchical version of the Non-dominated PSO, i.e., H-NSPSO is introduced in this section.
- H-NSPSO applies the main mechanisms of the NSGAII Deb et al. (2002).
- Similarly, Xiaodong Li proposed a Non-dominated PSO algorithm Li (2003).
- In the NSPSO algorithm, once a particle has updated its position, instead of comparing the new position only against the ~xpbest position of the particle, all the ~xpbest positions of the swarm and all the new positions recently obtained are combined in just one set (given a total of 2N solutions, where N is the size of the swarm).
- This approach also selects the leaders randomly from the leaders set (stored in an external archive) among the best of them, based on two different mechanisms: a niche count and a nearest neighbor density estimator.

### 3.1 Hierarchical topology

- In H-NSPSO all particles are arranged in several tree networks that define the neighborhood structure.
- Each particle is neighbored to itself and the parent in the tree.
- To this end, the authors apply the fast non-dominated sorting algorithm proposed in Deb et al. (2002), obtaining three fronts in Figure 2.
- Front 1 is the best non-dominated set, since all particles in Front 1 are not dominated by any other particles in the entire population.
- The design of multi-objective optimization algorithm not only requires good convergence quality, but also demands the appropriate distribution quality of the founded solutions in the whole objective space.

### 3.2 Dynamic setting of inertia weight

- The inertia weight (W ) value plays a crucial role in the convergence quality of particle swarm optimization algorithms.
- It controls the effect of the historic speed on the present one, and balances the use of the global research and the partial one.
- Thus, the crowding distance serves in their case as an estimate of the size of the largest cuboid shape enclosing the particle i without including any other particle in the population.
- Such behavior promotes diversity, since a small crowding distance results into a large density of particles.

### 3.3 Learning factors

- In the velocity update equation (5), higher values of C1 ensure larger deviation of the particle in the search space, while the higher values of C2 imply the convergence to the leader.
- To incorporate better compromise between the exploration and exploitation of the search space in PSO, time variant acceleration coefficients have been introduced in Ratnaweera et al. (2004).

### 3.4 Mutation operator

- In general, when the velocities of the particles are almost zero, it is not possible to generate new solutions which might lead the swarm out of this state.
- Since the leader attracts all members of its sub-swarm, it is possible to move the sub-swarm away from a current location by mutating a single particle, if the mutated particle becomes the new leader.
- This mechanism potentially provides a means both of escaping local optima and of speeding up the search Stacey et al. (2003).
- The use of a mutation operator is very important to avoid local optima and to improve the exploratory capabilities of PSO.
- Moreover, different mutation operators have been proposed in the literature, which mutate components of either the position or the velocity of a particle.

### 3.5 H-NSPSO algorithm

- As the previous algorithm shows, initially, a random swarm pop is created.
- Next, the authors iterate the following procedure until the termination condition is satisfied: First, they create a copy of pop, called childPop.
- Then, the authors assign the crowding distance to each particle in the swarm.
- The fast non-dominated sorting algorithm is employed, dividing particles into non-dominated fronts.
- By means of equation (6) the authors calculate the learning factors (C1 and C2).

### 4 MEMORY OPTIMIZATION

- For having a comparison with the previous proposed HNSPSO, a real world example on embedded applications design is studied here.
- The authors compare their algorithm with other state-of-the art results.
- Latest multimedia embedded devices are enhancing their capabilities and are able to run applications reserved to powerful desktop computers (e.g., 3D games, video players).
- DDT library DDT Description AR Array AR(P), also known as One major Table 1.
- Array of pointers SLL Singly-linked list DLL Doubly-linked list SLL(O).

### 4.1 The Dynamic Data Types exploration problem

- The implementation of a DDT has two main components.
- In the second phase, using this detailed report of the accesses, the authors extract all the information needed by the optimization phase.
- The authors also obtain the gain on memory accesses, memory usage and energy consumption.
- The first benchmark is VDrift, which is a driving simulation game.
- It includes 3128 dynamic variables in its source code for which the authors select the optimal DDT implementation.

### 4.2 Optimization model

- The first term (Nr +Nw) × (1−Npa) × CaccT is for calculating the amount of time taken for the processor to access the cache.
- The bus communication time cost is supposed to be constant (Tbus).
- Table 2 shows the representation of a candidate solution (gray shaded cells).

### 4.3 Experimental methodology

- The model of the embedded system architecture consisted of a processor with an instruction cache, a data cache, and embedded DRAM as main memory.
- The data cache uses a write-through strategy.
- Since this metric is not free from arbitrary scaling of objectives, the authors have evaluated the metric by using normalized objective function values using PISA Bleuler et al. (2003).
- Finally, to compare the performance of five algorithms, all parameters are set as follows: Population/Swarm size: 100 in the case of VDrift and 200 in the case of Physics.
- Polynomial mutation operator (ηm = 20), with probability inversely proportional to the chromosome length (as suggested in Deb et al. (2002)) for NSGA-II and SPEA2, also known as Mutation.

### 4.4 Results

- The authors have explored DDTs for VDrift and Physics with each of the five algorithms proposed (H-NSPSO, NSGA-II, NSPSO, OMOPSO and SPEA2).
- The hypervolume values are calculated by averaging results of 30 trials.
- With respect to VDrift, Figure 4 shows that the surfaces for H-NSPSO, OMOPSO and SPEA2 outperform the surfaces offered by NSPSO and NSGA-II.
- For comparison reasons the authors present Figure 5 to illustrate the optimization process that their methodology performs.
- In both cases, H-NSPSO algorithm reaches better values compared to the other MOEAs.

### 5 CONCLUSIONS

- In the present article, a novel multi-objective PSO algorithm, called H-NSPSO, has been presented.
- This adaptation enables it to attain a good balance between the exploration and the exploitation of the search space.
- The selection of the leader is done from this archive, where all particles are arranged in several tree networks that define the neighborhood structure.
- Results show that H-NSPSO offers better results with respect to the other four algorithms tested.
- Hypervolume metric for VDrift/Physics, also known as Table 4.

Did you find this useful? Give us your feedback

##### Citations

40 citations

27 citations

8 citations

##### References

37,111 citations

### "Particle swarm optimisation of memo..." refers background or methods in this paper

...The particles in the population of the H-NSPSO are divided into sub-swarms after each generation by using the fast non-dominated sorting method in Deb et al. (2002), and subsequently these sub-swarms take the responsibility to recover the POF....

[...]

...We construct the trees by means of the fast non-dominated sorting algorithm proposed in Deb et al. (2002)....

[...]

...To this end, we apply the fast non-dominated sorting algorithm proposed in Deb et al. (2002), obtaining three fronts in Figure 2....

[...]

...H-NSPSO applies the main mechanisms of the NSGAII Deb et al. (2002)....

[...]

...Crossover probability of 0.9 (as suggested in Deb et al. (2002)) for NSGA-II and SPEA2....

[...]

12,134 citations

3,702 citations

### "Particle swarm optimisation of memo..." refers methods in this paper

...Since the size of possible DDT implementations is large and it is not possible to cover the exact set of the POF, we compare the obtained Pareto Front (PF) with each other using the hypervolume metric Zitzler et al. (2003)....

[...]

2,753 citations

### "Particle swarm optimisation of memo..." refers background or methods in this paper

...• C1(0) = 2.5, C1(T ) = 0.5, C2(0) = 0.5, and C2(T ) = 2.5 (as suggested in Ratnaweera et al. (2004)) for HNSPSO....

[...]

...Using the following equation as in Ratnaweera et al. (2004), the values of C1 and C2 are evaluated as follows: C1(t) = (C1(T )− C1(0)) · t T + C1(0) C2(t) = (C2(T )− C2(0)) · t T + C2(0) (6)...

[...]

...Moore and Chapman proposed the first extension of the PSO strategy for solving multi-objective problems in an unpublished manuscript from 1991 Moore and Chapman (1999)....

[...]

...To incorporate better compromise between the exploration and exploitation of the search space in PSO, time variant acceleration coefficients have been introduced in Ratnaweera et al. (2004)....

[...]

2,198 citations