GREEDY RANDOMIZED ADAPTIVE SEARCH PROCEDURES
MAURICIO G.C. RESENDE AND CELSO C. RIBEIRO
Abstract. GRASP is a multi-start metaheuristic for combinatorial problems,
in which each iteration consists basically of two phases: construction and local
search. The construction phase builds a feasible solution, whose neighborhood
is investigated until a local minimum is found during the local search phase.
The best overall solution is kept as the result. In this chapter, we first describe
the basic components of GRASP. Successful implementation techniques and
parameter tuning strategies are discussed and illustrated by numerical results
obtained for different applications. Enhanced or alternative solution construc-
tion mechanisms and techniques to speed up the search are also described:
Reactive GRASP, cost perturbations, bias functions, memory and learning,
local search on partially constructed solutions, hashing, and filtering. We also
discuss in detail implementation strategies of memory-based intensification and
post-optimization techniques using path-relinking. Hybridizations with other
metaheuristics, parallelization strategies, and applications are also reviewed.
1. Introduction
We consider in this chapter a combinatorial optimization problem, defined by a
finite ground set E = {1, . . . , n}, a set of feasible solutions F ⊆ 2
E
, and an objective
function f : 2
E
→
. In the minimization version, we search an optimal solution
S
∗
∈ F such that f(S
∗
) ≤ f(S), ∀S ∈ F . The ground set E, the cost function
f, and the set of feasible solutions F are defined for each specific problem. For
instance, in the case of the traveling salesman problem, the ground set E is that of
all edges connecting the cities to be visited, f(S) is the sum of the costs of all edges
e ∈ S, and F is formed by all egde subsets that determine a Hamiltonian cycle.
The GRASP (Greedy Randomized Adaptive Search Procedure) metaheuristic
[38, 39] is a multi-start or iterative process, in which each iteration consists of two
phases: construction and local search. The construction phase builds a feasible
solution, whose neighborhood is investigated until a local minimum is found during
the local search phase. The best overall solution is kept as the result. An extensive
survey of the literature is given in [44]. The pseudo-code in Figure 1 illustrates the
main blocks of a GRASP procedure for minimization, in which Max Iterations
iterations are performed and Seed is used as the initial seed for the pseudorandom
number generator.
Figure 2 illustrates the construction phase with its pseudo-code. At each itera-
tion of this phase, let the set of candidate elements be formed by all elements that
can be incorporated to the partial solution under construction without destroying
feasibility. The selection of the next element for incorporation is determined by
the evaluation of all candidate elements according to a greedy evaluation function.
Date: August 29, 2002.
AT&T Labs Research Technical Report TD-53RSJY, version 2. To appear in State of the Art
Handbook in Metaheuristics, F. Glover and G. Kochenberger, eds., Kluwer, 2002.
1
2 M.G.C. RESENDE AND C.C. RIBEIRO
procedure GRASP(Max Iterations,Seed)
1 Read Input();
2 for k = 1, . . . , Max Iterations do
3 Solution ← Greedy Randomized Construction(Seed);
4 Solution ← Local Search(Solution);
5 Update Solution(Solution,Best Solution);
6 end;
7 return Best Solution;
end GRASP.
Figure 1. Pseudo-code of the GRASP metaheuristic.
This greedy function usually represents the incremental increase in the cost func-
tion due to the incorporation of this element into the solution under construction.
The evaluation of the elements by this function leads to the creation of a restricted
candidate list (RCL) formed by the best elements, i.e. those whose incorporation
to the current partial solution results in the smallest incremental costs (this is the
greedy aspect of the algorithm). The element to be incorporated into the partial
solution is randomly selected from those in the RCL (this is the probabilistic aspect
of the heuristic). Once the selected element is incorporated to the partial solution,
the candidate list is updated and the incremental costs are reevaluated (this is the
adaptive aspect of the heuristic). This strategy is similar to the semi-greedy heuris-
tic proposed by Hart and Shogan [55], which is also a multi-start approach based
on greedy randomized constructions, but without local search.
procedure Greedy Randomized Construction(Seed)
1 Solution ← ∅;
2 Evaluate the incremental costs of the candidate elements;
3 while Solution is not a complete solution do
4 Build the restricted candidate list (RCL);
5 Select an element s from the RCL at random;
6 Solution ← Solution ∪ {s};
7 Reevaluate the incremental costs;
8 end;
9 return Solution;
end Greedy Randomized Construction.
Figure 2. Pseudo-code of the construction phase.
The solutions generated by a greedy randomized construction are not necessarily
optimal, even with respect to simple neighborhoods. The local search phase usually
improves the constructed solution. A local search algorithm works in an iterative
fashion by successively replacing the current solution by a better solution in the
neighborhood of the current solution. It terminates when no better solution is found
in the neighborhood. The pseudo-code of a basic local search algorithm starting
from the solution Solution constructed in the first phase and using a neighborhood
N is given in Figure 2.
GRASP 3
procedure Local Search(Solution)
1 while Solution is not locally optimal do
2 Find s
0
∈ N(Solution) with f(s
0
) < f(Solution);
3 Solution ← s
0
;
4 end;
5 return Solution;
end Local Search.
Figure 3. Pseudo-code of the local search phase.
The effectiveness of a local search procedure depends on several aspects, such
as the neighborhood structure, the neighborhood search technique, the fast eval-
uation of the cost function of the neighbors, and the starting solution itself. The
construction phase plays a very important role with respect to this last aspect,
building high-quality starting solutions for the local search. Simple neighborhoods
are usually used. The neighborhood search may be implemented using either a best-
improving or a first-improving strategy. In the case of the best-improving strategy,
all neighbors are investigated and the current solution is replaced by the best neigh-
bor. In the case of a first-improving strategy, the current solution moves to the first
neighbor whose cost function value is smaller than that of the current solution. In
practice, we observed on many applications that quite often both strategies lead to
the same final solution, but in smaller computation times when the first-improving
strategy is used. We also observed that premature convergence to a non-global
local minimum is more likely to occur with a best-improving strategy.
2. Construction of the restricted candidate list
An especially appealing characteristic of GRASP is the ease with which it can
be implemented. Few parameters need to be set and tuned. Therefore, develop-
ment can focus on implementing efficient data structures to assure quick iterations.
GRASP has two main parameters: one related to the stopping criterion and another
to the quality of the elements in the restricted candidate list.
The stopping criterion used in the pseudo-code described in Figure 1 is deter-
mined by the number Max
Iterations of iterations. Although the probability of
finding a new solution improving the currently best decreases with the number of
iterations, the quality of the best solution found may only improve with the latter.
Since the computation time does not vary much from iteration to iteration, the
total computation time is predictable and increases linearly with the number of
iterations. Consequently, the larger the number of iterations, the larger will be the
computation time and the better will be the solution found.
For the construction of the RCL used in the first phase we consider, without
loss of generality, a minimization problem as the one formulated in Section 1. We
denote by c(e) the incremental cost associated with the incorporation of element
e ∈ E into the solution under construction. At any GRASP iteration, let c
min
and
c
max
be, respectively, the smallest and the largest incremental costs.
The restricted candidate list RCL is made up of elements e ∈ E with the best
(i.e., the smallest) incremental costs c(e). This list can be limited either by the
number of elements (cardinality-based) or by their quality (value-based). In the
first case, it is made up of the p elements with the best incremental costs, where
4 M.G.C. RESENDE AND C.C. RIBEIRO
p is a parameter. In this chapter, the RCL is associated with a threshold param-
eter α ∈ [0, 1]. The restricted candidate list is formed by all “feasible” elements
e ∈ E which can be inserted into the partial solution under construction with-
out destroying feasibility and whose quality is superior to the threshold value, i.e.,
c(e) ∈ [c
min
, c
min
+ α(c
max
− c
min
)]. The case α = 0 corresponds to a pure greedy
algorithm, while α = 1 is equivalent to a random construction. The pseudo code in
Figure 4 is a refinement of the greedy randomized contruction pseudo-code shown
in Figure 2. It shows that the parameter α controls the amounts of greediness and
randomness in the algorithm.
procedure Greedy Randomized Construction(α, Seed)
1 Solution ← ∅;
2 Initialize the candidate set: C ← E;
3 Evaluate the incremental cost c(e) for all e ∈ C;
4 while C 6= ∅ do
5 c
min
← min{c(e) | e ∈ C};
6 c
max
← max{c(e) | e ∈ C};
7 RCL ← {e ∈ C | c(e) ≤ c
min
+ α(c
max
− c
min
)};
8 Select an element s from the RCL at random;
9 Solution ← Solution ∪ {s};
10 Update the candidate set C;
11 Reevaluate the incremental costs c(e) for all e ∈ C;
12 end;
13 return Solution;
end Greedy Randomized Construction.
Figure 4. Refined pseudo-code of the construction phase.
GRASP may be viewed as a repetitive sampling technique. Each iteration pro-
duces a sample solution from an unknown distribution, whose mean and variance
are functions of the restrictive nature of the RCL. For example, if the RCL is re-
stricted to a single element, then the same solution will be produced at all iterations.
The variance of the distribution will be zero and the mean will be equal to the value
of the greedy solution. If the RCL is allowed to have more elements, then many
different solutions will be produced, implying a larger variance. Since greediness
plays a smaller role in this case, the mean solution value should be worse. However,
the value of the best solution found outperforms the mean value and very often
is optimal. The histograms in Figure 5 illustrate this situation on an instance of
MAXSAT with 100 variables and 850 clauses, depicting results obtained with 1000
independent constructions using the first phase of the GRASP described in [83, 84].
Since this is a maximization problem, the purely greedy construction corresponds
to α = 1, whereas the random construction occurs with α = 0. We notice that when
the value of α increases from 0 to 1, the mean solution value increases towards the
purely greedy solution value, while the variance approaches zero.
For each value of α, we present in Figure 6 histograms with the results obtained
by applying local search to each of the 1000 constructed solutions. Figure 7 sum-
marizes the overall results of this experiment in terms of solution diversity, solution
quality, and computation time. We first observe that the larger the variance of the
GRASP 5
0
100
200
300
400
500
600
405000 410000 415000 420000 425000 430000 435000
occurrences
cost of constructed solution
(a) RCL parameter alpha = 0.0 (random
construction)
0
100
200
300
400
500
600
405000 410000 415000 420000 425000 430000 435000
occurrences
cost of constructed solution
(b) RCL parameter alpha = 0.2
0
100
200
300
400
500
600
405000 410000 415000 420000 425000 430000 435000
occurrences
cost of constructed solution
(c) RCL parameter alpha = 0.4
0
100
200
300
400
500
600
405000 410000 415000 420000 425000 430000 435000
occurrences
cost of constructed solution
(d) RCL parameter alpha = 0.6
0
100
200
300
400
500
600
405000 410000 415000 420000 425000 430000 435000
occurrences
cost of constructed solution
(e) RCL parameter alpha = 0.8
0
200
400
600
800
1000
405000 410000 415000 420000 425000 430000 435000
occurrences
cost of constructed solution
(f) RCL parameter alpha = 1.0 (greedy con-
struction)
Figure 5. Distribution of construction phase solution values as a
function of the RCL parameter α (1000 repetitions were recorded
for each value of α).
