ZurichOpenRepositoryand
Archive
UniversityofZurich
UniversityLibrary
Strickhofstrasse39
CH-8057Zurich
www.zora.uzh.ch
Year:2012
Softwarearchitectureoptimizationmethods:Asystematicliteraturereview
Aleti,Aldeida;Buhnova,Barbora;Grunske,Lars;Koziolek,Anne;Meedeniya,Indika
Abstract:Duetosignicantindustrialdemandstowardsoftwaresystemswithincreasingcomplexityand
challengingqualityrequirements, softwarearchitecturedesignhasbecome animportantdevelopment
activityandtheresearchdomainisrapidlyevolving.Inthelastdecades,softwarearchitectureopti-
mizationmethods,whichaimtoautomatethesearchforanoptimalarchitecturedesignwithrespect
toa(setof)qualityattribute(s),haveproliferated.However,thereportedresultsarefragmentedover
dierentresearchcommunities,multiplesystemdomains,andmultiplequalityattributes.Tointegrate
theexistingresearchresults,wehaveperformedasystematicliteraturereviewandanalyzedtheresults
of188researchpapersfromthedierentresearchcommunities.Basedonthissurvey,ataxonomyhas
beencreatedwhichisusedtoclassifytheexistingresearch.
DOI:https://doi.org/10.1109/TSE.2012.64
PostedattheZurichOpenRepositoryandArchive,UniversityofZurich
ZORAURL:https://doi.org/10.5167/uzh-72268
JournalArticle
AcceptedVersion
Originallypublishedat:
Aleti,Aldeida;Buhnova,Barbora;Grunske,Lars;Koziolek,Anne;Meedeniya,Indika(2012).Software
architecture optimization methods:A systematic literature review.IEEE Transactions onSoftware
Engineering,PP(99):1.
DOI:https://doi.org/10.1109/TSE.2012.64
ACCEPTED FOR IEEE TRANSACTION ON SOFTWARE ENGINEERING – AUTHOR COPY 1
Software Architecture Optimization Methods:
A Systematic Literature Review
Aldeida Aleti, Barbora Buhnova, Lars Grunske, Member, IEEE, Anne Koziolek, Member, IEEE, and Indika
Meedeniya
Abstract—Due to significant industrial demands toward software systems with increasing complexity and challenging quality
requirements, software architecture design has become an important development activity and the research domain is rapidly evolving.
In the last decades, software architecture optimization methods, which aim to automate the search for an optimal architecture design
with respect to a (set of) quality attr ibute(s), have proliferated. However, the reported results are fragmented over different research
communities, multiple system domains, and multiple quality attributes. To integrate the existing research results, we have performed
a systematic literature review and analyzed the results of 188 research papers from the different research communities. Based on
this survey, a taxonomy has been created which is used to classify the existing research. Further more, the systematic analysis of
the research literature provided in this review aims to help the research community in consolidating the existing research efforts and
deriving a research agenda for future developments.
Index Terms—Software Architecture Optimization, Systematic Literature Review, Optimization Methods, Problem Overview
✦
1 INTRODUCTION
Architecture specifications and models [120] are used to
structure complex software systems and to provide a
blueprint that is the foundation for later software engi-
neering activities. Thanks to architecture specifications,
software engineers are better supported in coping with
the increasing complexity of today’s software systems.
Thus, the architecture design phase is considered one of
the most important activities in a software engineering
project [24]. The decisions made during architecture
design have significant implications for economic and
quality goals. Examples of architecture-level decisions
include the selection of software and hardware compo-
nents, their replication, the mapping of software com-
ponents to available hardware nodes, and the overall
system topology.
Problem Description and Motivation. Due to the in-
creasing system complexity, software architects have to
choose from a combinatorially growing number of de-
sign options when searching for an optimal architec-
ture design with respect to a defined (set of) quality
attribute(s) and constraints. This results in a design
space search that is often beyond human capabilities
and makes the architectural design a challenging task
[105]. The need for automated design space exploration
• A. Aleti is with the Faculty of Information Technology, Monash University,
Australia. E-mail: aldeida.aleti@monash.edu
• I. Meedeniya is with the Faculty of ICT, Swinburne University of Tech-
nology, Australia. E-mail: imeedeniya@swin.edu.au
• B. Buhnova is with the Faculty of Informatics, Masaryk University, Czech
Republic. E-mail: buhnova@fi.muni.cz
• L. Grunske is with the Faculty of Computer Science, University of
Kaiserslautern, Germany. E-mail: grunske@cs.uni-kl.de
• A. Koziolek is with the Department of Informatics, University of Zurich,
Switzerland. E-mail: koziolek@ifi.uzh.ch
that improves an existing architecture specification has
been recognized [191] and a plethora of architecture
optimization approaches based on formal architecture
specifications have been developed. To handle the com-
plexity of the task, the optimization approaches restrict
the variability of architectural decisions, optimizing the
architecture by modifying one of its specific aspects (al-
location, replication, selection of architectural elements
etc.). Hence the research activities are scattered across
many research communities, system domains (such as
embedded systems or information systems), and quality
attributes. Similar approaches are proposed in multiple
domains without being aware of each other.
Research Approach and Contribution. To connect the
knowledge and provide a comprehensive overview of
the current state of the art, this article provides a
systematic literature review of the existing architecture
optimization approaches. As a result, a gateway to new
approaches of architecture optimization can be opened,
combining different types of architectural decisions dur-
ing the optimization or using unconventional optimiza-
tion techniques. Moreover, new trade-off analysis tech-
niques can be developed by combining results from
different optimization domains. All this can bring sig-
nificant benefits to the general practice of architecture
optimization. In general, with the survey we aim to
achieve the following objectives:
• Provide a basic classification framework in form of a
taxonomy to classify existing architecture optimiza-
tion approaches.
• Provide an overview of the current state of the art
in the architecture optimization domain.
• Point out current trends, gaps, and directions for
future research.
2 ACCEPTED FOR IEEE TRANSACTION ON SOFTWARE ENGINEERING – AUTHOR COPY
We examined 188 papers from multiple research sub-
areas, published in software-engineering journals and
conferences. Initially, we derived a taxonomy by per-
forming a formal content analysis. More specifically,
based on the initial set of keywords and defined inclu-
sion and exclusion criteria, we collected a set of papers,
which we iteratively analyzed to identify the taxonomy
concepts. The taxonomy was then used to classify and
analyze the papers, which provided a comprehensive
overview of the current research in architecture opti-
mization. The data was then used to perform a cross
analysis of different concepts in the taxonomy and derive
gaps and possible directions for further research.
Related surveys. Architecture optimization can be cat-
egorized into the general research discipline of Search
Based Software Engineering (SBSE) [110] as it applies
efficient search strategies to identify an optimal or near-
optimal architecture specification. SBSE is applied in all
phases of the software engineering process including
requirements engineering, project management, design,
maintenance, reverse engineering, and software testing.
A comprehensive survey of different optimization tech-
niques applied to software engineering tasks is provided
by Harman et al. [111]. The survey indicates that in the
past years, a particular increase in SBSE activity has been
witnessed, with many new applications being addressed.
The paper identifies research trends and relationships
between the search techniques and the applications to
which they have been applied. The focus of Harman et
al.’s survey is on the broad field of SBSE, especially on
approaches in the software testing phase which are also
covered in detailed surveys [156], [163]. However, the
area of architecture optimization has not been investi-
gated in detail. The SBSE survey lists several approaches
to optimizing software design, but does not analyze
properties of these approaches except naming the used
optimization strategy.
Beside this general SBSE survey, other surveys de-
scribe sub-areas of architecture optimization and design-
space exploration that are only concerned with a specific
system domains, or a specific optimization method. For
instance, the survey of Grunske et al. [105] is concerned
with the domain of safety-critical embedded systems and
compares 15 architecture optimization methods. Another
example is the survey of Villegas et al. [230], which
evaluates 16 approaches that target run-time architecture
optimizations with a focus on self-adaptive systems. In
the research sub-area of systems with high reliability
demands, Kuo and Wan [140] have published a survey
in 2007 comparing different redundancy allocation ap-
proaches. Finally, several surveys are concerned with the
application of a specific optimization technique, typically
related to Genetic Algorithms [4], [125] or metaheuristics
in general [195].
Although these surveys provide a good overview of
a specific application domain, optimization method, or
even a design phase, none of them is suitable in giving a
comprehensive overview of the existing research in the
area of architecture optimization.
Organization. The rest of the paper is organized as
follows. First, Section 2 outlines the research method
and the underlying protocol for the systematic literature
review. The first contribution of this article, a taxonomy
for architecture optimization approaches that has been
derived from an iterative analysis of the existing research
literature is presented in Section 3. The second contribu-
tion, a classification of existing architecture optimization
approaches according to this taxonomy, is presented in
Section 4. This section contains both a classification into
the categories of the taxonomy including some descrip-
tive statistics as well as a cross-category analysis between
the different taxonomy areas. Finally, Section 5 identifies
future research directions based on the survey results
and Section 6 presents the conclusions.
2 RESEARCH METHOD
Our literature review follows the guidelines proposed by
Kitchenham [129], which structure the stages involved
in a systematic literature review into three phases: plan-
ning, conducting, and reporting the review. Based on the
guidelines, this section details the research questions, the
performed research steps, and the protocol of the litera-
ture review. First, Section 2.1 describes the research ques-
tions underlying our survey. Then, Section 2.2 derives
the research tasks we conducted, and thus describes our
procedure. Section 2.3 then details the literature search
step and highlights the inclusion and exclusion criteria.
Finally, Section 2.4 discusses threats to the validity of our
study.
2.1 Research Questions
Based on the objectives described in the introduction, the
following research questions have been derived, which
form the basis for the literature review:
• RQ1 How can the current research on software
architecture optimization be classified?
• RQ2 What is the current state of software archi-
tecture optimization research with respect to this
classification?
• RQ3 What can be learned from the current research
results that will lead to topics for further investiga-
tion?
2.2 Research Tasks
To answer the three research questions RQ1-3, four
research tasks have been conducted: one task to set up
the literature review, and three research tasks dedicated
to the identified research questions. The tasks have been
conducted in a sequential manner and interconnected
through a number of artifacts generated by their sub-
tasks. The overall research method is outlined in Figure 1
and detailed in the following text.
ALETI et al.: A SYSTEMATIC LITERATURE REVIEW ON SOFTWARE ARCHITECTURE OPTIMIZATION METHODS 3
Definition of the review protocol
Selection of search engines
Keyword definition
Forward and backward
citation search
Raw paper collection
Set-up Task
Formulation of an initial
keyword-based taxonomy
Keywords
New
concepts
found
All papers
reviewed
RQ1 Task - Taxonomy
Classification of the papers
Consistency checking
Cross analysis
Recommendations for future research
RQ2 Task - Paper Classification
RQ3 Task - Research Directions
New keywords found
No new
keywords found
Paper review
Taxonomy update
Taxonomy
consolidation
Descriptive quantitative
analysis
Inclusion/exclusion-criteria
based review of papers
Keyword-based search for papers
Analysis
Taxonomy
Refined paper collection
Classified paper collection
Fig. 1: The process model of our research method.
The set-up task includes the definition of the review
protocol, the selection of search engines, the definition
of a keyword list, a keyword-based collection of pub-
lished architecture optimization papers, and a review
filtering the papers according to a defined set of in-
clusion and exclusion criteria. The search step and the
inclusion/exclusion review step are explained in more
detail in Section 2.3.
Based on the set of selected papers, we performed a
content analysis [135] of the papers in the first research
task (RQ1). The goal was to derive a taxonomy to classify
the current architecture optimization approaches. We
used an iterative coding process to identify the main
categories of the taxonomy. The coding process was
based on the grounded theory [94] qualitative research
method. First we analyzed each paper with the goal to
identify new concepts for the taxonomy. Second, after all
papers have been reviewed and the taxonomy updated
with newly identified concepts, we consolidated the
taxonomy terms, mainly by merging the synonyms and
unifying the concepts on different levels of abstraction.
Section 3 presents the findings.
In the second research task (RQ2), each paper collected
in the set-up task was classified based on the taxonomy
derived in the first research task. Within our team of au-
thors, one person was nominated as a data extractor for
each paper. Furthermore, one person was nominated as a
data checker for each top-level taxonomy category. While
the responsibility of the data extractors was to classify
the papers, data checkers cross-checked the classification
and discussed any inconsistencies with data extractors.
Extracted data was stored in a database, which enabled
a descriptive quantitative analysis. The aim of the data
extraction and the resulting classification was to provide
a significant overview of the current research effort and
the archived results in this domain. Sections 4.1 to 4.3
present the findings.
In the third research task (RQ3), we cross-analyzed
the survey results and synthesized possible directions
for further research. The derivation of possible future
research directions was specifically enabled by the vari-
ety of papers from multiple research sub-areas each of
which has its own strengths. Consequently, the survey
enables the knowledge transfer from one research sub-
area to another and thus aims at improving the overall
research area. Section 4.4 presents the cross-analysis
results, while Section 5 provides our recommendations
for future research.
2.3 Literature Search Process
The search strategy for the review was primarily directed
toward finding published papers in journals and con-
ference proceedings via the widely accepted literature
search engines and databases Google Scholar, IEEE Ex-
plore, ACM Digital Library, Springer Digital Library, and
Elsevier ScienceDirect.
For the search we focused on selected keywords, based
on the aimed scope of the literature review. Examples
of the keywords are: automated selection of software
components, component deployment optimization, en-
ergy consumption optimization, component selection
optimization, automated component selection, reliability
optimization, software safety optimization, redundancy
allocation, optimal scheduling, hardware-software co-
synthesis, search based software engineering, run-time
and design-time architecture optimization, software en-
gineering optimization, self-adaptive software systems.
4 ACCEPTED FOR IEEE TRANSACTION ON SOFTWARE ENGINEERING – AUTHOR COPY
The keywords were refined and extended during the
search process. The final keyword list is available at the
project website [6].
In the subsequent phase, we reviewed the abstracts
(and keywords) of the collected papers with respect
to the defined set of inclusion and exclusion criteria
(Sections 2.3.1 and 2.3.2 below), and further extended the
collection with additional papers based on an analysis
of the cited papers and the ones citing it (forward and
backward citation search). As a result, we included 188
peer-reviewed papers in the survey comprising of papers
from 1992 to 2011, with more than 50% of the papers
published in the last years between 2005 and 2010.
Although the selection process was primarily based on
the review of paper abstracts and keywords, in the cases
where these two were insufficient, we also considered
parts of the introduction, contribution and conclusion
sections.
2.3.1 Inclusion Criteria
The focus of this literature review is on software archi-
tecture optimization. We understand the architecture of a
software system to be “the fundamental organization of
a system embodied in its components, their relationships
to each other, and to the environment, and the princi-
ples guiding its design and evolution” [120]. Software
architecture optimization is understood as an automated
method aiming to reach an optimal architecture design
with respect to a (set of) quality attribute(s). The main
criteria for inclusion were based on the automation of
software architecture optimization, both at run time and
at design time. To enable automated optimization of
software architectures, three basic prerequisites need to
be fulfilled:
1) A machine-processable representation of the soft-
ware architecture must be available as an input
for automated search (e.g. a UML model with
agreed semantics, models described in any archi-
tecture description language, or representations in
formalisms such as Markov chains). Such a repre-
sentation may be an architecture model as defined in
[120], but can also be another machine-processable
representation such as a Markov chain.
2) A function or procedure that automatically eval-
uates an aspect of quality for a given software
architecture is required (called quality evaluation
function/procedure in this work). Different quality
attributes used during the optimization process
were included as long as they were quantifiable
by such a quality evaluation function/procedure.
Cost was also considered since it is a commonly
addressed optimization objective in conjunction
with quality attributes. Both single-objective and
multi-objective problems were taken into account.
Furthermore, papers that solved any type of con-
strained problem were included, not excluding the
papers that did not include constraints.
3) A definition of the considered design space is
required that describes how a given software archi-
tecture representation can be changed or enhanced
by the optimization. We call this information “ar-
chitectural degrees of freedom” [132] in this work
as there is no other agreed term in the context
of architecture optimization. Example architectural
degrees of freedom are allocation, component se-
lection, or hardware parameter change.
Papers that provide these three aspects are included
in our review.
2.3.2 Exclusion Criteria
We excluded papers that: (a) optimize a single com-
ponent without integrating context and interactions
with other architectural elements, (b) focus on an
architecture-irrelevant problem (e.g. requirements pri-
oritization, compiler optimization, or task allocation to
agents that cooperate in executing and finishing the
tasks), (c) optimize hardware with no relation to soft-
ware (e.g. FPGA optimization), or (d) solely optimize
cost without considering any other quality attribute.
Moreover, due to the goal of approach classification,
we excluded the papers discussing an approach already
included in the collection (recognized based on the au-
thor list and approach attributes) and we excluded non-
reviewed papers. We did not exclude papers for quality
reasons, because the quality of the papers was generally
acceptable. Evidence for the quality of the papers can
be found in a post selection analysis of the citations of
each paper via Google Scholar, which in 2012 revealed
that each of the papers has been cited at least once and
the average citation count for the papers included in the
survey was 76.5. The h-index and g-index of the included
papers was 57 and 128 respectively.
2.4 Threats to Validity
One of the main threats to the validity of this systematic
literature review is the incompleteness. The risk of this
threat highly depends on the selected list of keywords
and the limitations of the employed search engines. To
decrease the risk of an incomplete keyword list, we have
used an iterative approach to keyword-list construction.
A well-known set of papers was used to build the initial
taxonomy which evolved over time. New keywords
were added when the keyword list was not able to
find the state-of-the-art in the respective area of study.
In order to omit the limitations implied by employing
a particular search engine, we used multiple search
engines. Moreover, the authors’ expertise in different
system domains, quality attributes, and optimization
approaches reduced the search bias.
Another important issue is whether our taxonomy is
robust enough for the analysis and classification of the
papers. To avoid a taxonomy with insufficient capability
to classify the selected papers, we used an iterative
