Systematizing requirements elicitation technique selection
Dante Carrizo
a,
,
Oscar Dieste
b
,
Natalia Juristo
b
a
Atacama University, Avda Copayapu 485, Copiapó, Chile
b
Technical University of Madrid, Boadilla del Monte, Madrid 28660, Spain
abstract
Context: This research deals with requirements elicitation technique selection for software product requirements and the overselection of open
interviews.
Objectives: This paper proposes and validates a framework to help requirements engineers select the most adequate elicitation techniques at any
time.
Method:
We have explored both the existing underlying theory and the results of empirical research to build the framework. Based on this, we have
deduced and put together justified proposals about the framework components. We have also had to add information not found in theoretical or empirical
sources. In these
cases,
we drew on our own experience and expertise.
Results: A new validated approach for requirements technique selection. This new approach selects tech-niques other than open interview, offers a wider
range of possible techniques and captures more require-ments information.
Conclusions: The framework is easily extensible and changeable. Whenever any theoretical or empirical evidence for an attribute, technique or
adequacy value is unearthed, the information can be easily added to the framework.
1.
Introduction
The requirements engineering (RE) process is composed of [1]:
requirements elicitation, analysis, specification, validation and
management. Requirements elicitation covers the capture and dis-
covery of stakeholder needs. Its aim is to identify information
determining what features the software system should have. This
activity is carried out recurrently throughout the requirements
stage.
It often takes place iteratively and is interlinked with other
activities in this stage. Each of these iterations for capturing key
information about requirements is called an elicitation session.
Each session requires preparation, execution and later analysis.
Requirements engineers have to select which elicitation technique
to use in each session. According to Zowghi and Coulin [2], the
preparation of an elicitation session entails: (i) understanding the
application domain by exploring the policy, organizational and
social aspects of the current environment, as well as system and
development constraints; (ii) identifying requirement sources, that
is,
users, experts and any relevant project, process and system, as
well as existing documentation like manuals, forms, reports and
Corresponding author.
Tel.:
+56 52 206809; fax: +56 52 206683.
E-mail
addresses: dante.carrizo@uda.cl (D. Carrizo), odieste@fi.upm.es
(O.
Dieste), natalia@fi.upm.es (N. Juristo).
earlier requirement specifications; (iii) analyzing stakeholders
and identifying key representative users; (iv) selecting techniques,
approaches and tools for use in the requirements process; and (v)
eliciting requirements from stakeholders and other sources. Our
research focuses on the task of selecting techniques for eliciting
requirements (iv).
Software engineers tend tochoosea technique toapply onone of
the following grounds [22]: it is the only technique they are
acquainted
with;
it is their favorite technique for all situations; they
are using a methodology that prescribes a particular technique; or
they guess that the technique is effective under the existing
circumstances. This subjective decision can bias the elicitation
results, degrade the quality of the output requirements, and,
ulti-
mately, have an impact on the quality of the final software product
[9].
In practice, when analysts set out to determine a software sys-
tem’s requirements, they very often use only one technique, inter-
views,
to capture information, even though they are probably
acquainted with several other methods [3]. This could be because
they are unaware of the benefits of each technique, there is no
methodological guidance for the elicitation process or it is standard
procedure: in many cases, an elicitation method or technique is
chosen not for its features or strengths, but simply on the grounds
of history or familiarity [4].
According to Beyer and Holtzblatt [38], there is evidence, how-
ever, that traditional interviews are inadequate for the develop-
ment of systems today. They propose techniques that reflect the
typical master-apprentice relationship in which the stakeholder
is a master of his job but does not necessarily have teaching skills
to communicate actions to the developer. In addition, watching
and experiencing stakeholders doing their job reveals issues, de-
tails,
structure and work strategies that are difficult to capture
clearly using interviews. Other authors, such as Maiden and Rugg,
indicate that probably more than one elicitation technique may be
necessary to get the full range of requirements for most complex
software systems [16]. In short, it must be recognized that there
are contextual conditions that hinder the use of interviews to elicit
requirements, such as stakeholder inaccessibility, schedule coordi-
nation problems, problem domain complexity, stakeholder profile
diversity, disagreement on the problem to be solved, deficient
communication skills or cultural differences between require-
ments engineers and stakeholders.
Some reviews account for tens of elicitation techniques [5].
Many of these techniques have been imported from fields like
cog-
nitive psychology, anthropology, sociology and linguistics [6], and
have been successfully used in knowledge engineering [7] and,
lately, software engineering. Most requirements engineers are
nonetheless unfamiliar with this range of techniques and miss
the chance of optimizing requirements elicitation. This can be
con-
sidered as another side of the well-known breach between theory
and practice [8].
The differences between elicitation techniques are characteris-
tically very large, the inference being that some are likely to per-
form better in some situations than in others. An elicitation
technique’s intrinsic features dictate how it should be applied,
but is not enough to ascertain its adequacy. Methodological sup-
port to help requirements engineers to select the most adequate
technique for the conditions that they face during elicitation could
turn out to be very useful.
This research tackles the problem of selecting elicitation
tech-
niques based on the idiosyncrasy of each technique and the partic-
ularities of the development context. An appropriate choice of
technique optimizes the productivity of the information captured
in the elicitation sessions. This makes it possible to generate a
more correct and complete specification which then results in a
quality final product through a process with fewer holdups [9].
We aim to set up a framework that in practice helps require-
ments engineers to choose a suitable technique to elicit the key
information at any time during development project requirements
elicitation.
Section 2 discusses work related to the problem. Section 3
shows an overview of the proposed framework. Section 4 deter-
mines and defines the relevant contextual attributes for elicitation
and their values, which is the initial information required to ana-
lyze technique fitness. Section 5 establishes the values of elicita-
tion technique fitness for each key contextual attributes value.
Section 6 presents the procedure for selecting the best techniques
for a particular project. Section 7 shows an example in which the
framework is applied. Section 8 presents the evaluation of the pro-
posed framework through two experiments. Finally, Section 9 dis-
cusses the results and presents the final conclusions.
2.
Related work
The requirements research field has expanded over recent dec-
ades.
Even so, there is little research addressing how to support
analysts in decision making on the choice of elicitation techniques,
methods or tools for capturing information for specifying
requirements.
Several papers describe elicitation techniques and provide some
instructions on their use [5,10,11,6]. There are also some empirical
studies comparing elicitation techniques. The shortage of experi-
ments and the non-uniformity of the experimental conditions,
variables and techniques that they study make it difficult to infer
the application conditions for elicitation techniques [12]. Some re-
search on elicitation techniques, mainly empirical, comes from the
knowledge engineering area. Although these studies were carried
out before 2000, their results continue to be
valid.
After reviewing about three hundred articles and thirty books
related to requirements engineering, we found that the first studies
designed to prescribe techniques based on contextual attributes
were conducted only ten years ago. This dearth of studies is a sign
of how little research has focused on the selection of elicitation
techniques. Elicitation technique selection is the central goal of
only ten papers. Table 1 summarizes these studies. It uses a set
of criteria for evaluating and comparing the related work. These
criteria are:
• Discipline in which the proposal is applicable: proposals have
been made in knowledge engineering and software engineering
(some comparative studies of techniques have been conducted
in the fields of economics, marketing, psychology). Proposal
objectives are discipline dependent. Thus, the goal in knowl-
edge engineering is to elicit knowledge from experts. The goal
in software engineering is to capture relevant information for
requirements specification. This area is more important for
our purposes, because our proposal aims to support novice
requirements engineers.
• Scope that the proposal aims to cover: proposals may be
designed to help select techniques for broader processes like
the software development process or the requirements process
or specifically for requirements elicitation. The techniques may
differ depending on the scope of use. For our study, the elicita-
tion techniques have the distinctive trait of being user interac-
tion intensive. On this ground, we are interested in the
techniques used in the requirements elicitation activity.
• Type of information on which the proposal is based: proposals
can be based on the expert opinions of their authors and/or
on empirical studies. We believe that this type of research to
support technique selection should be based primarily on
empirical evidence as its recommendations are more reliable.
The proposals should evolve towards this
goal.
• Number of elicitation techniques covered: requirements
engi-
neers reckon with more and more alternatives for capturing
requirements information, close to fifty at present. However,
the proposals should consider, at least initially, a sizeable num-
ber of the most popular techniques.
• Types of elicitation process contextual factors accounted for:
the contextual attributes of the elicitation activity have been
grouped under five factors that influence technique effective-
ness (elicitor, informant, problem domain, solution domain,
and elicitation process). Proposals should, at least initially,
con-
sider all these contextual factors.
• Specification of the contextual attribute values: the possible
values of the contextual attributes at any time during a project
may or may not be available in the proposals. This is important
as the workability of the method depends on the possibility of
quickly and easily determining such values.
• Evolvability of the proposal: proposals may or may not offer
facilities for updating the method. However, we consider that
the determination of other influential attributes, the addition
of other techniques or the incorporation of evidence on their
effectiveness is essential for the validity and use of the proposal
over time.
Table 1
Summary of related work.
Aspect
Proposals
Value
Kim and Dhaliwal and Byrd, Cossick Maiden and Robertson and Davis and Lauesen Zowghi and Jiang and Kheirkhah and
Courtney [13] Benbazat[14] and Zmud [15] Rugg[16] Robertson [17] Hickey[18] [19] Coulin [2] Eberlein [20] Deraman[21]
Area
Target
Basic information
Technique coverage
Contextual factors
Attribute value
Upgrading
Adequacy concept
Practical Supporting Tool
Software engineering
Knowledge engineering
Global process
Elicitation process
Theoretical
Empirical
Both
Low (0-5)
Medium (6-14)
High (>15)
Analyst/developer
Informant
Problem domain
Solution domain
Process
Available
Unclear
Easy
Hard
Product quality
Relevance
Effectiveness
Available
Probably
No
Fig.
1. Proposed technique selection method.
• Variable defined for measuring technique adequacy: software
product quality, effectiveness, productivity or subjective mea-
sures of relevance are all examples of possible technique ade-
quacy variables reported in the literature. We believe that
work should have an objective metric that measures the effec-
tiveness of the technique to capture a certain amount of rele-
vant information to shape the requirements.
• Support software: research can end with a theoretical research
proposal or advance towards a tool that helps practitioners
decide about how to go about elicitation sessions. This is rele-
vant to reduce the gap between practice and theory.
The above criteria are useful for defining the ideal proposal. This
method should deal with the problem of selecting elicitation
tech-
niques to capture relevant information for putting together the
software system requirements. It should be based preferably on
empirical information and cover a wide range of techniques (over
15) and contextual factors (at least 4). The method should help to
easily identify the possible values of the contextual factor
attri-
butes.
Additionally, it should output the most effective techniques
for an elicitation session, ordered or rated quantitatively to help
requirements engineers with selection. Finally, the method should
be implemented in a tool to help use and upgrade the method with
new techniques or attributes.
We can see that Lauesen’s [19] is the proposal that comes clos-
est to this model. This proposal fails to meet three criteria: basic
information, which is based solely on the authors’ opinion; contex-
tual factors, which consider only the informant and problem do-
main attributes; and practical supporting
tool,
as it does not
provide support software. Davis and Hickey’s proposal [18] falls
short on four criteria: basic information, which is theoretical;
attri-
bute values, which are unclear; adequacy measure, which is sub-
jective relevance; and practical supporting
tool,
which it does not
appear to provide. Another four proposals are unsuccessful on five
criteria [14,16,17,2].
Consequently, there is no elicitation requirements technique
selection method that holds the minimal properties that we be-
lieve to be necessary to warrant consideration as a significant
and systematic aid for requirements engineering practitioners. This
is the gap that our research sets out to
fill.
3. Approach overview
According to our vision of the elicitation process, a systematic
way of selecting a technique in order to execute each elicitation
session would be to follow the procedure shown in Fig. 1:
1.
Identify Contextual Situation: determine the values associated
with the attributes or features of the development context
where the elicitation techniques are to be applied.
2.
Situation-Technique Adequacy Fit: evaluate the adequacy of
each technique for the problem context configuration.
3. Obtain a
Session
Plan: select one or more techniques in order of
priority and application for the following session or sessions.
Contextual conditions
(e.g.,
the amount and type of available
information), vary in the course of the elicitation process, leading
to a new selection starting from step 1.
So,
elicitation technique selection relies on the following knowl-
edge (also shown in Fig. 1):
• Which attributes influence the effectiveness of the elicitation
techniques? The usual line of attack taken in earlier works to
discover which attributes influence technique effectiveness
has been to identify the characteristics of the elicitation
tech-
niques. We believe, though, that the important factor for
tech-
nique selection is the context in which the technique is
applicable. In other words, we sidestep the intrinsic technique
characteristics and focus on the characteristics of the context
in which the techniques are applicable. The intrinsic technique
characteristics are obviously what decide whether or not a
tech-
nique is adequate for use in a particular situation, but they do
not have to be formally specified in the selection. To be more
precise, our approach primarily establishes the contextual char-
acteristics that make the difference between technique
effectiveness.
• How adequate is each technique for each key attribute value?
Running experiments that turn up the required technique effec-
tiveness information is not a feasible short-term option for set-
tling the technique adequacy issue. Experimentation [23] on all
the techniques and all the configurations of possible attribute
values would be a long-term
goal.
In other words, the empirical
information required by an elicitation technique selection
method is not available at present. Pragmatically speaking,
then,
we have gathered the available theoretical and empirical
information to build the measure of adequacy of each technique
for each value of each attribute. For attribute-technique value
combinations where no such information was available, we
have applied our own experience and reasoning.
• How will the candidate techniques be evaluated to choose the
most adequate technique for use in the next elicitation ses-
sions? Our proposal collects the particular technique adequacy
values for all attribute values for comparison and outputs a pos-
sible elicitation plan (the most adequate elicitation technique or
techniques) for the session. If necessary, guidelines will be pro-
vided to improve the contextual situation and be able to use
more elicitation techniques or improve proposal adequacy.
Each of these components will be developed in the following
sections.
4.
Determining key attributes
Before deciding on the most adequate elicitation technique to
apply at any time during a project, it is first necessary to determine
the contextual attributes that influence technique selection, as re-
ported in Section 3. The procedure enacted to determine the key
contextual attributes was (see Fig. 2):
1.
Review the related literature in search of theoretical proposals
and empirical studies directly reporting or inferring contextual
attributes that possibly influence elicitation technique
effectiveness.
2.
Identify and group the candidate attributes, that is, classify the
identified attributes by the factor to which they belong. We
have defined five factors (elicitor, informant, problem domain,
solution domain and elicitation process)
3. Analyze the candidate attributes by acceptance and rejection
criteria (assessability, instrumentability and theoretical
justifiability).
4.
Determine framework attributes, that is, set up the final
attri-
butes by possibly merging, changing the name or adding
attributes.
To identify the contextual attributes, we reviewed two types of
studies: proposals of elicitation technique selection frameworks
and empirical studies comparing elicitation technique effective-
ness.
We found six elicitation technique selection frameworks
and 11 empirical studies comparing technique effectiveness. The
framework proposals define attributes that their authors suggest
are relevant for elicitation techniques selection. We use a broader
definition for empirical studies, namely, any way of gaining knowl-
edge by means of direct and indirect observation or experience. So,
we consider empirical research ranging from higher quality studies
(like randomized parallel or crossover intervention trials) to lower
quality studies (non-randomized studies, longitudinal studies, case
studies, etc.) [39]. We focus especially on experiments run to dem-
onstrate a difference of effectiveness between certain techniques
after altering a contextual attribute.
From these sources, we output a preliminary set of 34 possible
influential attributes. Framework attributes can be aggregated or
modified whenever any new evidence appears.
We grouped the identified attributes by five factors:
•
Elicitor:
Development team agent that elicits information on
software system requirements. Other names, such as analyst
or requirements engineer, are used in the literature to refer to
this role.
• Informant: Human agent that has information regarding
requirements. Informants can be customers, users and, gener-
ally, any software development stakeholders. Non-human
sources have not been considered in this research.
• Problem domain: The problem that the software system under
development is to solve.
• Solution domain: The software product being developed to solve
the problem.
• Elicitation process: The requirements gathering process.
To decide whether an attribute should be selected to be part of
the final attribute set we evaluated whether or not the attribute
can be defined and justified using the assessability, instrument-
ability or theoretical justifiability criteria.
Assessability (A): Possibility of establishing ratings for the differ-
ent attribute values. In other words, this criterion answers the
question How possible is it to set a finite and manageable universe
of admissible attribute value? The possible values are: low (L),
medium (M) and high (H). For example, there are no proposals or
mature theories about the universe of possible values for the Prob-
lem-Solving Methods attribute. Neither is it clear whether these
values could be exclusive to one development or whether several
Fig.
2. Procedure followed to determine framework attributes.