722 IEEE TRANSACTIONS ON SOFTWARE ENGINEERING, VOL. 25, NO. 5, SEPTEMBER/OCTOBER 1999
Defining and Validating Measures
for Object-Based High-Level Design
Lionel C. Briand, Sandro Morasca,
Member
,
IEEE
Computer
Society
, and Victor R. Basili,
Fellow
,
IEEE
Abstract—The availability of significant measures in the early phases of the software development life-cycle allows for better
management of the later phases, and more effective quality assessment when quality can be more easily affected by preventive or
corrective actions. In this paper, we introduce and compare various high-level design measures for object-based software systems.
The measures are derived based on an experimental goal, identifying fault-prone software parts, and several experimental
hypotheses arising from the development of Ada systems for Flight Dynamics Software at the NASA Goddard Space Flight Center
(NASA/GSFC). Specifically, we define a set of measures for cohesion and coupling, which satisfy a previously published set of
mathematical properties that are necessary for any such measures to be valid. We then investigate the measures’ relationship to
fault-proneness on three large scale projects, to provide empirical support for their practical significance and usefulness.
Index Terms—Measurement, object-based design, high-level design, Ada, cohesion, coupling.
——————————
F
——————————
1 INTRODUCTION
OFTWARE measures can help address the most critical
issues in software development and provide support for
planning, predicting, monitoring, controlling, and evaluat-
ing the quality of both software products and processes
[15], [23]. Most existing software measures attempt to cap-
ture attributes of the software code [23]; however, software
code is just one of the artifacts produced during software
development, and, moreover, it is only available at a late
stage. It is widely recognized that the production of better
software requires the improvement of the early develop-
ment phases and the artifacts they produce. The production
of better specifications and designs reduces the need for
extensive review, modification, and rewriting not only of
code, but of specifications and designs. As a result, a soft-
ware organization can save time, cut production costs, and
raise the final product’s quality.
Early availability of measures is a key factor in the suc-
cessful management of software development, since it al-
lows for:
1) the early detection of problems in the artifacts pro-
duced in the initial phases of the life-cycle (specifica-
tion and design documents) and, therefore, reduc-
tion of the cost of change—late identification and
correction of problems are much more costly than
early ones;
2) better software quality monitoring from the early phases
of the life-cycle;
3) quantitative comparison of techniques and empirical
refinement of the processes to which they are applied;
4) more accurate planning of resource allocation, based
upon the predicted quality of the system and its con-
stituent parts.
In this paper, we focus on measures for the high-level de-
sign of object-based
1
software systems, to study whether
information available at this development stage can be used
to support the issues raised in points 1), 2), 3), and 4). We
worked in the context of high-level designs for Flight Dy-
namics software, written in Ada83 [22], in the Software En-
gineering Laboratory at NASA Goddard Space Flight Cen-
ter (GSFC). Our goal was to
define and validate a set of high-level design measures to
evaluate the quality of the high-level design of a software
system with respect to its fault-proneness and understand
which high-level design attributes are likely to make soft-
ware fault-prone.
We set a number of experimental hypotheses that were be-
lieved to be true in the environment of our study. In our
study, we define three families of measures to set the hy-
potheses in measurable terms. These hypotheses were em-
pirically validated based on three projects conducted at the
NASA/GSFC. As with many empirical studies, some of the
hypotheses were supported by the empirical results, while
others were not. In this paper, due to space constraints, we
only report those hypotheses and measures that were sup-
ported by the empirical results.
Specifically, we introduce and theoretically validate, based
on the properties of [12], a family of measures for cohesion
and coupling of high-level object-based software designs.
1. Object-based systems differ from object-oriented systems in that inheri-
tance is not allowed.
0098-5589/99/$10.00 © 1999 IEEE
²²²²²²²²²²²²²²²²
L.C. Briand is with the Fraunhofer-Institute for Experimental Software
Engineering, Technologiepark II, Sauerwiesen 6, D-67661 Kaiserslautern,
Germany. E-mail: briand@iese.fhg.de.
S. Morasca is with the Dip. di Elettronica e Informazione, Politecnico di
Milano, Piazza Leonardo da Vinci 32, I-20133 Milano, Italy.
E-mail: morasca@elet.polimi.it.
V.R. Basili is with the Computer Science Department, University of Mary-
land, College Park, MD 20742. E-mail: basili@cs.umd.edu.
Manuscript received 9 June 1994; revised 17 Feb. 1998.
Recommended for acceptance by not listed.
For information on obtaining reprints of this article, please send e-mail to:
tse@computer.org, and reference IEEECS Log Number 101168.
S
BRIAND ET AL.: DEFINING AND VALIDATING MEASURES FOR OBJECT-BASED HIGH-LEVEL DESIGN 723
Our measures focus and are based on one specific facet of
cohesion and coupling, i.e., that related to declaration links
among data and subroutines appearing in high-level design
module interfaces. Therefore, our measures are not meant to
capture all aspects of cohesion and coupling. For the sake of
comparison and completeness, we also define two simpler
measures based on USES and IS_COMPONENT_OF [25]
relationships between modules. This appears necessary at
this stage of knowledge, where we can only rely on very lim-
ited theoretical and empirical grounds to help us identify
interesting concepts, relationships and objects of study. If our
measures add complexity to the analysis, they should also be
complementary to simpler design measures already pro-
posed in the literature. One of the results of this investigation
is to provide directions for focusing our research on a smaller
set of strategies and concepts.
A number of studies have been published on software
design measures in recent years. It has been shown that
system architecture has an impact on maintainability and
fault-proneness [26], [24], [38], [30], [39], [16], [40], [41], [43],
[1], [17], [2], [44]. These studies have attempted to capture
the design attributes affecting the ease of maintaining and
debugging a software system. Most of the design measures
are based on information flow between subroutines or dec-
laration counts. We think that, even though they provide
interesting insights into the program structure, these
should not be the only strategies to be investigated, since
many other types of program features and relationships
are a priori worth studying. Moreover, there is a need for
comparison among strategies in order to identify worth-
while research directions and build accurate quality pre-
diction models.
In addition, the success and widespread diffusion of ob-
ject-oriented software systems have drawn a good deal of
interest towards the study of the attributes of object-
oriented software systems. A number of studies have been
published (see for instance [19], [8], [28] and [5], [6] for an
extensive survey). These studies generally deal with the
proposal of new measures or the reuse of existing ones in
the framework of object-oriented software code. Our study
goes one step in the direction of object-orientation, at the
high-level design stage, in that it addresses object-based sys-
tems. Therefore, we take into account several important
characteristics of object-oriented software, with one impor-
tant exception—inheritance.
The paper is organized as follows. In Section 2, we con-
cisely outline the overall structure of our study and explain
the process we have carried out and its rationale. Section 3
contains the basic definitions and concepts that are used in
the paper. The cohesion and coupling measures we intro-
duce are presented in Sections 4 and 5, respectively. Based
on the USES and IS_COMPONENT_OF relationships [25],
we also define two simpler measures (Section 6), which are
commonly proposed in the literature and against which we
wish to compare our cohesion and coupling measures.
These two measures were part of a larger set but turned out
to be the only ones yielding positive results as indicators of
fault-proneness (see [10] for further details). Empirical vali-
dation of the measures is shown in Section 7. In Section 8,
we summarize the lessons we have learned, and outline
directions for future research activities.
2 OUTLINE OF THE STUDY
We now describe the measurement activities we carried out,
to provide the reader with a better interpretation frame-
work for our study. The steps we carried out follow the sci-
entific method and concern the setting of experimental
goals and hypotheses, the definition of appropriate meas-
ures, and the theoretical and experimental validation of
those measures. The steps below were basically executed in
a sequential fashion. However, some steps were, to some
extent, executed in parallel, e.g., steps 3 and 4; in addition,
the need occasionally arose in a few points of the execution
to go back to steps that had been already executed.
1) Establish measurement goals. Empirical software en-
gineering fosters the improvement of software prod-
ucts and processes. In this context, measurement
should be seen as a tool for acquiring information that
can be useful for specific improvement purposes.
Thus, precise measurement goals should be set, to en-
sure specific improvement issues of interest are ad-
dressed. It is our opinion that, at this stage, the defini-
tion of universal measures (like in physical sciences)
is a long-term goal, which, however, is only achiev-
able (if at all) after we gain better insights into specific
environments and from specific perspectives in the
short term. Therefore, the definition of a measure
should be driven by both the characteristics of the
context or family of contexts in which it is used and
one or more clearly stated goals that it helps reach.
The goal of our study was to analyze the high-level
design of three software systems in order to under-
stand which high-level design attributes are likely to
make software fault-prone in our application context,
NASA/GFSC.
2) Set experimental hypotheses. Experimental hy-
potheses, derived from the measurement goals, are
necessary to define measures that are somewhat sup-
ported by an underlying theory to be confirmed or
disconfirmed. Thus, we avoid a random search for sta-
tistical significance. Experimental hypotheses estab-
lish a link between the attribute of interest (software
code fault-proneness, in our case) and some attribute
of the object of study, e.g., size, complexity, cohesion,
coupling of software high-level design.
Each measure we introduce in our study is accom-
panied by an experimental hypothesis. However, we
do not claim that these hypotheses are universally
true in any environment: a priori, they may not even
be true in our environment, since they can be discon-
firmed by the empirical validation. Also, other hy-
potheses could be set: other people may come up with
different hypotheses in the same environment, since
our hypotheses capture our beliefs. In addition, we do
not assume that all of these experimental hypotheses
are equally important towards our experimental goal,
i.e., not all of the attributes we take into account have
an equal impact on software fault-proneness. In this
paper, we will only report on those hypotheses that
were confirmed by the empirical validation (Section 7),
and, therefore, we will only introduce those measures
724 IEEE TRANSACTIONS ON SOFTWARE ENGINEERNG, VOL. 25 NO. 5, SEPTEMBER/OCTOBER 1999
that allowed us to quantify these hypotheses. The
reader interested in the negative results of this study
may consult [10].
3) Characterize formally the attributes to be studied.
Experimental hypotheses are stated in terms of attrib-
utes, which are to be quantified by means of measures.
The introduction of appropriate measures is facilitated
by the availability of precise definitions for the attrib-
utes of interest. Unfortunately, such attributes, e.g., size,
complexity, cohesion, coupling, are hardly ever defined
in a precise and unambiguous way, if they are defined
at all. However, approaches have appeared in the re-
cent literature to provide these attributes with less
fuzzy and ambiguous definitions, using mathematical
properties to characterize them [42], [12].
In our study, we have used an instantiation of the
property-based approach of [12] for our object-based
Ada context, to provide theoretical support for the
definition of our measures of cohesion and coupling
based on data declaration dependency links. These
properties allow us to characterize—to the best of our
understanding and knowledge—the two attributes,
and provided us with guidance for measure defini-
tion. They provide supporting evidence that the
measures are theoretically valid, i.e., we measure
what we purport to measure (see step 5).
We want to point out that acceptance of our cohe-
sion and coupling properties is, to some extent, a sub-
jective matter, as with any other set of properties or
rationalization of informal concepts. Also, our proper-
ties are to be interpreted as necessary, but not suffi-
cient. This is the case even for the most consolidated
and well-known ones, such as the properties for dis-
tance. As a consequence, measures might be built that
satisfy our properties but cannot be taken as sensible
cohesion or coupling measures. However, we believe
that, by providing desirable properties for the meas-
ures of cohesion and coupling, we have better clari-
fied our ideas about cohesion and coupling. The
reader has much more solid grounds on which he or
she can either accept our ideas about cohesion and
coupling, or reject them and replace them with other
properties.
4) Identify abstractions of the object of study. Ap-
propriate representations (abstractions according to
[37]) of the object of study are used in measurement
to capture the information needed to build measures
for the software attributes mentioned in the experi-
mental hypotheses. Some examples of product ab-
stractions are data flow graphs and control flow
graphs. In our study, we use graphs based on de-
pendency links between data and subroutines in
high-level software design.
5) Define measures. A measure is defined for capturing
some intuitive concept [31], e.g., size, complexity,
cohesion, coupling, such as those used in the ex-
perimental hypotheses. In our study, we define
measures for cohesion and coupling based on de-
pendency links among data and subroutines in high-
level software design.
The definition of sound measures requires that
they be theoretically validated, to show that they ac-
tually quantify the attributes they purport to measure.
This is argued for our cohesion and coupling meas-
ures because they satisfy the properties for those
measures we established in step 3, and because they
do not satisfy any set of properties for other attributes
such as complexity or size. One of the goals of [12]
was to define sets of properties to identify similarities
and differences across software attributes.
At any rate, as explained in step 3, some caution
must be used in interpreting the results of our theo-
retical validation, as with any theoretical validation,
due to the inherent degree of subjectivity in the for-
malization of intuition and the fact that properties are
necessary but not sufficient. Therefore, the satisfaction
of our cohesion and coupling properties cannot be
strictly taken as conclusive evidence that the meas-
ures we define are cohesion and coupling measures,
but only as supporting evidence. In addition, we do
not claim nor believe that our measures are the “de-
finitive” measures for cohesion and coupling. They
address only one possible aspect of cohesion and
coupling, and, even in our context, they will need fur-
ther refinements.
6) Validate measures empirically. The empirical vali-
dation of a measure actually entails the validation of
the experimental hypotheses involving the attribute
quantified by the measure. Empirical validation ascer-
tains the practical usefulness of a measure in the stud-
ied environment, by showing if the attributes it meas-
ures, e.g., cohesion, influences an external quality at-
tribute [23] of practical interest, e.g., fault-proneness,
and the extent of this influence.
In our empirical validation, based on data collected
at the NASA/GSFC, we have applied a statistical
technique to study the influence of cohesion and cou-
pling on fault-proneness. Validation was facilitated by
the fact that we had defined experimental goals and
hypotheses at the beginning of the study. At any rate,
the external validity of the experimental hypotheses
and measures remains to be investigated in order to
determine whether they are applicable to different
environments and problem domains.
More details about the approach we have followed can be
found in [11].
3 BASIC DEFINITIONS
In this section, we first introduce the basic concepts and the
terminology that we will use in the paper (Section 3.1). We
then define interactions, the data dependency links on which
our cohesion and coupling measures are based (Section 3.2).
3.1 Modules and High-Level Design
Our object of study is the high-level design of a software
system. To define it, we will start from its elementary com-
ponents: software modules. In the literature, there are two
commonly accepted definitions of modules. The first one
sees a module as a subroutine, and has been used in most of
BRIAND ET AL.: DEFINING AND VALIDATING MEASURES FOR OBJECT-BASED HIGH-LEVEL DESIGN 725
the design measurement publications [35], [20], [26], [38],
[40]. The second definition, which takes an object-oriented
perspective, sees a module as a collection of type, data, and
subroutine definitions, i.e., a provider of computational
services [13], [25]. In this view, a module is the implementa-
tion of an Abstract Data Type (ADT), e.g., a package in Ada,
a class in C++. In this paper, unless otherwise specified, we
will use the term subroutine for the first category, and re-
serve the term module for the second category. Modules are
composed of two parts: interface and body (which may be
empty). The interface contains the computational resources
that the module makes visible for use to other modules. The
body contains the implementation details that are not to be
exported.
Modules and subroutines may be related to each other by
IS_COMPONENT_OF and USES relationships [25]. In general,
module/subroutine A is related to module/subroutine B by
an IS_COMPONENT_OF relationship if A is defined within B.
Module/subroutine A is related to module/subroutine B by a
USES relationship if A uses computational services that B
makes available.
Modules and subroutines can be seen as the components
of higher level aggregations, as defined below.
D
EFINITION 1 (Library Module Hierarchy (LMH)). A library
module hierarchy is a hierarchy where nodes are modules and
subroutines, arcs between nodes are IS_COMPONENT_OF
relationships, and there is exactly one top level node, which is
a module.
In the remainder of this paper, we will define concepts and
measures that can be applied to both modules and LMHs,
which are the most significant syntactic aggregation levels
below the subsystem level. For short, we will use the term
software part (sp) to denote either a module or a LMH.
In the high-level design phase of a software system in
our context, only module and subroutine interfaces and
their relationships are defined—detailed design of module
bodies and subroutines is carried out at low-level design
time. Therefore, we define the high-level design of a soft-
ware system as follows.
D
EFINITION 2 (High-level Design). The high-level design of a
software system is a collection of module and subroutine in-
terfaces related to each other by means of USES and
IS_COMPONENT_OF relationships. Precise and formal-
ized information on module or subroutine bodies is not yet
available at this stage.
3.2 Interactions
In this section, we will specifically focus on the dependen-
cies among data declarations and subroutines, which can
propagate inconsistencies when changes are made to a
software system. In this context, data declarations may be
types, variables, or constants. Those dependencies will be
called interactions and will be used to define measures cap-
turing cohesion and coupling within and between software
parts, respectively.
There are four possible kinds of interactionsfrom:
1) data declarations to data declarations,
2) data declarations to subroutines,
3) subroutines to subroutines, and
4) subroutines to data declarations.
However, not all of these dependencies can be detected at
high-level design time. Therefore, we will investigate the
interactions from data declarations to data declarations or
from data declarations to subroutines, which we may detect
from the high-level design of a software system.
D
EFINITION 3 (Data Declaration-Data Declaration (DD). Inter-
action). A data declaration A DD-interacts with another
data declaration B if a change in A’s declaration or use may
cause the need for a change in B’s declaration or use.
The DD-interaction relationship is transitive. If A DD-
interacts with B, and B DD-interacts with C, then a change in
A may cause a change in C, i.e., A DD-interacts with C. Data
declarations can DD-interact with each other regardless of
their location in the designed system. Therefore, the DD-
interaction relationship can link data declarations belonging
to the same software part or different software parts.
The DD-interaction relationships can be defined in terms
of the basic relationships between data declarations al-
lowed by the language, which represent direct DD-
interactions, i.e., not obtained by virtue of the transitivity of
interaction relationships. Data declaration A directly DD-
interacts with data declaration B if A is used in B’s declara-
tion or in a statement where B is assigned a value. As a con-
sequence, as bodies are not available at high-level design
time in our application context, we will only consider inter-
actions detectable from the interfaces.
DD-interactions provide a means to represent the de-
pendency relationships between individual data declara-
tions. Yet, DD-interactions per se are not able to capture the
relationships between individual data declarations and
subroutines.
D
EFINITION 4 (Data Declaration-Subroutine (DS) Interac-
tion). A data declaration DS-interacts with a subroutine if
it DD-interacts with at least one of its data declarations.
Whenever a data declaration DD-interacts with at least one
of the data declarations contained in a subroutine interface,
the DS-interaction relationship between the data declara-
tion and the subroutine can be detected by examining the
high-level design. For instance, from the Ada-like code
fragment in Fig. 1, it is apparent that both type T1 and ob-
ject OBJECT11 DS-interact with procedure SR11, since they
both DD-interact with parameter PAR11, which belongs to
procedure SR11’s interface data declaration.
For graphical convenience, both sets of interaction rela-
tionships will be represented by directed graphs, the DD-
interaction graph, and the DS-interaction graph, respectively.
In both graphs (see Fig. 2, which shows DD- and DS-
interaction graphs for the code fragment of Fig. 1), data
declarations are represented by rounded nodes, subroutines
by thick lined boxes, modules by thin lined boxes, and in-
teractions by arcs.
726 IEEE TRANSACTIONS ON SOFTWARE ENGINEERNG, VOL. 25 NO. 5, SEPTEMBER/OCTOBER 1999
package M1 is
…
type T1 is …;
OBJECT11, OBJECT12: T1:= …;
procedure SR11(PAR11: in T1:= OBJECT11, PAR12: in T1);
…
package M2 is
…
OBJECT21: T1;
type T2 is array (1..100) of T1;
OBJECT22: T2;
procedure SR21(PAR21: in out T2);
…
end M2;
…
OBJECT13: M2.T2;
…
end M1;
with M1; use M1;
package M3 is
…
type T3 is array (1..100) of T1;
OBJECT31, OBJECT32: T1;
procedure SR31(PAR31: in T3, PAR32: in M2.T2);
OBJECT33: T3;
…
end M3;
Fig. 1. Ada-like code fragment.
The notion of interaction can be applied to other object-
based design methods and formalisms such as HOOD (one
of the main object-based design methods [29]) with no basic
changes. For instance, HOOD does not allow direct access
to data in module interfaces, i.e., objects’ provided inter-
face. Using HOOD’s terminology, data must be encapsu-
lated in the internal part of each object (i.e., module) and
must be accessed through public operations provided by
the object. In that case, by looking at the visible part of a
HOOD object description, we would analyze interactions
between type definitions, constants, and operations, i.e., the
same kind of information we have in our Ada context.
When working with other design techniques, one can use
all the available information on the interactions between
the elements of a design. If mechanisms for describing such
interactions exist, then one can apply our approach based
on more information than is available in our case and in the
HOOD case, and obtain more accurate models.
In this study, interactions are used to define measures for
object-based high-level software design, which we introduce
next. It is generally acknowledged that system architecture
should have low coupling and high cohesion [20]. This is
assumed to improve the capability of a system to be decom-
posed in highly independent and easy to understand pieces.
However, the reader should bear in mind that high cohesion
and low coupling may be conflicting goals, i.e., a trade-off
between the two may exist. For instance, a software system
can be made of small modules with a high degree of internal
cohesion but very closely related to each other and, therefore,
with a high level of coupling. Conversely, a software system
(a) (b)
Fig. 2. Graphs for the code fragment in Fig. 1. (a) DD-interaction; (b) DS-interaction.
