A study about HCI in practice of interactive system development using CMMI-DEV

TL;DR: This study is developed on top of one of the widest used software process improvement model: CMMI (Capability Maturity Model Integration) and some initial results are presented.

Abstract: Human-Computer Interaction (HCI) community has proposed many methods, techniques and standards for the development of interactive systems. Despite the great recognition of many of these approaches in academia, little is known about the state of practice in industry. To what extent are they known and used, while using a defined software process, to develop interactive systems? What is their level of use compared to software engineering approaches? This paper presents a study that we have conducted to answer these questions. This study is developed on top of one of the widest used software process improvement model: CMMI (Capability Maturity Model Integration). This paper presents the planning of this study, the research protocol and some initial results.

Summary

1 INTRODUCTION

• Undoubtedly, software process capability and maturity (SPCM) models are reached a large use in industry [14].
• These models are collection of software engineering best practices that help organization to improve their process.
• Based on this belief and taking profit of the large use of CMMI in industry, this paper presents a study that aims to investigate to what extent the HCI approaches are known and used by software developers.
• The authors present this research with some initial results from industry.
• Section 4 and 5 present the planning and execution of the study.

2 A BRIEF OVERVIEW OF CMMI

• In the 1980s, the US Department of Defense (DoD) requested the development of a framework of criteria for evaluating its software providers.
• The CMMI-DEV is a process model (best practices repository) for the realization of any type of product (or system).
• They encompass 40 specific practices (SP) grouped in 14 specific goals (SG).
• Maturity levels are the most applied in industry organizing the process areas in a staged representation where each level is composed of several process areas.
• Process areas of engineering are in the Defined level (3), which means the organization uses a defined process to develop its systems.

3 SUPPORTING CMMI-DEV WITH HCI APPROACHES

• To apply the best practices of CMMI-DEV, software developers should use the existing workbench of techniques, methods, standards and methodologies.
• Explicit and implicit citations were highlighted in the text and reviewed together in a final reading.
• Table 1 presents some categories and their examples of approaches.
• Each interview took in average 1.5 hours.
• A study about HCI in practice using CMMI-DEV IHM’17, August 2017, Poitiers, FRANCE Considering that the authors had 20 experts and 33 propositions, they had 660 responses to analyze.

4 PLANNING THE INVESTIGATION IN INDUSTRY

• To apply the best practices of CMMI-DEV, software developers should use the existing workbench of techniques, methods, standards and methodologies.
• The study was planned to be conducted off-line.
• A questionnaire was available in a web site being composed of two parts: (1) characterization of participants2 and (2) survey questions3 .
• For each category (HCI and SE), participants should answer to what extent they know and use the approaches when implementing the practices, using a Visual Analogue Scale (VAS) [15] – a horizontal line with two anchor points, from “None”(0) to “A lot” (10).
• The software developers know and use HCI approaches as well as they know and use SE approaches when applying the same specific practices of CMMI-DEV engineering process area.

5 STUDY EXECUTION AND INITIAL RESULTS

• The CMMI Institute’s database presents the data of 281 Partner Organizations that implement and evaluate CMMI-DEV from different countries.
• Table 3 presents these countries that totalize 207 organizations, their population for this study.
• They have between 9 and 28 years of experience in capability maturity models implementations.
• In the other hand, categories 8 and 9 presented the lowest level of knowledge and use, probably because the categories from design patterns and interaction modeling for HCI are very recently proposed.
• As previously explained, the approaches exemplified in category 1 are quite similar for SE and HCI.

6 RELATED WORK

• For the best of their knowledge, no empirical study investigates the use of HCI approaches in the development of interactive systems following the CMMI-DEV.
• The authors did not used a measure to investigate the level of knowledge about HCI, but they concluded that many software developers do not know well what usability is, and they know even less about UX.
• Ogunyemi et al. [9] performed an exploratory investigation in Nigerian companies; they found that seventeen organizations (77%) indicate to be aware of HCI and five organizations (23%) are not aware of HCI.
• The mean of eight out of fourteen HCI categories is less than 5 for the knowledge.
• The findings of the related studies corroborate with their findings.

7 FINAL REMARKS

• This paper presents the planning and the initial results of a study that aims to identify to what extent software developers that follows CMMI-DEV, a well disseminated model of software engineering best practices, know and use HCI approaches.
• According to their descriptive results, the authors conclude for now that the knowledge and the use of HCI approaches in the industry is not yet enough to develop useful and usable interactive systems.
• The authors continue to perform recalls to obtain more results of their survey and then perform complete statistics analysis.
• The authors next steps will be to focus this investigation in only few countries for a deeper study.
• Moreover, the authors intend to define guidelines of how perform each CMMI-DEV engineering practice considering HCI approaches.

ACKNOWLEDGMENTS

• The authors thank the organizations that responded the survey and CAPES - Science without Borders Program for the financial support to this work.
• The authors thank also the anonymous reviewers for their useful remarks.

Figures

Figure 4: Results by category for Use

Table 2: HCI categories x SE categories

Figure 6: Results by country for Use – Categories 1 and 12

Figure 3: Results by category for Knowledge

Figure 5: Results by country for Knowledge – Categories 1 and 12

Table 1: HCI categories x Examples

Figure 1: Simplified extract of the questionnaire (adapted from [7])

Full text

HAL Id: hal-01577918
https://hal.archives-ouvertes.fr/hal-01577918
Submitted on 28 Aug 2017
HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entic research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.
L’archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diusion de documents
scientiques de niveau recherche, publiés ou non,
émanant des établissements d’enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.
A study about HCI in practice of interactive system
development using CMMI-DEV
Taisa Guidini Gonçalves, Káthia Marçal de Oliveira, Christophe Kolski
To cite this version:
Taisa Guidini Gonçalves, Káthia Marçal de Oliveira, Christophe Kolski. A study about HCI in practice
of interactive system development using CMMI-DEV. 29ème conférence francophone sur l’Interaction
Homme-Machine, AFIHM, Aug 2017, Poitiers, France. pp.169-177, �10.1145/3132129.3132146�. �hal-
01577918�

A study about HCI in practice of
interactive system development
using CMMI-DEV
T. G. Gonçalves
LAMIH CNRS UMR 8201
Le Mont Houy cedex 9, France
taisa.guidinigoncalves@etu.univ-
valenciennes.fr
K. M. Oliveira
LAMIH CNRS UMR 8201
Le Mont Houy cedex 9, France
kathia.oliveira@univ-valenciennes.fr
C. Kolski
LAMIH CNRS UMR 8201
Le Mont Houy cedex 9, France
christophe.kolski@univ-valenciennes.fr
ABSTRACT
Human-Computer Interaction (HCI) community has proposed many methods, techniques and
standards for the development of interactive systems. Despite the great recognition of many of
these approaches in academia, little is known about the state of practice in industry. To what
extent are they known and used, while using a defined software process, to develop interactive
systems? What is their level of use compared to software engineering approaches? This paper
presents a study that we have conducted to answer these questions. This study is developed on top
___________________________
169
169
Permission to make digital or hard copies of part or all of this work for personal or classroom
use is granted without fee provided that copies are not made or distributed for profit or
commercial advantage and that copies bear this notice and the full citation on the first page.
Copyrights for third-party components of this work must be honored. For all other uses,
contact the Owner/Author.
IHM '17, August 29-September 1, 2017, Poitiers, France
© 2017 Copyright is held by the owner/author(s).
ACM ISBN 978-1-4503-5109-6/17/08. h!ps://doi.org/10.1145/3132129.3132146

A study about HCI in practice using CMMI-DEV
IHM’17, August 2017, Poitiers, FRANCE
of one of the widest used software process improvement model: CMMI (Capability Maturity
Model Integration). This paper presents the planning of this study, the research protocol and some
initial results.
CCS CONCEPTS
• Human-centered computing →; Empirical studies in HCI
KEYWORDS
Human-computer Interaction, Interactive systems, Capability Maturity Model, Software
engineering
ACM Reference format:
T. G. Gonçalves, K. M. Oliveira, and C. Kolski. 2017. A study about HCI in practice of interactive system
development using CMMI-DEV. In Proceedings of 29eme Conference Francophone sur l’Interaction Homme-
Machine , Poitiers- Futuroscope, FR, Août 2017 (IHM 2017), 9 pages.
DOI: 10.1145/3132129.3132146
1 INTRODUCTION
Undoubtedly, software process capability and maturity (SPCM) models are reached a large use in
industry [14]. These models are collection of software engineering best practices that help
organization to improve their process. SPCM models define practices for the development of any
kind of system. They do not indicate any specific technique, method, or any specific approach;
they describe what should be performed by the developers of software systems who are free to
choose the approaches in order to implement the practices. One can suppose therefore that
Human-Computer Interaction approaches should naturally be applied when SPCM models are
used in the development of interactive systems. Based on this belief and taking profit of the large
use of CMMI in industry, this paper presents a study that aims to investigate to what extent the
HCI approaches are known and used by software developers. Moreover, to better analyze the
results in a complete way considering the software development view we have also investigated if
the developers know and use HCI approaches as they know and use Software Engineering
approaches.
To that end, we first identified HCI approaches [6-7] adequate to support practices of the
CMMI for development [2] based on HCI experts’ opinion. After that, we looked in the literature
software engineering approaches used to support the practices. Then we defined a specific
research protocol and a questionnaire (survey), and we have surveyed enterprises of the top ten
170
170

A study about HCI in practice using CMMI-DEV
IHM’17, August 2017, Poitiers, FRANCE
countries that use CMMI according to CMMI Institute
1
. In this paper, we present this research
with some initial results from industry.
This paper is organized as follows. Section 2 briefly presents the main concepts of CMMI used
in this study. Section 3 presents how we defined the HCI and SE approaches to support the
practices of CMMI. Section 4 and 5 present the planning and execution of the study. Section 6
discusses the related works and section 7 presents some final remarks.
2 A BRIEF OVERVIEW OF CMMI
In the 1980s, the US Department of Defense (DoD) requested the development of a framework of
criteria for evaluating its software providers. They financed the Software Engineering Institute
(SEI) that presented in 1991 the Capability Maturity Model (CMM) [10]. In 2001, SEI proposed a
new version of this model, the CMMI-DEV (Capability Maturity Model Integration for
Developers), which encompasses the best practices of different models. The CMMI-DEV is a
process model (best practices repository) for the realization of any type of product (or system). It is
however in the development and maintenance of software that it is most used. It is currently in
version 1.3 [2].
The best practices of CMMI-DEV are organized in-group to address specific goals of different
process areas. The CMMI-DEV v1.3 presents 22 process areas that cover engineering, support,
project and process management activities. The engineering process area is directed related to the
development and evaluation of software systems (our interest in this paper) that are: Requirements
Development, Technical Solution, Product Integration, Validation and Verification They
encompass 40 specific practices (SP) grouped in 14 specific goals (SG). To illustrate, an example for
the Requirements Development process area is the SG1- Develop Customer Requirements composed
of two practices: SP1.1 Elicit Needs; SP1.2 Transform Stakeholder Needs into Customer Requirements.
Moreover, CMMI-DEV uses levels to describe an evolutionary path for an organization that
wants to improve its processes. Two types of levels are defined for software process improvement:
(i) capability levels, improvement of one or more chosen process areas; and (ii) maturity levels,
improvement of a set of pre-defined process area. Maturity levels are the most applied in industry
organizing the process areas in a staged representation where each level is composed of several
process areas. The staged representation has five maturity levels (1-5). Process areas of engineering
are in the Defined level (3), which means the organization uses a defined process to develop its
systems.
3 SUPPORTING CMMI-DEV WITH HCI APPROACHES
To apply the best practices of CMMI-DEV, software developers should use the existing workbench
of techniques, methods, standards and methodologies. From this idea, our first
1
http://partners.cmmiinstitute.com/find-partner-organization/
171
171

A study about HCI in practice using CMMI-DEV
IHM’17, August 2017, Poitiers, FRANCE
Table 1: HCI categories x Examples
HCI Category
Examples
Task Analysis
Methods for
HCI
CTT (Concur Task
Tree); K-MAD (Kernel
of Model for Activity
Description); HTA
(Hierarchical Task
Analysis); SADT
(Structured Analysis
and Design
Technique); GTA
(Groupware Task
Analysis).
Evaluation
methods for
HCI review
Heuristic evaluation;
Cognitive
walkthrough;
Groupware
walkthrough.
research activity was to scan all engineering specific practices from CMMI-DEV to identify where
it was indicated that HCI issues should be taken into account.
To analyze the documentation, we initially seek explicit citations of HCI engineering by looking
for: (i) HCI keywords (for example, external interface, end user, prototype); (ii) examples of
techniques or methods of HCI (e.g. end-user task analysis, HCI models); and (iii) examples of work
products (e.g. interface design specifications, user manual). Then, we looked for citations that were
not directly related to HCI Engineering but that we could interpret in benefit of the use of it. We
classify this information as implicit citations. Explicit and implicit citations were highlighted in the
text and reviewed together in a final reading.
With all the identified citations and considering the literature of HCI (for example, [3, 11, 12])
and existing reviews of HCI approaches that support system development (for example, [4,5,13]),
we proposed an initial set of approaches organized in 10 HCI categories. For each HCI category,
we included examples of methods, techniques, and standards. Table 1 presents some categories
and their examples of approaches. Considering the identified citations in each practice, we
associated each category as support for the accomplishment of the practices. For example, “Task
Analysis Methods for HCI” was proposed as approaches to support the specific practices (SP1.1
and SP1.2) of the specific goal 1 of requirements development previous presented. In total, we had
33 propositions supported by HCI categories that were related to specific practices.
With this initial set, we interviewed twenty HCI experts (19 years of experience in average) well
known in the HCI community. They were asked to analyze all propositions presented in a form
(see example in Figure 1) and answer about their level of agreement justifying it. Each interview
took in average 1.5 hours. The experts used the material to discuss and explain the use of several
approaches in the development of interactive systems based on their own experience in academia
and industry (14 experts declared have worked in the industry).
Process Area
and Specific
Goal (SG)
Specific Practice
(SP)
Methods,
techniques,
standards and
patterns of HCI
Answer
Justification
I agree
I partially
agree
I don’t
agree
Requirements
Development
SG1 Develop
Customer
Requirements
SP1.1 Elicit
Needs
Elicit stakeholder
needs,
expectations,
constraints, and
interfaces for all
phases of the
product lifecycle.
Task Analysis
Methods for HCI
x
“Task
Analysis is
further
modeling
activity than
an elicitation
activity.”
Examples: CTT,
K-MAD, HTA,
SADT, GTA.
Figure 1: Simplified extract of the questionnaire (adapted from [7])
172
172