A Comparative Evaluation of Two Computer Supported Collaborative Work Systems for Supporting Collaborative Business Process Modeling Activities

TL;DR: A comparative evaluation of different Computer-Supported Collaborative Work (CSCW) environments was conducted to reveal their constraints and affordances for supporting synchronous collaborative business process modeling (cBPM) activities online.

Abstract: In this study, a comparative evaluation of different Computer-Supported Collaborative Work (CSCW) environments was conducted to reveal their constraints and affordances for supporting synchronous collaborative business process modeling (cBPM) activities online. For this purpose, two case studies were carried out with two CSCW systems that differ in terms of their interaction design features for supporting joint work. The dual-eye tracking method was employed to monitor how the participants focused their attention on the shared workspace during cBPM tasks. An interaction analysis was performed on the communicational content exchanged by the participants in chat messages and activities performed on the shared working area in light of the coordination, communication, awareness, group decision-making and team-building aspects of collaboration. The interaction analysis suggested that the design of the systems significantly affected the participants’ performance and the interaction quality throughout cBPM. In addition, a content analysis was conducted to determine the effects of different interaction design methodologies on the formation of the Business Process Modeling (BP Modeling) phases namely elicitation, formalization, validation and verification. The interaction design features of the systems were found to significantly affect the organization of the BP Modeling phases. Based on the findings, some design suggestions were made to enhance the efficiency of collaboration in cBPM practices. It is expected that the results of the study will serve as a guideline for system designers in designing an effective synchronous cBPM tool, and for end users in choosing a system for their synchronous cBPM practices.

Summary

Introduction

• In recent years, organizations have started to describe their core procedures in terms of business processes and invested considerable effort in defining these processes within operational models (Roser & Bauer, 2005).
• In the VMTChat platform, the stakeholders can use the modeling area concurrently.
• In ARISalign, only the active user, who first accesses the model, has the right to edit the model and other users have to wait for the active user to complete his editing on the whiteboard.
• The results of the research can serve as a guideline for system designers in designing a synchronous collaborative BP Modeling tool, and for customers in choosing a tool for their synchronous modeling practices.

Theoretical Background

• Business process management life cycle consists of design and analysis, configuration, enactment and evaluation phases (Weske, 2007).
• This shows that BP Modeling is a multi-stakeholder activity in which the team members jointly discuss, design and document all business processes (Riemer et al., 2011).
• In recent years, cBPM has attracted the attention of several researchers, who investigated cBPM from different perspectives.
• Basheri (2010) examined computer-supported collaborative learning environments such as COLER, COLLECT-UML, CoLeMo and AUTOCOLLEAGUE using the Unified Modeling Language (UML).
• In addition to these studies concentrated on collaboration, Petrusel and Mendling (2013) performed a single user BP Modeling activity with eye tracking technology to investigate the formalization phase of process modeling to explore the relationship between the relevant region of a process model and the answers given to the comprehension questions.

Research Design

• A qualitative exploratory research methodology was employed to closely analyze the interactions among the participants while they were engaged with cBPM work in two different CSCW settings.
• Initially, two valid cases were selected from the ongoing business processes of the Informatics Institute of the Middle East Technical University (METU), Turkey.
• Before the case study was conducted, the team members were given a basic training in modeling elements and systems, in which the VMTChat environment was also introduced.
• The team members had the opportunity to see how each user interface element worked; including the chat window, messaging window and the referencing tool on the whiteboard.

Data Collection & Data Analysis

• The participants performed synchronous collaborative process modeling activities in VMTChat and ARISalign environments.
• The dual eye-tracking records of collaborative model building sessions, VMTChat’s system log and messages in the ARISalign constituted the data sources of the study.
• To understand how the interaction methodologies affected the collaboration between the team members, an interaction analysis was performed on the communicational content obtained from the chat messages and modifications on the whiteboard.
• The timestamps of interactions were then classified as elicitation, formalization, validation or verification based on the content of the messages.

Results

• VMT Chat: Interaction analysis based on the aspects of collaboration Coordination:.
• At this point, M2 asked for IP’s opinion to decide on the model element that best represented the case (Line 5).
• This interaction episode shows that the team members could discuss an issue and easily reach a consensus to create the process model.

Line Timestamp Team

• It is clear that the reference tool facilitated the communication and increased mutual intelligibility by making the team members aware of the objects on the whiteboard and the messages in the chat window.
• A detailed examination of each iteration shows that the IP gave brief information about the process, then the modelers initiated the formalization phase.
• Therefore, the team members could not effectively coordinate the modeling process.

Discussion

• This study aimed to reveal how different interaction methodologies qualitatively affect synchronous process modeling by examining the coordination, communication, awareness, group decision-making and team-building aspects of collaboration.
• This led to ineffective communication flow and poor coordination in terms of the messages and the modeling activities.
• The modeling environment should support both cooperative and collaborative modeling which means the modelers should be able to work both on the same model element and different parts of the model.
• The system should be supported with a communication tool, which enables the team members to send instant messages when required.

Limitations

• The study has inevitably some limitations given the complexities involved with designing CSCW systems that aim to support cBPM processes.
• In Proceedings of the ACM CSCW Conference on Computer-Supported Cooperative Work (pp. 107-114).
• In Proocedings of Business Process Management Workshops (pp.438-450).

Figures

Figure 1 - VMTChat Environment

Table 1 - Demographic profile of the team members

Figure 6 - Chat Window of VMTChat (The blue arrows represent referential links added by the users)

Figure 7 - Temporal Distribution of BP Modeling Phases in VMTChat

Figure 5 - Use of the Reference Tool in the Chat Window

Figure 8 - Formation of BP Modeling Phases in ARISalign

Figure 3 - ARISalign Discussion Panel

Figure 4 - Use of the Reference Tool on the Whiteboard (Red dots over the chat window represents the eye movements of the modeler who was reading the message including a referential link to the whiteboard.)

Figure 2 - ARISalign Whiteboard Area

Full text

AJIT-e: Online Academic Journal of Information Technology
2018 Yaz/Summer – Cilt/Vol: 9 ‐ Sayı/Num: 33
DOI: 10.5824/1309‐1581.2018.3.001.x
http://www.ajit‐e.org/?menu=pages&p=details_of_article&id=349
Received: 20.04.2018 Editorial Process Begin: 30.07.2018 Published: 05.10.2018
A Comparative Evaluation of Two Computer Supported
Collaborative Work Systems for Supporting Collaborative Business
Process Modeling Activities
Duygu FINDIK COŞKUNÇAY, Ass. Prof. Dr. Atatürk University, Faculty of Economics and Administrative Sciences,
Erzurum, Turkey. Middle East Technical University, Informatics Institute, Ankara, Turkey,
duygu.findik@atauni.edu.tr, ORCID: 0000-0002-8932-5615
Murat Perit ÇAKIR, Ass. Prof. Dr. Middle East Technical University, Informatics Institute, Ankara, Turkey,
perit@metu.edu.tr, ORCID: 0000-0003-4295-1245
ABSTRACT In this study, a comparative evaluation of different Computer-Supported Collaborative Work
(CSCW) environments was conducted to reveal their constraints and affordances for supporting
synchronous collaborative business process modeling (cBPM) activities online. For this purpose,
two case studies were carried out with two CSCW systems that differ in terms of their interaction
design features for supporting joint work. The dual-eye tracking method was employed to
monitor how the participants focused their attention on the shared workspace during cBPM
tasks. An interaction analysis was performed on the communicational content exchanged by the
participants in chat messages and activities performed on the shared working area in light of the
coordination, communication, awareness, group decision-making and team-building aspects of
collaboration. The interaction analysis suggested that the design of the systems significantly
affected the participants’ performance and the interaction quality throughout cBPM. In addition,
a content analysis was conducted to determine the effects of different interaction design
methodologies on the formation of the Business Process Modeling (BP Modeling) phases namely
elicitation, formalization, validation and verification. The interaction design features of the
systems were found to significantly affect the organization of the BP Modeling phases. Based on
the findings, some design suggestions were made to enhance the efficiency of collaboration in
cBPM practices. It is expected that the results of the study will serve as a guideline for system
designers in designing an effective synchronous cBPM tool, and for end users in choosing a
system for their synchronous cBPM practices.
Keywords : business process modeling, collaborative business process modeling, computer supported
modeling, group interaction

AJIT-e: Online Academic Journal of Information Technology
2018 Yaz/Summer – Cilt/Vol: 9 ‐ Sayı/Num: 33
DOI: 10.5824/1309‐1581.2018.3.001.x
http://www.ajit‐e.org/?menu=pages&p=details_of_article&id=349
8
İşbirlikli İş Süreç Modelleme Faaliyetlerini Destekleyen İki
Bilgisayar Destekli İşbirlikli Çalışma Sisteminin Karşılaştırmalı Bir
Değerlendirilmesi
ÖZ Bu çalışmada, bilgisayar destekli işbirlikli iş süreçleri modelleme (cBPM) etkinliklerinin
çevrimiçi olarak desteklenmesi sürecinde kısıtlamalar ve kolaylıkları ortaya koymak amacıyla
farklı Bilgisayar Destekli İşbirliği Çalışma (CSCW) ortamlarının karşılaştırmalı bir
değerlendirmesi yapılmıştır. Bu amaçla, etkileşim tasarım özellikleri açısından farklılık gösteren
iki CSCW sistemi ile ortak çalışmayı destekleyen iki durum çalışması yürütülmüştür.
Katılımcıların dikkatlerini cBPM görevleri sırasında paylaşılan çalışma alanına nasıl
odakladıklarını gözlemlemek için ikili göz izleme yöntemi kullanılmıştır. Katılımcılar tarafından
sohbet mesajları ile gönderilen iletişimsel içerik ve paylaşılan çalışma alanındaki aktiviteler
üzerinde koordinasyon, iletişim, farkındalık, grup olarak karar verebilme ve takım oluşturma
boyutları ışığında bir etkileşim analizi gerçekleştirilmiştir. Etkileşim analizi, sistem
tasarımlarının katılımcıların performansını ve etkileşim kalitesini cBPM boyunca önemli ölçüde
etkilediğini göstermiştir. Ayrıca, farklı etkileşim tasarım metodolojilerinin İş Süreçleri
Modelleme (BP Modelleme) aşamaları olan ortaya çıkarma, biçimlendirme, geçerleme ve
doğrulama üzerine etkilerini belirlemek için bir içerik analizi gerçekleştirilmiştir. Sistemlerin
etkileşim tasarım özelliklerinin, BP Modelleme aşamalarının organizasyonunu önemli ölçüde
etkilediği bulunmuştur. Bulgulara dayanarak, cBPM uygulamalarında işbirliği etkinliğini
artırmak için bazı tasarım önerileri yapılmıştır. Çalışma sonuçlarının, sistem tasarımcılarının
etkin bir eş zamanlı cBPM aracı tasarlamaları ve son kullanıcıların eş zamanlı cBPM
uygulamaları için sistem seçimlerinde bir kılavuz görevi görmesi beklenmektedir.
Anahtar Kelimeler: iş süreci modelleme, işbirlikli iş süreci modelleme, bilgisayar destekli modelleme, grup
etkileşimi
Introduction
In recent years, organizations have started to describe their core procedures in terms of
business processes and invested considerable effort in defining these processes within
operational models (Roser & Bauer, 2005). Business Process Modeling (BP Modeling) is a
collaborative activity that involves a number of stakeholders who possess the fundamental
knowledge of the processes or goals of an organization (Rittgen, 2010). Modeling and
managing collaborative business processes involves new challenges, mainly regarding the
ability to cope with change, decentralization, and the required support for interoperability
(Roser & Bauer, 2005). In overcoming these challenges, BP Modeling activities can potentially
benefit from Computer-Supported Collaborative Work (CSCW) practices. CSCW systems are
primarily based on computer-mediated communication tools, but they provide additional
awareness and coordination features tailored to the needs of group members who work
together to accomplish a particular goal at a specific work setting (Bannon & Schmidt, 1989;
Dourish & Bellotti, 1992). Process modeling is usually performed either in an asynchronous or

İşbirlikli İş Süreç Modelleme Faaliyetlerini Destekleyen İki Bilgisayar Destekli İşbirlikli Çalışma Sisteminin
Karşılaştırmalı Bir Değerlendirilmesi
D. FINDIK COŞKUNÇAY, M. P. ÇAKIR
http://www.ajit‐e.org/?menu=pages&p=details_of_article&id=349
9
synchronous manner with the help of computer-mediated communication tools (Riemer et al.,
2011). Asynchronous modeling is the most commonly used modeling approach in the BP
Modeling domain, where the process is initiated by one user while other users contribute to
the evolving model at a different time and most probably at a different location. Email,
collaborative writing and content management systems are often used for asynchronous
collaboration among stakeholders. In contrast, the synchronous modeling approach allows
stakeholders to engage in modeling at the same time without having to be at the same location.
In this case, communication is usually mediated by teleconferencing, video chat applications,
instant messengers, chat and shared drawing tools, which are based on the exchange of texts
and diagrams among multiple users (Zemel, 2005).
According to Riemer et al. (2011), commercial BP Modeling tools predominantly support the
asynchronous modeling approach. However, certain tools in the market such as ARISalign
(Software AG, 2012) and CoMoMod (Dollmann et al., 2011) use the synchronous process
modeling approach, which supports concurrent modeling activities during collaborative
process modeling. In addition, the Collaborative Cheetah Experimental Platform (cCEP)
developed for academic research provides an environment that supports the synchronous
approach (Forster et al., 2013). Despite using the same modeling approach, each environment
offers different features to facilitate synchronous collaborative work. For example, in some
systems including ARISalign, only one user has the right to edit the model at a time; however,
in others such as cCEP, all the stakeholders can use the whiteboard area concurrently and see
all the changes instantly without having to wait for the other users (Forster et al., 2013).
In this study, the constraints and affordances provided by systems with different CSCW
interaction methodologies for supporting cBPM processes were explored through two case
studies. To analyze the interaction between the participants during the cBPM practices,
publicly available tools ARISalign and VMTChat (Stahl, 2009) were used as a synchronous
CSCW environment. In the VMTChat platform, the stakeholders can use the modeling area
concurrently. An activity performed by a group member can be simultaneously seen by other
members. However, in ARISalign, only the active user, who first accesses the model, has the
right to edit the model and other users have to wait for the active user to complete his editing
on the whiteboard. The activities performed by the active user are temporarily disabled for
other group members when performing the modeling. In addition to the difference in this
interaction design of the shared whiteboard, the communication designs of the two systems
also differ. The VMTChat environment has a chat window placed on the right side of the
shared modeling area. Any message sent by a group member is open to all members. However,
in ARISalign, the participants can communicate using the discussion board, in which they can
initiate new topics and organize their communication. In this system, the discussion board and
shared modeling area are on two different pages. Therefore, messages cannot be seen while
the modeling activities are being performed.
In this study, an interaction analysis was performed on the communicational contents and
modeling activities of the two interaction methodologies in light of the following aspects of
the coordination theory; coordination, communication, awareness, group decision-making

AJIT-e: Online Academic Journal of Information Technology
2018 Yaz/Summer – Cilt/Vol: 9 ‐ Sayı/Num: 33
DOI: 10.5824/1309‐1581.2018.3.001.x
http://www.ajit‐e.org/?menu=pages&p=details_of_article&id=349
10
and team-building (Malone & Crowston, 1994). The examination of the interaction between
the participants based on these aspects demonstrated how different interaction methodologies
can affect the efficacy of interaction among participants and how a system should be designed
and improved to increase the efficiency of cBPM practices. In addition, the current study
aimed to make the cBPM phases of elicitation, formalization, validation and verification more
visible to understand the formation of these phases in different CSCW interaction designs. It
was observed that differences in the interaction design of the system significantly affect the
formation of the BP Modeling phases. At the end of this study, system design
recommendations are made to enhance collaboration in collaborative modeling. The results of
the research can serve as a guideline for system designers in designing a synchronous
collaborative BP Modeling tool, and for customers in choosing a tool for their synchronous
modeling practices. In this study, the following research questions were pursued:
“How different interaction methodologies of CSCW systems affect coordination,
communication, awareness, group decision-making and team building aspects of
collaborative group interactions throughout cBMP?”
“How different interaction methodologies of CSCW systems affect the organization of
elicitation, formalization, validation and verification phases of BP modeling throughout
cBPM?”
This paper is structured as follows: Section 2 presents the theoretical background and provides
an overview of BP Modeling and cBPM. In Sections 3 and 4, the research design, data collection
and data analysis are given, followed by Sections 5 and 6 that present the results, discussion
of the main findings and limitations, respectively.
Theoretical Background
Business process management life cycle consists of design and analysis, configuration,
enactment and evaluation phases (Weske, 2007). This life cycle starts with the design and
analysis phase. The business process design phase has a central role in the identification,
reviewing and representation of business processes (Weske, 2007) and ensures that these
processes are effective and optimized, meet customer requirements, and support and sustain
organizational development and growth (Cousins & Stewart, 2002). The design phase seeks
answers to the following questions; “Who does what, in what sequence, what services or
products are produced and what software systems and data are used to support the process?”
(Davis & Brabander, 2007).
Mauser et al. (2009) suggested that the early phases of a business process design is critical for
ensuring the validity of the outcome of the business processes. BP Modeling and identification
is performed to demonstrate the processes of an enterprise so that the current process can be
analyzed and improved. The early design of the BP Modeling processes consists of three
important phases: elicitation, formalization, validation and verification (Frederiks & Van der
Weide, 2006). Process of modeling is described as “a goal-driven dialogue between a number

İşbirlikli İş Süreç Modelleme Faaliyetlerini Destekleyen İki Bilgisayar Destekli İşbirlikli Çalışma Sisteminin
Karşılaştırmalı Bir Değerlendirilmesi
D. FINDIK COŞKUNÇAY, M. P. ÇAKIR
http://www.ajit‐e.org/?menu=pages&p=details_of_article&id=349
11
of participants who communicate with each other and build their discussion on what has been
discussed before. The participating roles are either domain experts who generate and validate
statements about the domain, or system analysts who create and verify formal models”
(Hoppenbrouwers et al., 2005). This shows that BP Modeling is a multi-stakeholder activity in
which the team members jointly discuss, design and document all business processes (Riemer
et al., 2011).
Communication, coordination, awareness, group decision-making and team-building each
play an important role in process modeling (Riemer et al., 2011). Providing software support
for these activities, CSCW tools are essential in terms of facilitating group interactions at a
distance (Malone & Crowston, 1994; Salas et al., 2008). Team members need to communicate
with each other to explain their changes, coordinate the modeling activities and agree on terms
and definitions. Throughout the process, stakeholders should be able to monitor the changes
made by other team members and access information about the time, author and type of
change that has been made. Different team members assume different roles; someone is in
charge of the modeling whereas others deliver information or review the model. Each team
member contributes to the joint modeling activities according to their respective roles. Using
the CSCW tools, these cBPM activities can be easily and effectively performed (Riemer et al.,
2011).
In recent years, cBPM has attracted the attention of several researchers, who investigated
cBPM from different perspectives. For instance, Baghaei et al. (2007) evaluated the
effectiveness of the COLLECT-UML tool and showed that the use of collaborative systems
improved students’ domain knowledge. Basheri (2010) examined computer-supported
collaborative learning environments such as COLER, COLLECT-UML, CoLeMo and AUTO-
COLLEAGUE using the Unified Modeling Language (UML). The study examined the type
and frequency of contributions of students in the chat system and evaluated these systems in
terms of their support for interactive learning. Similarly, Riemer et al. (2011) investigated the
collaborative nature of several existing BP Modeling tools; CA ERwin Process Modeler, ARIS
Design Platform 7, Enterprise Architect 8, iGrafx Process Modeler 2011, Microsoft Visio 2010,
Business Modeler Advanced 7, Signavio Process Editor, BONAPART Collaborative, Adonis,
Savvion Process Manager and Innovator for Business Analysts. These development tools were
evaluated using three types of criteria; process modeling, collaboration and technical. Based
on the results, the researchers concluded that tool designers perceive modeling as
predominantly asynchronous and none of the products allows synchronous modeling on the
same object. Hogrebe et al. (2011) performed an eye tracking experiment for the measurement
and assessment of user satisfaction with modeling languages such as the extended Event-
driven Process Chain (eEPC) and the oriented Event-driven Process Chain (oEPC). The
authors stated that the most important requirements for the modeling language in BP
Modeling are; being ‘complete’, ‘easy-to-understand’ and ‘easy-to-use’. Mendling et al. (2012)
proposed a collaborative process modeling framework evaluating the following three BP
Modeling tools; Collaborative Modeling Architecture (COMA), Signavio Process Editor and
Software AG ARISalign & ARIS Community in terms of their collaboration aspects, namely
awareness, communication, coordination, group decision making and team and community

Frequently asked questions

Q1. What are the contributions mentioned in the paper "A comparative evaluation of two computer supported collaborative work systems for supporting collaborative business process modeling activities" ?

In this study, a comparative evaluation of different Computer-Supported Collaborative Work ( CSCW ) environments was conducted to reveal their constraints and affordances for supporting synchronous collaborative business process modeling ( cBPM ) activities online. For this purpose, two case studies were carried out with two CSCW systems that differ in terms of their interaction design features for supporting joint work. It is expected that the results of the study will serve as a guideline for system designers in designing an effective synchronous cBPM tool, and for end users in choosing a system for their synchronous cBPM practices. The interaction analysis suggested that the design of the systems significantly affected the participants ’ performance and the interaction quality throughout cBPM. Based on the findings, some design suggestions were made to enhance the efficiency of collaboration in cBPM practices.