Journal of
.
TECHNICAL WRITING
AND COMMUNICATION
Baywood Publishing Company, Inc.
26 Austin Avenue, Amityville, NY 11701
call (516) 691-1270 fax (516) 691-1770 orderline (800) 638-7819
e-mail: baywood@baywood.com web site: http://baywood.com
Editor
Charles H. Sides
Consulting Editor
David L. Carson
Founder and Consulting
Editor
Jay R. Gould
Book Review Editor
Donald H. Cunningham
Executive Board
Paul V. Anderson
Virginia A. Book
Thomas C. Dixon
John S. Harris
Frederick T. Kiley
John Kirkman
Frederick C. Mish
Thomas E. Pearsall
Janice C. Redish
W. Keats Sparrow
Editorial Board
David B. Bradford
Joseph E. Harmon
Randy Allen Harris
Carel J. M. Jansen
John Nelson
Mary Elizabeth Raven
Marguerite K. Tsou
Jan H. Spyridakis
Dorota Zielinska
J. EDUCATIONAL COMPUTING RESEARCH, Vol. 17(4) 397-431,1997
AIMS & SCOPE
The Journal of Technical Writing and
Communication has served for over twenty-
four years as a major professional and scholarly
journal for practitioners and teachers of the
most functional forms of communication-
excluding fiction. The Journal welcomes
articles, both beginning and established
authors, but insists that all authors hew to
the best standards of technical communication
by submitting active, clear, concisely coherent,
audience-oriented prose.
This rigorously peer-refereed Journal
welcomes articles both theoretical and
practical, on professional writing; pedagogy,
communication, rhetorical, linguistic, infor-
mation, visual, ethnographic, reading, and
textual theory; technical journalism, rhetoric;
communication management; organizational,
business, intercultural, visual, and engineering
or scientific communication; user documen-
tation; audience analysis; hardware and soft-
ware documentation; CAI,CAD/CAM; desktop
publishing, on-line documentation; word
processing and on a variety of subjects drawn
from fields related to technical communication.
ANALYSIS OF A GLOBAL ONLINE DEBATE AND THE
DEVELOPMENT OF AN INTERACTION ANALYSIS MODEL
FOR EXAMINING SOCIAL CONSTRUCTION OF
KNOWLEDGE IN COMPUTER CONFERENCING
CHARLOTTE N. GUNAWARDENA, PH.D.
CONSTANCE A. LOWE, M.A.
University of New Mexico
TERRY ANDERSON, PH.D.
University of Alberta
ABSTRACT
This study attempts to find appropriate interaction analysis/content analysis
techniques that assist in examining the negotiation of meaning and
co-construction of knowledge in collaborative learning environments facili-
tated by computer conferencing. The authors review strengths and short-
comings of existing interaction analysis techniques and propose a new model
based on grounded theory building for analyzing the quality of CMC inter-
actions and learning experiences. This new Interaction Analysis Model for
Examining Social Construction of Knowledge in Computer Conferencing
was developed after proposing a new definition of "interaction" for the CMC
context and after analyzing interactions that occurred in a Global Online
Debate. The application of the new model for analysis of collaborative con-
struction of knowledge in the online debate and in a subsequent computer
conference are discussed and future research suggested.
Subscription Information:
Price per volume-4 issues yearly. Institutional
Rate: $143.00, Individual Rate: $42.00.
Postage: $6.50 postage and handling in the U.S.
and Canada, $11.75 elsewhere. ISSN: 004-2816
Complimentary sample issue available
INTRODUCTION
The exchange of messages among a group of participants by means of networked
computers, for the purpose of discussing a topic of mutual interest, is referred
to as computer-mediated conferencing or computer conferencing. Computer-
mediated conferencing is presently being employed with greater and greater
frequency as an environment for collaborative learning. However, the utilization
of the medium in education has in many respects outstripped the development of
397
@
1997, Baywood Publishing Co., Inc.
L...
398 I GUNAWARDENA, LOWE AND ANDERSON
ONLINE DEBATE/INTERACTION ANALYSIS MODEL FOR CMC I 399
theory on which to base such utilization. One significant q~estion. which. has
not yet been satisfactorily answered is how to assess the qua~ity of mteractI~ns
and the quality of the learning experience in a computer-mediated conferencmg
environment.
Questions that are often asked in the assessment. o~
evalua~ion ~f
co~puter
conferences related to participation patterns and participant satisfactiOn with the
conference have been answered fairly successfully using several methods.
Among them are participation analysis techniques which analy~e the capacity
of a conference to engage members or which analyze comparative patterns of
participation among learners from varying backgrounds .[1, 2]. Partici~ants' own
reports of learning or satisfaction with the learning expenence are also important;
these may be studied as found in the transcript of a conference or by means of
online or paper surveys. However, while participation analysis techniques ~ave
value in determining who participated, how actively, and for how long, neither
quantitative analysis of participation nor reports of part~cular satis~action yield
information on the construction of knowledge or the quahty of learnmg that took
place in a computer conference. In order to assess the quality .of interactions ~nd
the quality of the learning experience in a computer mediated confer~ncm.g
environment, content analysis or interaction analysis of computer transcnpts is
essential.
Mason, in her review of methodologies for evaluating computer conferencing,
notes that most research stops with quantitative analyses based on number of
messages sent, and by whom, or on frequency of logons, or on message maps
showing numbers of replies and message chains [3]. Many studies used surveys,
interviews, case studies, empirical experimentation, and statistical measurements
to evaluate computer conferencing, but these do not shed much light on the
quality of learning taking place. Mason urges researchers to take up the m~re
challenging methodology of content analysis in order to answer more. crucial
questions related to the quality of learning and knowledge constructiOn that
occurs through social negotiation in CMC.
2. Did individual participants change their understanding or create new per-
sonal constructions of knowledge as a result of interactions within the
group?
The article will examine the definition of "interaction" in a computer-mediated
communication (CMC) environment as definitions of "interaction" for interper-
sonal communication used by communication researchers to study face-to-face
dialogue do not transfer well to the CMC context. The article will point out the
strengths and shortcomings of interaction analysis techniques that have been
developed and will propose a model based on grounded theory building [4] for
analyzing the quality of CMC interactions and learning experiences. The inter-
action analysis model will be developed by analyzing the interactions that
occurred in a global online debate conducted through computer conferencing; the
authors contend that the debate forms a particularly good example of collaborative
construction of knowledge through social negotiation, a key feature of a con-
structivist learning environment [5]. The application of this model for analyzing
the global online debate as well as another professional development computer
conference will be discussed.
The detailed examination of transcripts provides both theoretical and practical
insight into the learning context and its outcomes. For many students, teachers,
and researchers, text based, asynchronous interaction is a novel environment,
with only short developmental history upon which to base quality assessment.
Techniques and systems developed in this article are critically important to
developing theoretical understanding of what occurs during the learning process
and as importantly they provide tools by"which practitioners can assess and then
modify the learning sequences and activities they facilitate.
THE INTERNATIONAL ONLINE DEBATE
PURPOSE
The online debate took place during the week of June 5-11, 1995, and formed
part of ICDE95 Online, a virtual pre-conference to the XVI World Conference of
the International Council on Distance Education (ICDE) held in Birmingham,
England. The online pre-conference provided an opportunity for those who could
not attend the conference to discuss by CMC issues that would be addressed
during the Birmingham conference [6].
A major goal of the ICDE95 online conference was to demonstrate and develop
effective learning activities which support quality virtual conferences. The
leaders of each session were responsible for carrying out the selected learning
activity in such a way as to maintain interest and participation throughout one
week. The learning activity selected for the first conference session was the
online debate, one of the first experiments in running a debate across international
time lines on the Internet. Terry Anderson, one author of the present article, was
overall moderator and technical coordinator for the entire conference; authors
The purpose of this article is to critically examine interaction analysis tech-
niques that have been developed for the analysis of computer conferences ~nd
determine which techniques work best in a given context to address specific
research questions. This study is interested in finding appropriate interaction
analysis techniques that help address the following two evaluation research [4]
questions:
1. Was knowledge constructed within the group by means of the exchanges
among participants? and
400 I GUNAWARDENA. LOWE AND ANDERSON
ONLINE DEBATE/INTERACTION ANALYSIS MODEL FOR CMC 1401
Gunawardena and Lowe designed the debate with graduate students at the
University of New Mexico and led the affirmative debate team. The ICDE95
online debate is hereafter referred to as "the debate."
The debate design invited the 554 list subscribers to participate on either the
affirmative or the negative side of a statement presented by the debate leaders.
Each team was headed by a leader and each day's contributions were reviewed
at the end of the day by a second team member, the "summarizer." In addition
to the authors, other team leaders and summarizers were located at the George
Washington University at Washington DC, and the Pennsylvania State University
at University Park, Pennsylvania. One major challenge of this project was to
design a debate which allowed equal opportunities for participants to contribute
even though they were located across international time lines. While asyn-
chronous CMC is a good medium for this kind of activity, the debate format
requires adherence to time lines if it is to function as a debate. The debate
designers adopted a structure which was divided into days measured according to
Greenwich Mean Time (GMT).
On the day before the debate began, the topic for the debate, the definition of
the topic, and the rules and format of the debate were posted to list subscribers.
The schedule on which arguments were to be posted was as follows:
Saturday, June 10, 00:01-23:00 GMT-Negative Restatement. Those who
o~pose the proposition were asked to answer the arguments raised the pre-
VIOUSday by the Affirmative Team and restate their case. Between 23:00
GMT and midnight, a summary of the day's arguments were posted.
Sunday, June 11,00:01-23:00 GMT-Volunteer judges were invited to discuss
the outcome of this debate.
The adherence to GMT was emphasized in the rules posted in order to be sure
that all participants, wherever they were located in the world, would be able to
take their turns during the twenty-four hour time periods reserved for their chosen
teams. All participants were asked to use the following subject lines when they
participated in the debate:
Monday, June 5, 00:01-23 :00 GMT-First Affirmative posted by Team Leader.
From the time this statement appeared until 23:00 GMT, everyone
who wished to argue in favor of the proposition (statement) was asked
to add their comments. Between 23:00 GMT and midnight, a summary
of the day's arguments was posted by the summarizer for the Affirmative
side.
Tuesday, June 6, 00:01-23:00 GMT-First Negative posted by Team Leader.
From the time this statement appeared until 23:00 GMT, everyone who
wished to argue against the proposition (statement) was asked to add their
comments. Between 23:00 GMT and midnight, a summary of the day's
arguments was posted by the summarizer for the Negative side.
Wednesday, June 7, 00:01-23:00 GMT-Affirmative Rebuttal. Those who
favor the proposition were asked to argue against the statements made the
previous day by the Negative Team. Between 23:00 GMT and midnight, a
summary of the day's arguments were posted.
Thursday, June 8, 00:01-23:00 GMT-Negative Rebuttal. Those who opposed
the proposition were asked to argue against the statements made on Monday
and Wednesday by the Affirmative Team. Between 23:00 GMT and mid-
night, a summary of the day's arguments were posted.
Friday, June 9, 00:01-23:00 GMT-Affirmative Restatement. Those who favor
the proposition were asked to answer the arguments raised the previous day
by the Negative Team and restate their case. Between 23:00 GMT and
midnight, a summary of the day's arguments were posted.
1. "Interaction Affirmative" for arguments in favor of the proposition.
2. "Interaction Negative" for arguments opposing the proposition.
The topic chosen for the debate focused on a controversial issue in current
research in distance education, the role and importance of "interaction" in
effective distance education. In order to maximize the difference in opinion
rep~esented by the positive and negative sides of the debate, this controversy was
delIberately reduced to the extreme statement: "No Interaction, No Education,"
representing the assertion that true distance education is impossible without
provision for interaction. Detailed discussion of the design of the debate is found
in Gunawardena, Lowe, and Anderson [7]. The debate transcript is archived in
the World Wide Web at (http://www.ualberta.cal-tanderso/icde95/interaction
wwwn.
-
The primary tool used to support ICDE95 was an unmoderated, open Listserv
[8] mailing list maintained at the University of Alberta. List participants repre-
sented approximately thirty-five countries. Most participants in the debate were
practicing specialists and advanced students in the field of distance education.
Due to the characteristics of the participants, who were predominantly profes-
sionals in the field of distance education, and the sharp focus of the conference
which resulted from use of the debate format, the conference offers a particularly
good example of the use of the computer conferencing medium in the co-creation
of knowledge.
THEORETICAL FRAMEWORK FOR EVALUATION OF
QUALITY IN COMPUTER CONFERENCING
A number of models for the evaluation of quality in computer conferencing are
available. Hiltz describes analysis of computer conferences along four dimen-
sions: 1) characteristics inherent to the technology, 2) social and psychological
characteristics of users, 3) characteristics of groups adopting the technology, and
4) interaction of the preceding factors [2]. Levin, Kim, and Riel describe
a
l
402 I GUNAWARDENA, LOWE AND ANDERSON
ONLINE DEBATE/INTERACTION ANALYSIS MODEL FOR CMC I 403
method of analyzing the structure and content of interactions by the creation of
"message maps" which display graphically the interrelationships among the mes-
sages submitted to a conference [1]. Levin and colleagues use this analysis to
identify "threads" within a conference and to display the "multithreaded" nature
of conference interaction [1]. They also practice identifying messages which are
particularly "influential" in producing numerous responses or lengthy sequences
of responses and they diagram message flow described as the ebbing or flowing
volume of messages in the conference. Henri proposes a system of content
analysis which involves breaking messages down into units of meaning and
classifying these units according to their content [9]. Henri includes a quasi-
quantitative "participative" dimension of analysis in her scheme for content
analysis which the authors feel is more properly considered as a separate issue
from the more qualitative analysis of message meaning units. Henri's other four
broad categories of content are described as 1) content which reflects the social
dimension of conference interchanges, 2) content relating to the interactive
dimension of the conference, 3) content indicating the a~plication of cognitive
skills, and 4) content showing metacognitive skills. Newman and colleagues [10,
11], in an attempt to study the quality of the learning experience in a computer
conference, have applied Henri's model [9] and Garrison's model of critical
thinking [12] to develop a content analysis method to measure critical thinking in
face-to-face and computer supported group learning. They observe that the stages
listed in Garrison's description of critical thinking as a five-stage process cor-
respond closely to the cognitive skills Henri recognizes as important to the
cognitive dimension of CMC.
These models serve as a useful starting point for analyzing CMC interactions.
However, they are not very specific on how to evaluate the process of knowledge
construction that occurs through social negotiation in CMC. Moreover, the defini-
tions of interaction these models present are either unclear or not very applicable
to the pattern of interaction observed in the debate.
a teacher or group of experts. Interaction analysis models that have been
developed to analyze online class interaction in a "teacher-centered" learning
environment may not be very appropriate, or may have to be extensively adapted,
for analyzing the interaction that occurs in professional development experiences
of this kind.
The following factors had to be kept in mind as we approached the analysis of
the debate transcript:
ISSUES IN THE ANALYSIS OF THE DEBATE
.The debate format described earlier imposed an organizational structure
which influenced the interactions. Since the debate format required par-
ticipants to take sides on an issue, those who wanted to argue on middle
ground found it difficult to fit their statements into either the affirmative or
negative category and to decide on which days they should post their argu-
ments. A related problem was that, due to technical transmission delays,
some of the messages did not get posted on the days they were sent.
.
A predominant influence of the debate format could be seen in the way it
affected the co-construction of knowledge. While the format supported well
the discussion of inconsistency among ideas, it kept the participants from
negotiating meaning to reach a compromise. The debate leaders, in the spirit
of competition appropriate to a debate, tried to keep their teams focused on
winning the argument and discouraged the group from trying to achieve a
consensus or compromise.
.Determining a unit of analysis was also a problem with this format. Par-
ticipants often apparently felt that they had to put forth several arguments to
prove their point and, therefore, each message was likely to include several
arguments which advanced the case. Consequently, separating a message into
meaning units following the Henri model [9] was difficult.
.The majority of messages in the debate were very task oriented, as it was a
highly structured activity for a period of one week. The debate lacked the
socialization element that is usually characteristic of many computer con-
ferences. This may have discouraged some participants from contributing.
The online debate was designed as an adult professional development experi-
ence and participants were either practicing professionals in the field of distance
education or graduate students conducting research in the field. The participants
could be described as a group of professionals of roughly equal stature coming
together to contribute their knowledge, negotiate meaning, and come to an under-
standing about an important issue in the theory and practice of distance education.
Therefore, the interaction that occurred among the participants could be described
as a collaborative construction of knowledge through social negotiation, or a
constructivist learning experience [5], rather than a one-way dissemination of
information from an expert group to a novice group. The dynamics of this group
of equal participants were also very different from the dynamics of a class led by
ANALYSIS OF THE DEBATE TRANSCRIPT BASED ON
PREVIOUS INTERACTION ANALYSIS MODELS
Jordan and Henderson describe interaction analysis as an interdisciplinary method
of investigating the interaction of human beings with each other and with objects
in their environment [13]. Quoting Garfinkel, they observe that interaction-
analytic studies see learning as a distributed, ongoing social process, in which
evidence that learning is occurring or has occurred must be found in under-
standing the ways in which people collaboratively do learning and recognize
learning as having occurred. Fortunately, a computer transcript provides the kind
!
404 I GUNAWARDENA.
LOWE AND ANDERSON
ONLINE DEBATE/INTERACTION ANALYSIS MODEL FOR CMC I 405
of data corpus that allows the close scrutiny for interaction analysis. Interaction
analysis employs content analysis techniques and focuses on studying the interac-
tions that took place between participants. Jordan and Henderson point out the
difficulty of describing interaction analysis and note that it is best learned by
doing, usually in a dyad or in a group.
The methodology adopted in developing a framework for analyzing the quality
of the learning experience of the debate included several stages. The first stage
was a critical review of currently available interaction analysis models and their
definitions of interaction and interaction analysis. Next, the applicability of cur-
rently available models for the analysis of the debate was tested. Then, in order to
overcome the shortcomings seen in the application of previous models, a new
definition of interaction was put forth and the learning environment that emerged
in the process of the debate was described. This was followed by the analysis of
the debate transcript to examine emerging patterns, themes, and phases related to
the social construction of knowledge. Finally, an interaction analysis model was
developed for analyzing the themes, patterns, and phases that emerged from the
debate. Thereafter, the model was applied to the analysis of the debate itself.
Based upon the review of models discussed in the above section, and other
published models for analyzing interactions in computer conferences [3], the
authors selected Henri's [9] model as the most promising starting point for the
interaction analysis of the debate transcript. The authors agreed with Henri that
computer conferencing is characterized by exceptional "richness and efficiency"
and that examination of the actual content of messages is the appropriate means
of evaluating whether or not the learning experience has made full use of the
potential of the medium. A decision was made to focus the content analysis of
the debate transcript on meaning units which Henri would describe as having
cognitive or metacognitive content.
One important aspect of the Henri [9] model which the authors chose not to
examine in evaluating the debate was the "social" content of conference mes-
sages. Henri describes social message content as "Statement or part of statement
not related to formal content of subject matter." Examples cited by Henri include
statements of self-introduction or mutual support among learners. The authors
agree that such statements are important in establishing social presence, building
rapport, and promoting the growth of community, especially in the construction
of a learning environment which is meant to join a set group of learners for an
extended period of time. However, the structured debate format did not lend itself
to social interaction and kept the participants task-oriented for a period of one
week. Thus, while it is important to analyze the social dimension in other con-
ferences, the authors decided it was not appropriate in the context of the debate.
Of the five dimensions for evaluation proposed in the Henri [9] model, the
authors felt that the participative dimension, which Henri defines as the compila-
tion of the number of messages or statements transmitted by one person or group,
should be studied separately from the fundamentally qualitative analysis of
message content because this type of analysis does not shed light on the quality of
the learning experience. Since for the reasons noted above the social dimension of
Henri's model was also set aside, the content analysis of the transcript focused on
the remaining three aspects of the model: content relating to the interactive
dimension of the conference, content indicating the application of cognitive
skills, and content showing metacognitive skills.
One of the first steps in conducting the interaction analysis was to decide on
the unit of analysis. Henri suggests dividing messages into "units of meaning"
because a message may contain more than one idea [9]. The debate transcript was
thus cut up into units of meaning (sometimes one statement and at other times
one or two paragraphs in a message). This was a very difficult process as we felt
that cutting up a message into units did not capture the essence of meaning
expressed in that message. The units of meaning were then scored on Henri's
three dimensions selected for this study: interactive, cognitive, and metacogni-
tive. With respect to the interactive dimension, a message map was created
showing the extent to which messages were interrelated. Here, a message was
considered the unit of analysis rather than a "unit of meaning," as defined by
Henri, because using a "unit of meaning" to determine interaction patterns
became very complicated. Instances of cognitive processing, as expressed in
"units of meaning" were coded following Henri's definitions for surface level and
in-depth processing. A separate analysis was done of the amount of metacognitive
knowledge and skills that appeared in message units. As the content analysis of
the debate transcript progressed, it rapidly became clear that many aspects
of Henri's model were unsuited for application to the debate.
The first shortcoming the authors found in the Henri model [9] as applied to the
debate is that, while Henri notes the advantages of CMC for collaborative work,
the model still appears to be based on a teacher-centered instructional paradigm.
For example, Henri states that ". . . the educator can offer input at three levels:
what is said on the subject or theme under discussion; how it is said; and the
processes and strategies adopted in dealing with it . . . The educator may favor
one or another level, according to his or her pedagogical aims and intentions"
[9, p. 123]. The paradigm Henri describes here is clearly one widely applied,
as educators new to distance education try to recreate the familiar patterns of
traditional teaching in a new medium. However, it is also clear that such a
paradigm is inappropriate for analyzing voluntary and informal continuing
professional education, as represented in the debate and other online discussions
such as e-mail lists and Usenet groups which feature exchange of views among
adult professionals. Therefore, in the analysis of Henri's three dimensions
selected for this study, a move from a teacher-centered view of learning to
constructivistic conceptions of learning based on shared construction of knowl-
edge seems more appropriate. In the debate, the objective was to evaluate the
learning process taking place among the group of participants, rather than to
assess individual student performance.
"
.~