1
A Constraint-based Collaborative Environment for
Learning UML Class Diagrams
Nilufar BAGHAEI and Antonija MITROVIC
Intelligent Computer Tutoring Group
Department of Computer Science and Software Engineering
University of Canterbury, Private Bag 4800, New Zealand
{n.baghaei, tanja}@cosc.canterbury.ac.nz
Abstract. COLLECT-UML is a constraint-based ITS that teaches object-
oriented design using Unified Modelling Language (UML). UML is easily the
most popular object-oriented modelling technology in current practice. We
started by developing a single-user ITS that supported students in learning UML
class diagrams. The system was evaluated in a real classroom, and the results
show that students’ performance increased significantly. In this paper, we
present our experiences in extending the system to provide support for
collaboration. We present the architecture, interface and support for
collaboration in the new, multi-user system. A full evaluation study has been
planned, the goal of which is to evaluate the effect of using the system on
students’ learning and collaboration.
1. Introduction
E-learning is becoming an increasingly popular educational paradigm as more
individuals who are working or are geographically isolated seek higher education.
As such students do not meet face to face with their peers and teachers, the support
for collaboration becomes extremely important [8]. Effective collaborative learning
includes both learning to effectively collaborate, and collaborate effectively to learn,
and therefore a collaborative system must be able to address collaboration issues as
well as task-oriented issues [17].
In the last decade, many researchers have contributed to the development of
CSCL and advantages of collaborative learning over individualised learning have
been identified [14]. Some particular benefits of collaborative problem-solving
include: encouraging students to verbalise their thinking; encouraging students to
work together, ask questions, explain and justify their opinions; increasing students’
responsibility for their own learning; increasing the possibility of students solving or
examining problems in a variety of ways; and encouraging them to articulate their
reasoning, and elaborate and reflect upon their knowledge [24, 27]. These benefits,
however, are only achieved by active and well-functioning learning teams [15].
Numerous systems for collaborative learning have been developed; however, the
concept of supporting peer-to-peer interaction in CSCL systems is still in its infancy.
Various strategies for computationally supporting online collaborative learning have
2
been proposed and used, while more studies are needed that test the utility of these
techniques [17].
This paper describes an Intelligent Tutoring System (ITS) that uses Constraint-
Based Modeling (CBM) approach to support both problem-solving and collaborative
learning. CBM has been used successfully in several tutors supporting individual
learning [20]. We have developed COLLECT-UML [2, 3], a single-user version of a
constraint-based ITS, that teaches UML class diagrams. In this paper, we describe
extensions to this tutor, which support multiple students solving problems
collaboratively. We start with a brief overview of related work in Section 2. Section
3 then presents COLLECT-UML and the evaluation study conducted with second-
year university students taking a course in Introduction to Software Engineering.
Section 4 describes the design and implementation of the collaborative interface as
well as the system’s architecture. Section 5 presents the collaborative model, which
has been implemented as a set of meta-constraints. Conclusions are given in the last
section.
2. Related Work
Three categories of CSCL systems can be distinguished in the context of the
collaboration support [1, 17]. The first category includes systems that reflect actions;
the basic level of support a system may offer involves making the students aware of
the participants’ actions. The systems in the second category monitor the state of
interactions; some of them aggregate the interaction data into a set of high-level
indicators, and display them to the participants (e.g. Sharlock II [21]), while others
internally compare the current state of interaction to a model of ideal interaction, but
do not reveal this information to the users (e.g. EPSILON [25]). In the latter case,
this information is either intended to be used later by a coaching agent, or analysed
by researchers in order to understand the interaction [17]. Finally, the third class of
systems offer advice on collaboration. The coach in these systems plays a role
similar to that of a teacher in a collaborative learning classroom. The systems can be
distinguished by the nature of the information in their models, and whether they
provide feedback on strictly collaboration issues or both social and task-oriented
issues. Examples of the systems focusing on the social aspects include Group Leader
Tutor [19] and DEGREE [6], and an example of the systems addressing both social
and task-oriented aspects of group learning is COLER [7].
Although many tutorials, textbooks and other resources on UML are available,
we are not aware of any attempt at developing a CSCL environment for UML
modelling. However, there has been an attempt [25] at developing a collaborative
learning environment for OO design problems using Object Modeling Technique
(OMT) – a precursor of UML. The system monitors group members’
communication patterns and problem solving actions in order to identify situations
in which students effectively share new knowledge with their peers while solving
OO design problems. The system first logs data describing the students’ speech acts
(e.g. Request Opinion, Suggest, and Apologise) and actions (e.g. Student 3 created a
new class). It then collects examples of effective and ineffective knowledge sharing,
and constructs two Hidden Markov Models which describe the students’ interaction
in these two cases. A knowledge sharing example is considered effective if one or
more students learn the newly shared knowledge (as shown by a difference in pre-
post test performance), and ineffective otherwise. The system dynamically assesses
3
a group’s interaction in the context of the constructed models, and determines when
and why the students are having trouble learning new concepts they share with each
other. The system does not evaluate the OMT diagrams and an instructor or
intelligent coach’s assistance is needed in mediating group knowledge sharing
activities. In this regard, even though the system is effective as a collaboration tool,
it would probably not be an effective teaching system for a group of novices with
the same level of expertise, as it could be common for a group of students to agree
on the same flawed argument.
CBM has been used successfully in several tutors supporting individual
learning. The main contribution of this research is the use of CBM technique to
support collaborative learning. The system provides feedback on both collaboration
issues (using the collaboration model, represented as a set of meta-constraints) and
task-oriented issues (using the domain model, represented as a set of syntax and
semantic constraints). CBM is also used to model student and group knowledge.
3. COLLECT-U
UU
UM
MM
ML
LL
L: Single-User Version
COLLECT-UML is a problem-solving environment, in which students construct
UML class diagrams that satisfy a given set of requirements. It assists students
during problem-solving, and guides them towards a correct solution by providing
feedback. The feedback is tailored towards each student depending on his/her
knowledge. COLLECT-UML is designed as a complement to classroom teaching and
when providing assistance, it assumes that the students are already familiar with the
fundamentals of UML. For details on system’s architecture, functionality and the
interface refer to [2, 3]; here we present only the basic features of the system.
At the beginning of interaction, a student is required to enter his/her name,
which is necessary in order to establish a session. The session manager requires the
student modeller to retrieve the model for the student, if there is one, or to create a
new model for a new student. Each action a student performs is sent to the session
manager, as it has to link it to the appropriate session and store it in the student’s
log. Then, the action is sent to the pedagogical module. If the submitted action is a
solution to the current problem, the student modeller diagnoses the solution, updates
the student model, and sends the result of the diagnosis back to the pedagogical
module, which generates appropriate feedback.
COLLECT-UML contains an ideal solution for each problem, which is compared
to the student’s solution according to the system’s domain model, represented as a
set of constraints [22]. The system’s domain model contains 133 constraints that
describe the basic principles of the domain. In order to develop constraints, we
studied material in textbooks, such as [12], and also used our own experience in
teaching UML and OO analysis and design.
Figure 1 illustrates a constraint from the UML domain. The relevance condition
identifies a relationship of type aggregation in the ideal solution, and then checks
whether the student’s solution contains the same type of relationship, or a
relationship of a different kind with the same name. The student’s solution is correct
if the satisfaction condition is met, when the matching relationship is of the same
type (i.e. aggregation). The constraint also contains a message which would be given
to the student if the constraint is violated. The last two elements of the constraint
specify that it covers some aspects of relationships, and also identifies the
relationship and the classes to which the constraint was applied.
4
Figure 1. An example constraint
We performed an evaluation study [3] in May 2005 with 38 students enrolled in
a Software Engineering course. The students learnt UML modelling concepts during
two weeks of lectures/tutorials. The study was conducted in two streams of two-hour
laboratory sessions. Each participant sat a pre-test, interacted with the system, and
then sat a post-test and filled a user questionnaire. The pre-test and post-test each
contained four multiple-choice questions, followed by a question where the students
were asked to design a simple UML class diagram. Table 1 presents some general
statistics about the study. The average mark on the post-test was significantly higher
than the pre-test mark (t = 2.71, p = 4.33E-08). The students spent on average 90
minutes interacting with the system.
Table 1. Some statistics about the study
Average s. d.
Attempted problems 5.71 2.59
Solved problems 47% 33%
Attempts per problem 7.42 4.76
Pre-test 52% 21%
Post-test 76% 17%
We also analyzed the log files, in order to identify how students learn the
underlying domain concepts. Figure 2 illustrates the probability of violating a
constraint plotted against the occasion number for which it was relevant, averaged
over all constraints and all participants. The data points show a regular decrease,
which is approximated by a power curve with a close fit of 0.93, thus showing that
students do learn constraints over time. The probability of violating a constraint on
the first occasion of application is halved by the tenth occasion, showing the effects
of learning.
Students were offered individualised feedback on their solutions upon
submission. The mean rating for the usefulness of feedback was 2.8. 67% of the
participants had indicated that they would have liked to see more details in the
feedback messages. The comments we received on open questions pointed out
several features of the system, which can be improved.
The results showed that COLLECT-UML is an effective learning environment.
The participants achieved significantly higher scores on the post-test, suggesting that
they acquired more knowledge in UML modelling. The learning curves also prove
that students do learn constraints during problem solving. Subjective evaluation
shows that most of the students felt spending more time with the system would have
resulted in more learning and that they found the system to be easy to use.
(78
"Check the type of your relationships. You need to use aggregations between some of
your classes."
(and (match IS RELATIONSHIPS (?* "@" ?rel_tag "aggregation" ?c1_tag ?c2_tag ?*))
(or-p (match SS RELATIONSHIPS (?* "@" ?rel_tag ?type ?c1_tag ?c2_tag ?*))
(match SS RELATIONSHIPS (?* "@" ?rel_tag ?type ?c2_tag ?c1_tag ?*))))
(test SS ("aggregation" ?type))
"relationships"
(?rel_tag ?c1_tag ?c2_tag))
5
Figure 2. Probability of constraint violation
4. COLLECT-U
UU
UM
MM
ML
LL
L: Multi-User Version
The collaborative version of COLLECT-UML is designed for sessions in which
students first solve problems individually and then join into small groups to create
group solutions. The system’s architecture is illustrated in Figure 3. The application
server consists of a session manager that manages sessions and student logs, a
student modeller that creates and maintains student models for individual users, a
domain model (i.e. the constraint set), a pedagogical module and a group modeller.
The system is implemented in Allegro Common Lisp.
The interface is shown in Figure 4. The problem description pane presents a
design problem. Students construct their individual solutions in the private
workspace (right), and use the shared workspace (left) to collaborate while
communicating via the chat window (bottom). The private workspace enables
students to try their own solutions and think about the problem before start
discussing it in the group. The group area is initially disabled. When all of the
students indicate readiness to work in the group by clicking on Join the Group
button, the shared workspace is activated. The students select the components’
names from the problem text. The Group Members panel shows the team-mates
already connected. Only one student, the one who has the pen, can update the shared
workspace at a given time. Additionally, this panel shows the name of the student
who has the control of this area and the students waiting for a turn.
A recent study [23] defines relevant characteristics of good collaboration and
the authors have considered turn-taking as one of those characteristics. According to
their results, explicitly handing over a turn can be a good way of compensating for
the limited communication channel. An implication of providing such protocol is
that deadlocks can be created in cases where one partner cannot proceed with