Assessment process for a simulation-based training environment in global software development
21 Jun 2014-pp 231-236
TL;DR: An assessment process based on educational theory adapted to the authors' simulation-based training environment is presented and results indicate that the assessment method yields meaningful results and proof of learning.
Abstract: Simulation has been applied in several Software Engineering fields, and is shown to be a useful method in industrial training. As part of our research work, we have used simulation to provide training in Global Software Development (GSD). We have developed a platform to strengthen GSD skills by simulating realistic settings in which learners interact with Virtual Agents of differing cultures. Thus, learners will experience multi-cultural problems and will develop specific GSD communication skills. The development of these skills must, however, be accurately assessed, bearing in mind that the training is aimed at learners with different characteristics and skills. In this paper we present an assessment process based on educational theory adapted to our simulation-based training environment. Methods to minimize the instructors' effort and tailor the assessment to specific training needs are proposed. The assessment process has been evaluated by 34 potential users. Results indicate that the assessment method yields meaningful results and proof of learning. Also that automated assessment can be achieved with minimal intervention from the instructor. Though tailored for GSD, this method could be applied to other domains, in both industry and academia.
Citations
More filters
04 Jul 2015
TL;DR: This study aims to ameliorate the very difficult task of teaching GSE by delineating the challenges and providing some recommendations for overcoming them, and indicates that in addition to the challenges posed by GSE in general, particular problems arise in educational situations.
Abstract: Context: Global Software Engineering (GSE) has become the predominant form of software development for global companies and has given rise to a demand for students trained in GSE. In response, universities are developing courses and curricula around GSE and researchers have begun to disseminate studies of these new approaches. Problem: GSE differs from most other computer science fields, however, in that practice is inseparable from theory. As a result, educators looking to create GSE courses face a daunting task: integrating global practice into the local classroom. Aim: This study aims to ameliorate the very difficult task of teaching GSE by delineating the challenges and providing some recommendations for overcoming them. Method: To meet our aims we pose two research questions ("When teaching GSE to students in Higher Education, what are the (a) challenges, and (b) recommendations for addressing them") and then conduct a systematic literature review (SLR) to determine the answers to these questions. Our SLR follows a carefully designed and validated protocol.Results: We found 82 papers that addressed our research questions. Our findings indicate that in addition to the challenges posed by GSE in general, particular problems arise in educational situations. The majority of these challenges fall into the "global distance" category, though teamwork challenges and people issues (such as trust) also commonly arise. Organizational differences between institutions, differing skill sets between students in different locations, and varying cultural work norms, for example, all operate within educational settings in quite different ways than in professional development teams. Integrating cultural training, conducting teamwork exercises to build trust, and instructor monitoring of team communication are all examples of techniques that have been used successfully by educators according to our review Conclusion: Despite the severity of the challenges in GSE education, many institutions have successfully developed courses and curricula targeting GSE. Indeed, for each of the challenges we have identified in the literature there are numerous recommendations for overcoming them. Instructors can use the recommendations given in this study as a starting point to running successful GSE courses.
56 citations
TL;DR: The original goal of this design was to ensure that students would authentically experience issues associated with realistically sized software projects, and learn to apply appropriate software engineering and collaboration practices to overcome them, in a course without significant extra staffing.
Abstract: One of the challenges of global software engineering courses is to bring the practices and experience of large geographically distributed teams into the local and time-limited environment of a classroom. Over the last 6 years, an on-campus studio course for software engineering has been developed at the University of Queensland (UQ) that places small teams of students on different features of a common product. This creates two layers of collaboration, as students work within their teams on individual features, and the teams must interoperate with many other teams on the common product. The class uses continuous integration practices and predominantly asynchronous communication channels (Slack and GitHub) to facilitate this collaboration. The original goal of this design was to ensure that students would authentically experience issues associated with realistically sized software projects, and learn to apply appropriate software engineering and collaboration practices to overcome them, in a course without significant extra staffing. Data from the development logs showed that most commits take place outside synchronous class hours, and the project operates as a temporally distributed team even though the students are geographically co-located. Since 2015, a course adapted from this format has also been taught at the University of New England (UNE), an Australian regional university that is also a longstanding provider of distance education. In this course, most students study online, and the class has to be able to work globally, because as well as students taking part from around Australia, there are also typically a small number of students taking part from overseas. Transferring the course to a smaller but predominantly online institution has allowed us to evaluate the distributed nature of the course, by considering what aspects of the course needed to change to support students who are geographically distributed, and comparing how the two cohorts behave. This has produced an overall course design, to teach professional distributed software engineering practices, that is adaptable from large classes to small, and from local to global.
11 citations
Cites background from "Assessment process for a simulation..."
...For example, through virtual environments that replicate challenges of communication [49] or...
[...]
01 Aug 2016
TL;DR: The findings based on student reflections about their first experience of virtual collaboration with a guest lecturer in a GSE course are presented, highlighting that participating in even a single virtual meeting can help the students experience some of the challenges in GSE.
Abstract: The rise of globalization in software engineering places a responsibility on educators to adequately prepare students for the unique challenges and demands of Global Software Engineering. Experiential Learning (EL) is an approach to teaching that emphasizes learning by doing. It can potentially enhance student engagement and, therefore, learning in GSE courses. In this paper, we present our findings based on student reflections about their first experience of virtual collaboration with a guest lecturer in a GSE course. In particular, we report on the challenges and learnings for students during this virtual meeting where they reflect on the importance of a pre-established relationship with the facilitator in cross-site communication. We compare our findings from student reflections with those shared by practitioners in our previous study about the challenges of distributed collaboration. We observed that both students and practitioners consider trust, goodwill, and a good relationship as important aspects in distributed communication and collaboration. Furthermore, we highlight that participating in even a single virtual meeting can help the students experience some of the challenges in GSE. We also provide the implications of such student experiences for the educators planning future GSE courses.
6 citations
Cites methods from "Assessment process for a simulation..."
...Furthermore, the additional cost and effort overhead required for setting up simulations tools as suggested by Monasor et al in [33] as a useful way to establish trust among distributed team members was also not possible in this study....
[...]
01 Oct 2014
TL;DR: A training framework based on simulation is developed with the aim of providing GSD interaction training that addresses various kinds of problems related to communication barriers.
Abstract: The evaluation of educational approaches is frequently challenging as it involves humans and requires the consideration of several angles depending on the educational objectives. On occasions, these objectives involve academic and industrial environments, thus implying additional challenges. Education in Global Software Development (GSD) has recently became important for companies that offer GSD training in specific GSD areas such as cultural, linguistic, leadership, negotiation or communication skills. We have developed a training framework based on simulation with the aim of providing GSD interaction training. This framework addresses various kinds of problems related to communication barriers. However, discovering the needs and requirements of both academia and industry is a key success factor when seeking an effective educational solution. This paper presents a multi-phased methodology that has been employed to: 1) research the training requirements in the field; 2) design the training framework based on these findings; 3) evaluate the adequacy of the framework using an Expert Feedback study and a subsequent Heuristic Evaluation; 4) prove that the framework can actually generate learning; 5) conduct a market-focused study to evaluate the commercial options of the framework and 6) define future work to fulfil commercial requirements and improve effectiveness.
1 citations
Cites methods from "Assessment process for a simulation..."
...In this study, partially described in [23], the participants were introduced to the evaluation in the form of a written explanation of VENTURE along with the objectives of the course....
[...]
06 Sep 2015
TL;DR: A systematic literature review of Global Software Engineering (GSE) education is presented in this paper, where the authors focus on the challenges of teaching GSE to students in higher education.
Abstract: Context: Global Software Engineering (GSE) has become the predominant form of software development for global companies and has given rise to a demand for students trained in GSE. In response, universities are developing courses and curricula around GSE and researchers have begun to disseminate studies of these new approaches. Problem: GSE differs from most other computer science fields, however, in that practice is inseparable from theory. As a result, educators looking to create GSE courses face a daunting task: integrating global practice into the local classroom. Aim: This study aims to ameliorate the very difficult task of teaching GSE by delineating the challenges and providing some recommendations for overcoming them. Method: To meet our aims we pose two research questions ("When teaching GSE to students in Higher Education, what are the (a) challenges, and (b) recommendations for addressing them") and then conduct a systematic literature review (SLR) to determine the answers to these questions. Our SLR follows a carefully designed and validated protocol. Results: We found 82 papers that addressed our research questions. Our findings indicate that in addition to the challenges posed by GSE in general, particular problems arise in educational situations. The majority of these challenges fall into the "global distance" category, though teamwork challenges and people issues (such as trust) also commonly arise. Organizational differences between institutions, differing skill sets between students in different locations, and varying cultural work norms, for example, all operate within educational settings in quite different ways than in professional development teams. Integrating cultural training, conducting teamwork exercises to build trust, and instructor monitoring of team communication are all examples of techniques that have been used successfully by educators according to our review Conclusion: Despite the severity of the challenges in GSE education, many institutions have successfully developed courses and curricula targeting GSE. Indeed, for each of the challenges we have identified in the literature there are numerous recommendations for overcoming them. Instructors can use the recommendations given in this study as a starting point to running successful GSE courses.
1 citations
References
More filters
01 Jan 1956
6,011 citations
"Assessment process for a simulation..." refers methods in this paper
...Most of the current research in Software Engineering education is focused on the original Bloom’s Taxonomy [5]....
[...]
Book•
01 Sep 2003TL;DR: Assessment for Learning as discussed by the authors is based on a two-year project involving thirty-six teachers in schools in Medway and Oxfordshire in the UK, and it provides valuable insights into assessment for learning as teachers describe in their own words how they turned the ideas into practical action in their schools.
Abstract: The starting point of this book was the realisation that research studies worldwide provide hard evidence that development of formative assessment raises students’ test scores. The significant improvement in the achievements of the students in this project confirms this research, while providing teachers, teacher trainers, school heads and others leaders with ideas and advice for improving formative assessment in the classroom.
Assessment for Learning is based on a two-year project involving thirty-six teachers in schools in Medway and Oxfordshire. After a brief review of the research background and of the project itself, successive chapters describe the specific practices which teachers found fruitful and the underlying ideas about learning that these developments illustrate. Later chapters discuss the problems that teachers encountered when implementing the new practices in their classroom and give guidance for school management and LEAs about promoting and supporting the changes.
This book offers valuable insights into assessment for learning as teachers describe in their own words how they turned the ideas into practical action in their schools.
1,500 citations
"Assessment process for a simulation..." refers background in this paper
...Peer- and self-assessment is often used to track the learning process where instructors delegate the assessment responsibility to their students [4]....
[...]
Proceedings Article•
01 Jan 2006
TL;DR: This work extends previous studies by developing a question set within two key pedagogical frameworks: the Bloom and SOLO taxonomies, and identifying interesting emergent patterns relating the cognitive level of the questions to student performance.
Abstract: In this paper we report on a multiinstitutional investigation into the reading and comprehension skills of novice programmers. This work extends previous studies (Lister 2004, McCracken 2001) by developing a question set within two key pedagogical frameworks: the Bloom and SOLO taxonomies. From this framework of analysis some interesting emergent patterns relating the cognitive level of the questions to student performance have been identified.
207 citations
"Assessment process for a simulation..." refers background in this paper
...[16] suggests some deficiencies in the Bloom taxonomy when applying it to programming problems....
[...]
28 May 2006
TL;DR: The experience of teaching a course to prepare graduates for software engineering (SE) in global customer-developer teams, and which was taught in three-University collaboration (Canada, Australia and Italy).
Abstract: In the context of increasing pressure to adopt global approaches to software development, the importance of teaching skills for geographically distributed software development (GSD) becomes essential. This paper reports the experience of teaching a course to prepare graduates for software engineering (SE) in global customer-developer teams, and which was taught in three-University collaboration (Canada, Australia and Italy). The course emphasized the learning of requirements management activities in frequent synchronous computer-mediated client-developer relationships and created a GSD environment with significant time zone and language differences. We describe our instructional approach and assessment strategies within a GSD instructional design framework which integrates (a) required GSD skills and strategies for aligning classroom projects with contemporary and authentic GSD conditions, (b) strategies for assessment of learning of GSD skills and (c) examples from our GSD course.
99 citations
"Assessment process for a simulation..." refers background in this paper
...The course emphasized requirements management activities through synchronous computer-mediated relationships, incorporating metrics for assessing GSD skills in the fields of [6]: international teamwork, distributed project management, computer mediated communications and ambiguity/uncertainty....
[...]
...Although lessons learned in creating GSD learning environments are reported [6], methods for GSD learning assessment is still an area in need of further research....
[...]
...[6] report on an experience of a GSD course conducted in collaboration with three universities....
[...]
Book•
30 Jan 2003
TL;DR: In this article, the authors bridge the gap between coaching and interculturalism by integrating, understanding and leveraging cultural differences across countries and corporations, in order to integrate, understand and leverage cultural differences.
Abstract: As coaches and clients increasingly realise, the demands of business mean that it is now vital to integrate, understand and leverage cultural differences across countries and corporations. This work bridges the gap between coaching and interculturalism.
95 citations
"Assessment process for a simulation..." refers background in this paper
...The Cultural Orientations Framework (COF) [15] is a framework to assess and compare cultures based on the following cultural dimensions: sense of power and responsibility, time management approaches, definitions of identity and purpose, organizational arrangements, notions of territory and boundaries, communication patterns, and modes of thinking....
[...]
...The Cultural Orientations Framework (COF) [15] is a framework to assess and compare cultures based on the following cultural dimensions: sense of power and responsibility, time management approaches, definitions of identity and purpose, organizational arrangements, notions of territory and boundaries, communication patterns, and modes of thinking....
[...]