When to update systematic literature reviews in software engineering

A systematic literature review (SLR) is an empirical method used to provide an overview of existing knowledge and to aggregate evidence within a domain. For computer science, several threats to the completeness of such reviews have been identified, leading to recommendations and guidelines on how to improve their quality. However, few studies address to what extent researchers can replicate an SLR. To conduct a replication, researchers have to first understand how the set of primary studies has been identified in the original study, and can ideally retrieve the same set when following the reported protocol. In this article, we focus on this initial step of a replication and report a two-fold empirical study: Initially, we performed a tertiary study using a sample of SLRs in computer science and identified what information that is needed to replicate the searches is reported. Based on the results, we conducted a descriptive, multi-case study on digital libraries to investigate to what extent these allow replications. The results reveal two threats to replications of SLRs: First, while researchers have improved the quality of their reports, relevant details are still missing—we refer to a reporting threat. Second, we found that some digital libraries are inconsistent in their query results—we refer to a searching threat. While researchers conducting a review can only overcome the first threat and the second may not be an issue for all kinds of replications, researchers should be aware of both threats when conducting, reviewing, and building on SLRs.

Search. Review. Repeat? An empirical study of threats to replicating SLR searches

Software Process Simulation Modeling: Systematic literature review

Software process simulation (SPS) has become an active research area for managing and improving software development processes since its introduction in the last two decades. Hybrid simulation, the combination of simulation paradigms to address a problem, is becoming more popular as the problems we are presented with become more complex. However, integrating multiple simulation paradigms faces the issues of compatibility, interoperatability and synchronization when executing simulation. The objective of this research is to present the state-of-the-art of this research area, the hybrid mechanism when integrating paradigms, and more importantly provide practical support for the effective adoption of hybrid simulation in SPS context. Based on an extended systematic literature review, this paper presents the preliminary results by answering the research questions. Depending upon the way these simulation paradigms represent different aspects and levels of software process and the context in which they can be modeled by SPS, two hybrid mechanisms: Hierarchical Mechanism and Interlinked Mechanism, have been frequently employed. The detailed discussions of integration strategies and recommendations when applying hybrid simulation may offer reference value to the SPS community.

Constructing hybrid software process simulation models

Crowdsourcing as a new emerging software development method contains crowdsourced mini-tasks as demand and online workers as suppliers. The major counter-argument in such systems is that suppliers are volunteers and are not bound by any contract, also, the size of available suppliers varies wieldy throughout the day. Such uncertainty about the receiving service may cause inefficiency and task failure. This research presents a hybrid simulation model to address the risk of task failure in competitive crowdsourcing platforms. The simulation model is composed of three components: the discrete event simulation which represents the task life cycle, the agent-based simulation which illustrates the crowd workers' decision-making process and the systems dynamic simulation which displays the platform.

A hybrid simulation model for crowdsourced software development

Software Process Simulation Modeling (SPSM) has raised research interest since 1980s. However, it is observed that SPSM studies published in the ICSSP community may have dropped in recent years. The objective of this research is to update the recent status of this area. We conducted a Systematic Literature Review (SLR) using the QGS-based search strategy. The review identified 74 primary studies in the past five years (2008-2012). This paper presents the preliminary results from this updated SLR by answering the first four research questions. Based on the findings from this updated review, it can be concluded that in terms of the number of SPSM studies found in the overall software engineering community, there is no significant change (drop) compared to the former review stage (1998-2007).

Software process simulation modeling: preliminary results from an updated systematic review

Context: Training and learning is one important purpose of Software Process Simulation (SPS). A noticeable number of studies employing SPS for Software Engineering Education (SEE) raised our interest. Objective: This research aims to reveal the-state-of-the-art and empirical effectiveness assessment of this meaningful subject area. Method: We conducted a Systematic Literature Review (SLR) to investigate this subject. Results: The review identified 34 primary studies and 10 simulators. Conclusions: The findings show that this is a continuous research area and SPS game appears more attractive to education than non-game SPS. In contrast, only one study on non-game SPS in education was reported after 2005. SimSE was identified as a landmark among the simulators for SEE in the past years.

Process simulation for software engineering education

Training and learning are one important purpose of Software Process Simulation (SPS). Some previous reviews showed a noticeable number of studies that combine SPS and Soft- ware Engineering Education (SEE). The objective of this research is to present the latest state-of-the-art of this area, and more importantly provide practical support for the effective adoption of SPS in educational context. We conducted an extended Systematic Literature Review (SLR) based on our previous reviews. The review identified 42 primary studies from 1992 to 2013. This paper presents the preliminary results by answering the research questions. The overall findings confirmed the positive impact of SPS on education. The detailed discussions and recommendations may offer reference value to the community.

Chao Gao

Papers

Constructing hybrid software process simulation models

Software process simulation modeling: preliminary results from an updated systematic review

Process simulation for software engineering education

Process simulation for software engineering education