When to update systematic literature reviews in software engineering

Speed up genetic algorithms in the cloud using software containers

In this paper, we conduct statistical analyses for two Amazon cloud pricing models: on demand and spot. On demand cloud instances are charged a fixed price and can only be terminated by the user, with very high availability. On the other hand, spot instances are charged a dynamic price determined by a market‐driven model and can be revoked by the provider when the spot price becomes higher than the user‐defined price, having possibly low availability. Our analysis for on‐demand instances resulted in multiple linear regression equations that represent the influence of characteristics of the processor and RAM memory in the composition of the price of different types of instances available on the Amazon EC2 provider. In order to analyze the Amazon spot pricing, we used time‐smoothed moving averages by 12‐hour periods, aiming to provide a price‐availability trade‐off to the user. Our experiments with spot price histories from September to November 2016 show that the user's bid can be set at 30% of the on‐demand price, with an availability above of 90%, depending on instance type.

Statistical analysis of Amazon EC2 cloud pricing models

Context The trust in systematic literature reviews (SLRs) to provide credible recommendations is critical for establishing evidence-based software engineering (EBSE) practice. The reliability of SLR as a method is not a given and largely depends on the rigor of the attempt to identify, appraise and aggregate evidence. Previous research, by comparing SLRs on the same topic, has identified search as one of the reasons for discrepancies in the included primary studies. This affects the reliability of an SLR, as the papers identified and included in it are likely to influence its conclusions. Objective We aim to propose a comprehensive evaluation checklist to assess the reliability of an automated-search strategy used in an SLR. Method Using a literature review, we identified guidelines for designing and reporting automated-search as a primary search strategy. Using the aggregated design, reporting and evaluation guidelines, we formulated a comprehensive evaluation checklist. The value of this checklist was demonstrated by assessing the reliability of search in 27 recent SLRs. Results Using the proposed evaluation checklist, several additional issues (not captured by the current evaluation checklist) related to the reliability of search in recent SLRs were identified. These issues severely limit the coverage of literature by the search and also the possibility to replicate it. Conclusion Instead of solely relying on expensive replications to assess the reliability of SLRs, this work provides means to objectively assess the likely reliability of a search-strategy used in an SLR. It highlights the often-assumed aspect of repeatability of search when using automated-search. Furthermore, by explicitly considering repeatability and consistency as sub-characteristics of a reliable search, it provides a more comprehensive evaluation checklist than the ones currently used in EBSE.

Reliability of search in systematic reviews: Towards a quality assessment framework for the automated-search strategy

Cloud Infrastructure as a Service (IaaS) providers continually seek higher resource utilization to better amortize capital costs Higher utilization not only can enable higher profit for IaaS providers but also provides a mechanism to raise energy efficiency; therefore creating greener cloud services Unfortunately, achieving high utilization is difficult mainly due to infrastructure providers needing to maintain spare capacity to service demand fluctuations Graceful degradation is a self-adaptation technique originally designed for constructing robust services that survive resource shortages Previous work has shown that graceful degradation can also be used to improve resource utilization in the cloud by absorbing demand fluctuations and reducing spare capacity In this work, we build a system and pricing model that enables infrastructure providers to incentivize their tenants to use graceful degradation By using graceful degradation with an appropriate pricing model, the infrastructure provider can realize higher resource utilization while simultaneously, its tenants can increase their profit Our proposed solution is based on a hybrid model which guarantees both reserved and peak on-demand capacities over flexible periods It also includes a global dynamic price pair for capacity which remains uniform during each tenant's Service Level Agreement (SLA) term We evaluate our scheme using simulations based on real-world traces and also implement a prototype using RUBiS on the Xen hypervisor as an end-to-end demonstration Our analysis shows that the proposed scheme never hurts a tenant's net profit, but can improve it by as much as 93% Simultaneously, it can also improve the effective utilization of contracts from 42% to as high as 99%

/pdf/incentivizing-self-capping-to-increase-cloud-utilization-2ijqjvkwat.pdf

Incentivizing self-capping to increase cloud utilization

https://portal.research.lu.se/portal/files/7608509/JSS_SpotPricing_v3_majorRevision.pdf

Spot Pricing in the Cloud Ecosystem: A Comparative Investigation

Background: Spot pricing is considered as a significant supplement for building a full-fledged market economy for the Cloud ecosystem. However, it seems that both providers and consumers are still hesitating to enter the Cloud spot market. The relevant academic community also has conflicting opinions about Cloud spot pricing in terms of revenue generation. Aim: This work aims to systematically identify, assess, synthesize and report the published evidence in favor of or against spot-price scheme compared with fixed-price scheme of Cloud computing, so as to help relieve the aforementioned conflict. Method: We employed the systematic literature review (SLR) method to collect and investigate the empirical studies of Cloud spot pricing indexed by major electronic libraries. Results: This SLR identified 61 primary studies that either delivered discussions or conducted experiments to perform comparison between spot pricing and fixed pricing in the Cloud domain. The reported benefits and limitations were summarized to facilitate cost-benefit analysis of being a Cloud spot pricing player, while four types of theories were distinguished to help both researchers and practitioners better understand the Cloud spot market. Conclusions: This SLR shows that the academic community strongly advocates the emerging Cloud spot market. Although there is still a lack of practical and easily deployable market-driven mechanisms, the overall findings of our work indicate that spot pricing plays a promising role in the sustainability of Cloud resource exploitation.

Software process simulation (SPS) has become an active research area for managing and improving software development processes since its introduction in the last two decades. Hybrid simulation, the combination of simulation paradigms to address a problem, is becoming more popular as the problems we are presented with become more complex. However, integrating multiple simulation paradigms faces the issues of compatibility, interoperatability and synchronization when executing simulation. The objective of this research is to present the state-of-the-art of this research area, the hybrid mechanism when integrating paradigms, and more importantly provide practical support for the effective adoption of hybrid simulation in SPS context. Based on an extended systematic literature review, this paper presents the preliminary results by answering the research questions. Depending upon the way these simulation paradigms represent different aspects and levels of software process and the context in which they can be modeled by SPS, two hybrid mechanisms: Hierarchical Mechanism and Interlinked Mechanism, have been frequently employed. The detailed discussions of integration strategies and recommendations when applying hybrid simulation may offer reference value to the SPS community.

Constructing hybrid software process simulation models

Software Process Simulation Modeling (SPSM) has raised research interest since 1980s. However, it is observed that SPSM studies published in the ICSSP community may have dropped in recent years. The objective of this research is to update the recent status of this area. We conducted a Systematic Literature Review (SLR) using the QGS-based search strategy. The review identified 74 primary studies in the past five years (2008-2012). This paper presents the preliminary results from this updated SLR by answering the first four research questions. Based on the findings from this updated review, it can be concluded that in terms of the number of SPSM studies found in the overall software engineering community, there is no significant change (drop) compared to the former review stage (1998-2007).

Software process simulation modeling: preliminary results from an updated systematic review

Context: Training and learning is one important purpose of Software Process Simulation (SPS). A noticeable number of studies employing SPS for Software Engineering Education (SEE) raised our interest. Objective: This research aims to reveal the-state-of-the-art and empirical effectiveness assessment of this meaningful subject area. Method: We conducted a Systematic Literature Review (SLR) to investigate this subject. Results: The review identified 34 primary studies and 10 simulators. Conclusions: The findings show that this is a continuous research area and SPS game appears more attractive to education than non-game SPS. In contrast, only one study on non-game SPS in education was reported after 2005. SimSE was identified as a landmark among the simulators for SEE in the past years.

Shu Jiang

Papers

Spot Pricing in the Cloud Ecosystem: A Comparative Investigation

Spot Pricing in the Cloud Ecosystem: A Comparative Investigation

Constructing hybrid software process simulation models

Software process simulation modeling: preliminary results from an updated systematic review

Process simulation for software engineering education