Context A Multivocal Literature Review (MLR) is a form of a Systematic Literature Review (SLR) which includes the grey literature (e.g., blog posts, videos and white papers) in addition to the published (formal) literature (e.g., journal and conference papers). MLRs are useful for both researchers and practitioners since they provide summaries both the state-of-the art and –practice in a given area. MLRs are popular in other fields and have recently started to appear in software engineering (SE). As more MLR studies are conducted and reported, it is important to have a set of guidelines to ensure high quality of MLR processes and their results. Objective There are several guidelines to conduct SLR studies in SE. However, several phases of MLRs differ from those of traditional SLRs, for instance with respect to the search process and source quality assessment. Therefore, SLR guidelines are only partially useful for conducting MLR studies. Our goal in this paper is to present guidelines on how to conduct MLR studies in SE. Method To develop the MLR guidelines, we benefit from several inputs: (1) existing SLR guidelines in SE, (2), a literature survey of MLR guidelines and experience papers in other fields, and (3) our own experiences in conducting several MLRs in SE. We took the popular SLR guidelines of Kitchenham and Charters as the baseline and extended/adopted them to conduct MLR studies in SE. All derived guidelines are discussed in the context of an already-published MLR in SE as the running example. Results The resulting guidelines cover all phases of conducting and reporting MLRs in SE from the planning phase, over conducting the review to the final reporting of the review. In particular, we believe that incorporating and adopting a vast set of experience-based recommendations from MLR guidelines and experience papers in other fields have enabled us to propose a set of guidelines with solid foundations. Conclusion Having been developed on the basis of several types of experience and evidence, the provided MLR guidelines will support researchers to effectively and efficiently conduct new MLRs in any area of SE. The authors recommend the researchers to utilize these guidelines in their MLR studies and then share their lessons learned and experiences.

/pdf/guidelines-for-including-grey-literature-and-conducting-35ylvjftyf.pdf

Guidelines for including grey literature and conducting multivocal literature reviews in software engineering

Microservices are a new trend rising fast from the enterprise world. Even though the design principles around microservices have been identified, it is difficult to have a clear view of existing research solutions for architecting microservices. In this paper we apply the systematic mapping study methodology to identify, classify, and evaluate the current state of the art on architecting microservices from the following three perspectives: publication trends, focus of research, and potential for industrial adoption. More specifically, we systematically define a classification framework for categorizing the research on architecting microservices and we rigorously apply it to the 71 selected studies. We synthesize the obtained data and produce a clear overview of the state of the art. This gives a solid basis to plan for future research and applications of architecting microservices.

/pdf/research-on-architecting-microservices-trends-focus-and-5a0o4ky1kw.pdf

Research on Architecting Microservices: Trends, Focus, and Potential for Industrial Adoption

Systematic Literature Reviews (SLR) may not provide insight into the "state of the practice" in SE, as they do not typically include the "grey" (non-published) literature A Multivocal Literature Review (MLR) is a form of a SLR which includes grey literature in addition to the published (formal) literature Only a few MLRs have been published in SE so far We aim at raising the awareness for MLRs in SE by addressing two research questions (RQs): (1) What types of knowledge are missed when a SLR does not include the multivocal literature in a SE field? and (2) What do we, as a community, gain when we include the multivocal literature and conduct MLRs? To answer these RQs, we sample a few example SLRs and MLRs and identify the missing and the gained knowledge due to excluding or including the grey literature We find that (1) grey literature can give substantial benefits in certain areas of SE, and that (2) the inclusion of grey literature brings forward certain challenges as evidence in them is often experience and opinion based Given these conflicting viewpoints, the authors are planning to prepare systematic guidelines for performing MLRs in SE

The need for multivocal literature reviews in software engineering: complementing systematic literature reviews with grey literature

Context: DevOps, the combination of Development and Operations, is a new way of thinking in the software engineering domain that recently received much attention. Given that DevOps is a new term and novel concept recently introduced, no common understanding of what it entails has been achieved yet. Consequently, definitions of DevOps often only represent a part that is relevant to the concept.Objective:This study aims to characterize DevOps by exploring central components of DevOps definitions reported in the literature, specifying practices explicitly proposed for DevOps and investigating the similarities and differences between DevOps and other existing methods in software engineering.Method: A systematic mapping study was conducted that used six electronic databases: IEEE, ACM, Inspec, Scopus, Wiley Online Library and Web of Science.Result: 44 studies have been selected that report a definition of DevOps, 15 studies explicitly stating DevOps practices, and 15 studies stating how DevOps is related to other existing methods. Papers in some cases stated a combination of a definition, practices, and relations to other methods, the total number of primary studies was 49.Conclusion: We proposed a definition for DevOps which may overcome inconsistencies over the various existing definitions of individual research studies. In addition, the practices explicitly proposed for DevOps have been presented as well as the relation to other software development methods.

What is DevOps?: A Systematic Mapping Study on Definitions and Practices

A key application of the Internet of Things (IoT) paradigm lies within industrial contexts. Indeed, the emerging Industrial Internet of Things (IIoT), commonly referred to as Industry 4.0, promises to revolutionize production and manufacturing through the use of large numbers of networked embedded sensing devices, and the combination of emerging computing technologies, such as Fog/Cloud Computing and Artificial Intelligence. The IIoT is characterized by an increased degree of inter-connectivity, which not only creates opportunities for the industries that adopt it, but also for cyber-criminals. Indeed, IoT security currently represents one of the major obstacles that prevent the widespread adoption of IIoT technology. Unsurprisingly, such concerns led to an exponential growth of published research over the last few years. To get an overview of the field, we deem it important to systematically survey the academic literature so far, and distill from it various security requirements as well as their popularity. This paper consists of two contributions: our primary contribution is a systematic review of the literature over the period 2011–2019 on IIoT Security, focusing in particular on the security requirements of the IIoT. Our secondary contribution is a reflection on how the relatively new paradigm of Fog computing can be leveraged to address these requirements, and thus improve the security of the IIoT.

https://backend.orbit.dtu.dk/ws/files/217216288/Hkkr_09146364.pdf

A Systematic Survey of Industrial Internet of Things Security: Requirements and Fog Computing Opportunities

Context Systematic mapping studies are used to structure a research area, while systematic reviews are focused on gathering and synthesizing evidence. The most recent guidelines for systematic mapping are from 2008. Since that time, many suggestions have been made of how to improve systematic literature reviews (SLRs). There is a need to evaluate how researchers conduct the process of systematic mapping and identify how the guidelines should be updated based on the lessons learned from the existing systematic maps and SLR guidelines. Objective To identify how the systematic mapping process is conducted (including search, study selection, analysis and presentation of data, etc.); to identify improvement potentials in conducting the systematic mapping process and updating the guidelines accordingly. Method We conducted a systematic mapping study of systematic maps, considering some practices of systematic review guidelines as well (in particular in relation to defining the search and to conduct a quality assessment). Results In a large number of studies multiple guidelines are used and combined, which leads to different ways in conducting mapping studies. The reason for combining guidelines was that they differed in the recommendations given. Conclusion The most frequently followed guidelines are not sufficient alone. Hence, there was a need to provide an update of how to conduct systematic mapping studies. New guidelines have been proposed consolidating existing findings.

Guidelines for conducting systematic mapping studies in software engineering : An update

Context: Modeling is an important activity, which is used, in different phases of software engineering. Without models and modeling, it is nearly impossible to design and develop software systems, which demands the need for modeling to be taught in software engineering. There exist a number of reported models, methods, tools and languages to teach modeling in software engineering, which suggests the need for a classification and an overview of the area. This research investigates the state of published research on teaching modeling in software engineering in order to provide a systematic overview and classification of these different ways of teaching modeling with an insight on their importance and relevance to this research area. Objectives: The overall goal of the research was achieved with fulfilling the following objectives: understanding how systematic mapping is conducted, developing a systematic mapping process that will properly provide data for investigating the published research, applying the process, and finally reflecting on the results of the mappings, analyzing the importance and evaluating relevance of the published research. Methods: Systematic literature review was used as a tool to understand and inspect how systematic mapping was carried out in the area of software engineering. Based on the results of systematic literature review, new guidelines were formulated to conduct systematic mapping. These guidelines were used to investigate the published research on teaching modeling in software engineering. The results obtained through the systematic mapping were evaluated based on Industrial relevance, Rigor and citation count to examine their importance and identify research gaps. Results: 131 articles were classified into five classes such as Languages, Course Design, Curriculum design, Diagrams, others using semi-manual classification scheme and classification facets such as the type of audience, type of contribution, type of research, type of publication, type of publication year, type of research method and type of study setting. After the evaluation of Industrial relevance, rigor & citation ranking on the obtained results of the classification, 8 processes, 4 tools, 3 methods, 2 measurement-metrics and 1 model were extracted to teach modeling in software engineering. Also, this classification when compared with an existing classification, which is based on interviews and discussions, showed that our classification provides a wider overview with a deeper insight of the different ways to teach modeling in software engineering. Conclusions: Results of this systematic mapping study indicate that there is an increase in the research activity on teaching modeling in software engineering, with Unified Modeling Language (UML) being the widely research area. Much research is emphasized on teaching modeling to students from academia which indicates a research gap in developing methods, models, tools and processes to teach modeling to students/practitioners from the industry. Also, considering the citation ranking, industrial relevance and rigor of the articles, areas such as course design and curriculum development are highly neglected, suggesting the need for more research focus.

/pdf/investigating-research-on-teaching-modeling-in-software-511t34e4uq.pdf

Investigating Research on Teaching Modeling in Software Engineering -A Systematic Mapping Study

Through this report, a classification of different job scheduling algorithms available for balancing the load on web servers is made. Types such as static and dynamic scheduling algorithms are thoroughly discussed and the strengths and weaknesses of these algorithms are put forth through this article.

A Classification of Job Scheduling Algorithms for Balancing Load on Web Servers

This paper presents a discussion on the process of requirements triage in market driven requirements engineering and also reports the challenges, consequences, solutions and the experiences with the proposed solutions. Analyses of the observed results are also presented by the authors before conclusion.

http://www.airccse.org/journal/ijsea/papers/3212ijsea04.pdf

Requirements triage - challenges and solutions

The main goal of this paper is to design a tool for the evaluation of academic website, taking into account perspectives of different user groups. A literature review was conducted on the existing models, and a list of the factors affecting the quality of academic websites was identified. A framework was developed based on the identified quality factors, to evaluate the new framework, a questionnaire was devised, and a survey was conducted on the reliability of this questionnaire. To assess the effectiveness of the framework, an experiment was conducted, considering six academic websites and 6300 people from different user groups. The threats encountered during the study were also discussed with recommendations for future work.

Sairam Vakkalanka

Papers

Guidelines for conducting systematic mapping studies in software engineering : An update

Investigating Research on Teaching Modeling in Software Engineering -A Systematic Mapping Study

A Classification of Job Scheduling Algorithms for Balancing Load on Web Servers

Requirements triage - challenges and solutions

A Framework for Evaluating the Quality of Academic Websites