Context The number of maturity models proposed in the area of Business Process Management (BPM) has increased considerably in the last decade. However, there are a number of challenges, such as the limited empirical studies on their validation and a limited extent of actionable properties of these models in guiding their application. These challenges hinder the widespread usage of the maturity models in the BPM field. Objective In order to better understand the state of the research on business process maturity models (BPMMs) and identify opportunities for future research, we conducted a systematic literature review. Method We searched the studies between the years 1990 and 2014 in established digital libraries to identify empirical studies of BPMMs by focusing on their development, validation, and application. We targeted studies on generic models proposed for business process maturity, business process management or orientation maturity, and selected 61 studies out of 2899 retrieved initially. Results We found that despite that many BPMMs were proposed in the last decade, the level of empirical evidence that reveals the validity and usefulness of these models is scarce. Conclusion The current state of research on BPM maturity is in its early phases, and academic literature lacks methodical applications of many mainstream BPMMs that have been proposed. Future research should be directed towards: (1) reconciling existing models with a strong emphasis on prescriptive properties, (2) conducting empirical studies to demonstrate the validity and usefulness of BPMMs, and (3) separating the assessment method used to evaluate the maturity level from the maturity model which acts as the reference framework for the assessment.

/pdf/business-process-maturity-models-a-systematic-literature-2proxcik9c.pdf

Business process maturity models : A systematic literature review

ContextA number of systematic literature reviews and mapping studies (SLRs) covering numerous primary research studies on various aspects of agile software development (ASD) exist. ObjectiveThe aim of this paper is to provide an overview of the SLRs on ASD research topics for software engineering researchers and practitioners. MethodWe followed the tertiary study guidelines by Kitchenham etal. to find SLRs published between late 1990s to December 2015. ResultsWe found 28 SLRs focusing on ten different ASD research areas: adoption, methods, practices, human and social aspects, CMMI, usability, global software engineering (GSE), organizational agility, embedded systems, and software product line engineering. The number of SLRs on ASD topics, similar to those on software engineering (SE) topics in general, is on the rise. A majority of the SLRs applied standardized guidelines and the quality of these SLRs on ASD topics was found to be slightly higher for journal publications than for conferences. While some individuals and institutions seem to lead this area, the spread of authors and institutions is wide. With respect to prior review recommendations, significant progress was noticed in the area of connecting agile to established domains such as usability, CMMI, and GSE; and considerable progress was observed in focusing on management-oriented approaches as Scrum and sustaining ASD in different contexts such as embedded systems. ConclusionSLRs of ASD studies are on the rise and cover a variety of ASD aspects, ranging from early adoption issues to newer applications of ASD such as in product line engineering. ASD research can benefit from further primary and secondary studies on evaluating benefits and challenges of ASD methods, agile hybrids in large-scale setups, sustainability, motivation, teamwork, and project management; as well as a fresh review of empirical studies in ASD to cover the period post 2008.

Systematic literature reviews in agile software development

Evidence-based medicine

http://www.cs.rug.nl/~paris/papers/JSS16a.pdf

A systematic mapping study on the combination of software architecture and agile development

Context Many of the systematic reviews published in software engineering are related to research or methodological issues and hence are unlikely to be of direct benefit to practitioners or teachers. Those that are relevant to practice and teaching need to be presented in a form that makes their findings usable with minimum interpretation. Objective We have examined a sample of the many systematic reviews that have been published over a period of six years, in order to assess how well these are reported and identify useful lessons about how this might be done. Method We undertook a tertiary study, performing a systematic review of systematic reviews. Our study found 178 systematic reviews published in a set of major software engineering journals over the period 2010–2015. Of these, 37 provided recommendations or conclusions of relevance to education and/or practice and we used the DARE criteria as well as other attributes related to the systematic review process to analyse how well they were reported. Results We have derived a set of 12 ‘lessons’ that could help authors with reporting the outcomes of a systematic review in software engineering. We also provide an associated checklist for use by journal and conference referees. Conclusion There are several areas where better reporting is needed, including quality assessment, synthesis, and the procedures followed by the reviewers. Researchers, practitioners, teachers and journal referees would all benefit from better reporting of systematic reviews, both for clarity and also for establishing the provenance of any findings.

/pdf/reporting-systematic-reviews-some-lessons-from-a-tertiary-2hxul6600u.pdf

Reporting systematic reviews: Some lessons from a tertiary study

Background The search for adherence to maturity levels by using lightweight processes that require low levels of effort is regarded as a challenge for software development organizations. Objective This study seeks to evaluate, synthesize, and present results on the use of the Capability Maturity Model Integration (CMMI) in combination with agile software development, and thereafter to give an overview of the topics researched, which includes a discussion of their benefits and limitations, the strength of the findings, and the implications for research and practice. Methods The method applied was a Systematic Literature Review on studies published up to (and including) 2011. Results The search strategy identified 3193 results, of which 81 included studies on the use of CMMI together with agile methodologies. The benefits found were grouped into two main categories: those related to the organization in general and those related to the development process, and were organized into subcategories, according to the area to which they refer. The limitations were also grouped into these categories. Using the criteria defined, the strength of the evidence found was considered low. The implications of the results for research and practice are discussed. Conclusion Agile methodologies can be used by companies to reduce efforts in getting to levels 2 and 3 of CMMI, there even being reports of applying agile practices that led to achieving level 5. However, agile methodologies alone, according to the studies, were not sufficient to obtain a rating at a given level, it being necessary to resort to additional practices to do so.

https://romisatriawahono.net/lecture/rm/survey/software%20engineering/Software%20Process%20Improvement/Silva%20-%20CMMI%20and%20Agile%20-%202015.pdf

Using CMMI together with agile software development: A systematic review

Programmers often use forums, such as StackOverflow, to easily and quickly solve their issues. Researchers then investigate those questions to better understand the state-of-use of software engineering techniques. Also, due to the quality and the great number questions and answers, the results found using such method might be difficult, or even impossible, to find using common survey techniques. In this study, we conducted a qualitative and quantitative research in order to categorize questions about refactoring tools. As a result, we presented a comprehensive classification of flaws and desirable features in refactoring tools. We also reported that programmers do not often rely on refactoring tools, but, at the same time, they are desiring number of unimplemented features.

What programmers say about refactoring tools?: an empirical investigation of stack overflow

Context: Grey Literature (GL) recently has grown in Software Engineering (SE) research since the increased use of online communication channels by software engineers. However, there is still a limited understanding of how SE research is taking advantage of GL. Objective: This research aimed to understand how SE researchers use GL in their secondary studies. Methods: We conducted a tertiary study of studies published between 2011 and 2018 in high-quality software engineering conferences and journals. We then applied qualitative and quantitative analysis to investigate 446 potential studies. Results: From the 446 selected studies, 126 studies cited GL but only 95 of those used GL to answer a specific research question representing almost 21% of all the 446 secondary studies. Interestingly, we identified that few studies employed specific search mechanisms and used additional criteria for assessing GL. Moreover, by the time we conducted this research, 49% of the GL URLs are not working anymore. Based on our findings, we discuss some challenges in using GL and potential mitigation plans. Conclusion: In this paper, we summarized the last 10 years of software engineering research that uses GL, showing that GL has been essential for bringing practical new perspectives that are scarce in traditional literature. By drawing the current landscape of use, we also raise some awareness of related challenges (and strategies to deal with them).

Grey Literature in Software Engineering: A critical review

Questions and Answers (Q&A) websites maintain a long history of needs, problems, and challenges that software developers face. In contrast to Q&A websites, which are strongly tied to practitioners' needs, there are systematic reviews (SRs), which, according to recent studies, lack a connection with software engineering practice. In this paper, we investigate this claim by assessing to what extent systematic reviews help to solve questions posted on Q&A websites. To achieve this goal, we propose and evaluate a coverage method. We applied this method to a set of more than 600 questions related to agile software development. Results suggest that 12% of the related questions were covered. When considering specific agile methods, the majority of them have coverage below 50% or were not covered at all. We also identified 27 recurrent questions.

Using Q&A websites as a method for assessing systematic reviews

Context: For more than 15 years, Agile Software Development (ASD) has been used to improve software development, process, and quality. However, there are scenarios where the effectiveness of these methods and practices has not been rigorously evaluated.Objective: Understand the benefits and limitations related to these methods and practices in a particular context: two software companies based on Pernambuco's Technology Park, Brazil.Method: In this paper, we conducted 22 semi-structured interviews to understand the benefits and limitations of ASD in an industrial context. The data were extracted using open coding and analyzed through qualitative techniques.Results: Our preliminary analysis identified a core of 28 benefits and 20 limitations with the usage of ASD. As for benefits, we found that facilitates project monitoring and tracking as well as the interaction and collaboration. As for limitations, we found that it difficulty working with user stories and to work with large teams.Conclusion: This study serves as a practical guide for software companies interested in adopting and improving the use of ASD.

Fernando Kamei

Papers

Using CMMI together with agile software development: A systematic review

What programmers say about refactoring tools?: an empirical investigation of stack overflow

Grey Literature in Software Engineering: A critical review

Using Q&A websites as a method for assessing systematic reviews

On the Benefits/Limitations of Agile Software Development: An Interview Study with Brazilian Companies