Business Research: A Practical Guide for Undergraduate and Postgraduate Students

This paper summarizes the current state of the art and recent trends in software engineering economics. It provides an overview of economic analysis techniques and their applicability to software engineering and management. It surveys the field of software cost estimation, including the major estimation techniques available, the state of the art in algorithmic cost models, and the outstanding research issues in software cost estimation.

Software engineering economics

DevOpsis a collaborative and multidisciplinary organizational effort to automate continuous delivery of new software updates while guaranteeing their correctness and reliability. The present survey investigates and discusses DevOps challenges from the perspective of engineers, managers, and researchers. We review the literature and develop a DevOps conceptual map, correlating the DevOps automation tools with these concepts. We then discuss their practical implications for engineers, managers, and researchers. Finally, we critically explore some of the most relevant DevOps challenges reported by the literature.

A Survey of DevOps Concepts and Challenges

Continuous Integration (CI) has become a disruptive innovation in software development: with proper tool support and adoption, positive effects have been demonstrated for pull request throughput and scaling up of project sizes. As any other innovation, adopting CI implies adapting existing practices in order to take full advantage of its potential, and "best practices" to that end have been proposed. Here we study the adaptation and evolution of code writing and submission, issue and pull request closing, and testing practices as Travis CI is adopted by hundreds of established projects on GitHub. To help essentialize the quantitative results, we also survey a sample of GitHub developers about their experiences with adopting Travis CI. Our findings suggest a more nuanced picture of how GitHub teams are adapting to, and benefiting from, continuous integration technology than suggested by prior work.

The impact of continuous integration on other software development practices: a large-scale empirical study

Context: DevOps is considered important in the ability to frequently and reliably update a system in operational state. DevOps presumes cross-functional collaboration and automation between software development and operations. DevOps adoption and implementation in companies is non-trivial due to required changes in technical, organisational and cultural aspects. Objectives: This exploratory study presents detailed descriptions of how DevOps is implemented in practice. The context of our empirical investigation is web application and service development in small and medium sized companies. Method: A multiple-case study was conducted in five different development contexts with successful DevOps implementations since its benefits, such as quick releases and minimum deployment errors, were achieved. Data was mainly collected through interviews with 26 practitioners and observations made at the companies. Data was analysed by first coding each case individually using a set of predefined themes and thereafter perform a cross-case synthesis. Results: Our analysis yielded some of the following results: (i) software development team attaining ownership and responsibility to deploy software changes in production is crucial in DevOps. (ii) toolchain usage and support in deployment pipeline activities accelerates the delivery of software changes, bug fixes and handling of production incidents. (ii) the delivery speed to production is affected by context factors, such as manual approvals by the product owner (iii) steep learning curve for new skills is experienced by both software developers and operations staff, who also have to cope with working under pressure. Conclusion: Our findings contributes to the overall understanding of DevOps concept, practices and its perceived impacts, particularly in small and medium sized companies. We discuss two practical implications of the results.

/pdf/devops-in-practice-a-multiple-case-study-of-five-companies-248koxjq03.pdf

DevOps in practice: A multiple case study of five companies

DevOps and continuous practices are attracting steadily growing attention by both practitioners and researchers in the software engineering community. The terms are often used inconsistently, interchangeably and with unclear meaning, however. Taking the position that this ambiguity and miscommunication renders the community great harm, in that it impedes our ability to critically appraise these practices, their effects and interplay between them, we analyze how published literature on both continuous practices and DevOps treat the terms. Based on this analysis, along with statements by often cited sources in the community as well as personal experience from researching and practicing these concepts, we propose guidelines to help authors reduce ambiguity in their publications. Additionally, definitions designed to reflect mainstream interpretation while disentangling the terms from one another are presented.

Continuous practices and devops: beyond the buzz, what does it all mean?

In this paper we present a summary of factors that must be taken into account when applying continuous integration to software-intensive embedded systems. Experiences are presented from two study cases regarding seven topics: complex user scenarios, compliance to standards, long build times, many technology fields, security aspects, architectural runway and test environments. In the analysis we show how issues within these topics obstruct the organization from working according to the practices of continuous integration. The identified impediments are mapped to a list of continuous integration corner-stones proposed in literature.

Continuous Integration Applied to Software-Intensive Embedded Systems – Problems and Experiences

Based on interviews with 20 developers from two case study companies that develop large-scale software-intensive embedded systems, this paper presents the main factors that affect how often developers deliver software to the mainline Further on, the paper describes the continuous integration behaviors in projects where up to 1,000 developers commit to the same mainline The main factors that could enable more frequent integration of software are: "Activity planning and execution", "System thinking", "Speed" and "Confidence through test activities" Behind these main themes we also present a wide range of sub-categories ("Modular and loosely coupled architecture", "Test selection" etc) which summarizes what the developers themselves see as the continuous integration impediments in large-scale industry projects

Continuous Integration Impediments in Large-Scale Industry Projects

Enable more frequent integration of software in industry projects

In this paper, we show how exploratory testing plays a role as part of a continuous integration and delivery pipeline for large-scale and complex software products. We propose a test method that incorporates exploratory testing as an activity in the continuous integration and delivery pipeline, and is based on elements from other testing techniques such as scenario-based testing, testing in teams and testing in time-boxed sessions. The test method has been validated during ten months by 28 individuals (21 engineers and 7 flight test pilots) in a case study where the system under test is a fighter aircraft. Quantitative data from the case study company shows that the exploratory test teams produced more problem reports than other test teams. The interview results show that both engineers and test pilots were generally positive or very positive when they described their experiences from the case study, and consider the test method to be an efficient way of testing the system in the case study.

Torvald Martensson

Papers

Continuous practices and devops: beyond the buzz, what does it all mean?

Continuous Integration Applied to Software-Intensive Embedded Systems – Problems and Experiences

Continuous Integration Impediments in Large-Scale Industry Projects

Enable more frequent integration of software in industry projects

Exploratory testing of large-scale systems – Testing in the continuous integration and delivery pipeline