Showing papers in "Information & Software Technology in 2017"

Static analysis of android apps

Abstract: ContextStatic analysis exploits techniques that parse program source code or bytecode, often traversing program paths to check some program properties. Static analysis approaches have been proposed for different tasks, including for assessing the security of Android apps, detecting app clones, automating test cases generation, or for uncovering non-functional issues related to performance or energy. The literature thus has proposed a large body of works, each of which attempts to tackle one or more of the several challenges that program analyzers face when dealing with Android apps. ObjectiveWe aim to provide a clear view of the state-of-the-art works that statically analyze Android apps, from which we highlight the trends of static analysis approaches, pinpoint where the focus has been put, and enumerate the key aspects where future researches are still needed. MethodWe have performed a systematic literature review (SLR) which involves studying 124 research papers published in software engineering, programming languages and security venues in the last 5 years (January 2011December 2015). This review is performed mainly in five dimensions: problems targeted by the approach, fundamental techniques used by authors, static analysis sensitivities considered, android characteristics taken into account and the scale of evaluation performed. ResultsOur in-depth examination has led to several key findings: 1) Static analysis is largely performed to uncover security and privacy issues; 2) The Soot framework and the Jimple intermediate representation are the most adopted basic support tool and format, respectively; 3) Taint analysis remains the most applied technique in research approaches; 4) Most approaches support several analysis sensitivities, but very few approaches consider path-sensitivity; 5) There is no single work that has been proposed to tackle all challenges of static analysis that are related to Android programming; and 6) Only a small portion of state-of-the-art works have made their artifacts publicly available. ConclusionThe research community is still facing a number of challenges for building approaches that are aware altogether of implicit-Flows, dynamic code loading features, reflective calls, native code and multi-threading, in order to implement sound and highly precise static analyzers.

How to design gamification? A method for engineering gamified software

Abstract: Context Since its inception around 2010, gamification has become one of the top technology and software trends. However, gamification has also been regarded as one of the most challenging areas of software engineering. Beyond traditional software design requirements, designing gamification requires the command of disciplines such as (motivational/behavioral) psychology, game design, and narratology, making the development of gamified software a challenge for traditional software developers. Gamification software inhabits a finely tuned niche of software engineering that seeks for both high functionality and engagement; beyond technical flawlessness, gamification has to motivate and affect users. Consequently, it has also been projected that most gamified software is doomed to fail. Objective This paper seeks to advance the understanding of designing gamification and to provide a comprehensive method for developing gamified software. Method We approach the research problem via a design science research approach; firstly, by synthesizing the current body of literature on gamification design methods and by interviewing 25 gamification experts, producing a comprehensive list of design principles for developing gamified software. Secondly, and more importantly, we develop a detailed method for engineering of gamified software based on the gathered knowledge and design principles. Finally, we conduct an evaluation of the artifacts via interviews of ten gamification experts and implementation of the engineering method in a gamification project. Results As results of the study, we present the method and key design principles for engineering gamified software. Based on the empirical and expert evaluation, the developed method was deemed as comprehensive, implementable, complete, and useful. We deliver a comprehensive overview of gamification guidelines and shed novel insights into the nature of gamification development and design discourse. Conclusion This paper takes first steps towards a comprehensive method for gamified software engineering.

TLEL: A two-layer ensemble learning approach for just-in-time defect prediction

Abstract: Context Defect prediction is a very meaningful topic, particularly at change-level. Change-level defect prediction, which is also referred as just-in-time defect prediction, could not only ensure software quality in the development process, but also make the developers check and fix the defects in time [1]. Objective Ensemble learning becomes a hot topic in recent years. There have been several studies about applying ensemble learning to defect prediction [2–5]. Traditional ensemble learning approaches only have one layer, i.e., they use ensemble learning once. There are few studies that leverages ensemble learning twice or more. To bridge this research gap, we try to hybridize various ensemble learning methods to see if it will improve the performance of just-in-time defect prediction. In particular, we focus on one way to do this by hybridizing bagging and stacking together and leave other possibly hybridization strategies for future work. Method In this paper, we propose a two-layer ensemble learning approach TLEL which leverages decision tree and ensemble learning to improve the performance of just-in-time defect prediction. In the inner layer, we combine decision tree and bagging to build a Random Forest model. In the outer layer, we use random under-sampling to train many different Random Forest models and use stacking to ensemble them once more. Results To evaluate the performance of TLEL, we use two metrics, i.e., cost effectiveness and F1-score. We perform experiments on the datasets from six large open source projects, i.e., Bugzilla, Columba, JDT, Platform, Mozilla, and PostgreSQL, containing a total of 137,417 changes. Also, we compare our approach with three baselines, i.e., Deeper, the approach proposed by us [6], DNC, the approach proposed by Wang et al. [2], and MKEL, the approach proposed by Wang et al. [3]. The experimental results show that on average across the six datasets, TLEL could discover over 70% of the bugs by reviewing only 20% of the lines of code, as compared with about 50% for the baselines. In addition, the F1-scores TLEL can achieve are substantially and statistically significantly higher than those of three baselines across the six datasets. Conclusion TLEL can achieve a substantial and statistically significant improvement over the state-of-the-art methods, i.e., Deeper, DNC and MKEL. Moreover, TLEL could discover over 70% of the bugs by reviewing only 20% of the lines of code.

Systematic literature reviews in agile software development

Abstract: ContextA number of systematic literature reviews and mapping studies (SLRs) covering numerous primary research studies on various aspects of agile software development (ASD) exist. ObjectiveThe aim of this paper is to provide an overview of the SLRs on ASD research topics for software engineering researchers and practitioners. MethodWe followed the tertiary study guidelines by Kitchenham etal. to find SLRs published between late 1990s to December 2015. ResultsWe found 28 SLRs focusing on ten different ASD research areas: adoption, methods, practices, human and social aspects, CMMI, usability, global software engineering (GSE), organizational agility, embedded systems, and software product line engineering. The number of SLRs on ASD topics, similar to those on software engineering (SE) topics in general, is on the rise. A majority of the SLRs applied standardized guidelines and the quality of these SLRs on ASD topics was found to be slightly higher for journal publications than for conferences. While some individuals and institutions seem to lead this area, the spread of authors and institutions is wide. With respect to prior review recommendations, significant progress was noticed in the area of connecting agile to established domains such as usability, CMMI, and GSE; and considerable progress was observed in focusing on management-oriented approaches as Scrum and sustaining ASD in different contexts such as embedded systems. ConclusionSLRs of ASD studies are on the rise and cover a variety of ASD aspects, ranging from early adoption issues to newer applications of ASD such as in product line engineering. ASD research can benefit from further primary and secondary studies on evaluating benefits and challenges of ASD methods, agile hybrids in large-scale setups, sustainability, motivation, teamwork, and project management; as well as a fresh review of empirical studies in ASD to cover the period post 2008.

Software defect prediction using stacked denoising autoencoders and two-stage ensemble learning

Abstract: Context Software defect prediction (SDP) plays an important role in allocating testing resources reasonably, reducing testing costs, and ensuring software quality. However, software metrics used for SDP are almost entirely traditional features compared with deep representations (DPs) from deep learning. Although stacked denoising autoencoders (SDAEs) are powerful for feature learning and have been successfully applied in other fields, to the best of our knowledge, it has not been investigated in the field of SDP. Meanwhile, class-imbalance is still a pressing problem needing to be addressed. Objective In this paper, we propose a novel SDP approach, SDAEsTSE, which takes advantages of SDAEs and ensemble learning, namely the proposed two-stage ensemble (TSE). Method Our method mainly includes two phases: the deep learning phase and two-stage ensemble (TSE) phase. We first use SDAEs to extract the DPs from the traditional software metrics, and then a novel ensemble learning approach, TSE, is proposed to address the class-imbalance problem. Results Experiments are performed on 12 NASA datasets to demonstrate the effectiveness of DPs, the proposed TSE, and SDAEsTSE, respectively. The performance is evaluated in terms of F-measure, the area under the curve (AUC), and Matthews correlation coefficient (MCC). Generally, DPs, TSE, and SDAEsTSE contribute to significantly higher performance compared with corresponding traditional metrics, classic ensemble methods, and benchmark SDP models. Conclusions It can be concluded that (1) deep representations are promising for SDP compared with traditional software metrics, (2) TSE is more effective for addressing the class-imbalance problem in SDP compared with classic ensemble learning methods, and (3) the proposed SDAEsTSE is significantly effective for SDP.

Systematic literature review and empirical investigation of barriers to process improvement in global software development

Abstract: ContextIncreasingly, software development organizations are adopting global software development (GSD) strategies, mainly because of the significant return on investment they produce. However, there are many challenges associated with GSD, particularly with regards to software process improvement (SPI). SPI can play a significant role in the successful execution of GSD projects. ObjectiveThe aim of the present study was to identify barriers that can negatively affect SPI initiatives in GSD organizations from both client and vendor perspectives. MethodA systematic literature review (SLR) and survey questionnaire were used to identify and validate the barriers. ResultsTwenty-two barriers to successful SPI programs were identified. Results illustrate that the barriers identified using SLR and survey approaches have more similarities However, there were significant differences between the ranking of these barriers in the SLR and survey approaches, as indicated by the results of t-tests (for instance, t=2.28, p=0.011<0.05). Our findings demonstrate that there is a moderate positive correlation between the ranks obtained from the SLR and the empirical study (rs (22)=0.567, p=0.006). ConclusionsThe identified barriers can assist both client and vendor GSD organizations during initiation of an SPI program. Client-vendor classification was used to provide a broad picture of SPI programs, and their respective barriers. The top-ranked barriers can be used as a guide for GSD organizations prior to the initiation of an SPI program. We believe that the results of this study can be useful in tackling the problems associated with the implementation of SPI, which is vital to the success and progression of GSD organizations.

Taxonomies in software engineering

Abstract: Context: Software Engineering (SE) is an evolving discipline with new subareas being continuously developed and added. To structure and better understand the SE body of knowledge, taxonomies have been proposed in all SE knowledge areas.Objective: The objective of this paper is to characterize the state-of-the-art research on SE taxonomies.Method: A systematic mapping study was conducted, based on 270 primary studies.Results: An increasing number of SE taxonomies have been published since 2000 in a broad range of venues, including the top SE journals and conferences. The majority of taxonomies can be grouped into the following SWEBOK knowledge areas: construction (19.55%), design (19.55%), requirements (15.50%) and maintenance (11.81%). Illustration (45.76%) is the most frequently used approach for taxonomy validation. Hierarchy (53.14%) and faceted analysis (39.48%) are the most frequently used classification structures. Most taxonomies rely on qualitative procedures to classify subject matter instances, but in most cases (86.53%) these procedures are not described in sufficient detail. The majority of the taxonomies (97%) target unique subject matters and many taxonomy-papers are cited frequently. Most SE taxonomies are designed in an ad-hoc way. To address this issue, we have revised an existing method for developing taxonomies in a more systematic way.Conclusion: There is a strong interest in taxonomies in SE, but few taxonomies are extended or revised. Taxonomy design decisions regarding the used classification structures, procedures and descriptive bases are usually not well described and motivated.

Software test maturity assessment and test process improvement

Abstract: ContextSoftware testing practices and processes in many companies are far from being mature and are usually conducted in ad-hoc fashions. Such immature practices lead to various negative outcomes, e.g., ineffectiveness of testing practices in detecting all the defects, and cost and schedule overruns of testing activities. To conduct test maturity assessment (TMA) and test process improvement (TPI) in a systematic manner, various TMA/TPI models and approaches have been proposed. ObjectiveIt is important to identify the state-of-the-art and the practice in this area to consolidate the list of all various test maturity models proposed by practitioners and researchers, the drivers of TMA/TPI, the associated challenges and the benefits and results of TMA/TPI. Our article aims to benefit the readers (both practitioners and researchers) by providing the most comprehensive survey of the area, to this date, in assessing and improving the maturity of test processes. MethodTo achieve the above objective, we have performed a Multivocal Literature Review (MLR) study to find out what we know about TMA/TPI. A MLR is a form of a Systematic Literature Review (SLR) which includes the grey literature (e.g., blog posts and white papers) in addition to the published (formal) literature (e.g., journal and conference papers). We searched the academic literature using the Google Scholar and the grey literature using the regular Google search engine. ResultsOur MLR and its results are based on 181 sources, 51 (29%) of which were grey literature and 130 (71%) were formally published sources. By summarizing what we know about TMA/TPI, our review identified 58 different test maturity models and a large number of sources with varying degrees of empirical evidence on this topic. We also conducted qualitative analysis (coding) to synthesize the drivers, challenges and benefits of TMA/TPI from the primary sources. ConclusionWe show that current maturity models and techniques in TMA/TPI provides reasonable advice for industry and the research community. We suggest directions for follow-up work, e.g., using the findings of this MLR in industry-academia collaborative projects and empirical evaluation of models and techniques in the area of TMA/TPI as reported in this article.

Problems, Causes and Solutions When Adopting Continuous Delivery - A Systematic Literature Review

Abstract: Context: Continuous delivery is a software development discipline in which software is always kept releasable. The literature contains instructions on how to adopt continuous delivery, but the adoption has been challenging in practice. Objective: In this study, a systematic literature review is conducted to survey the faced problems when adopting continuous delivery. In addition, we identify causes for and solutions to the problems. Method: By searching five major bibliographic databases, we identified 293 articles related to continuous delivery. We selected 30 of them for further analysis based on them containing empirical evidence of adoption of continuous delivery, and focus on practice instead of only tooling. We analyzed the selected articles qualitatively and extracted problems, causes and solutions. The problems and solutions were thematically synthesized into seven themes: build design, system design, integration, testing, release, human and organizational and resource. Results: We identified a total of 40 problems, 28 causal relationships and 29 solutions related to adoption of continuous delivery. Testing and integration problems were reported most often, while the most critical reported problems were related to testing and system design. Causally, system design and testing were most connected to other themes. Solutions in the system design, resource and human and organizational themes had the most significant impact on the other themes. The system design and build design themes had the least reported solutions. Conclusions: When adopting continuous delivery, problems related to system design are common, critical and little studied. The found problems, causes and solutions can be used to solve problems when adopting continuous delivery in practice.

On code reuse from StackOverflow

Abstract: Context: Source code reuse has been widely accepted as a fundamental activity in software development. Recent studies showed that StackOverflow has emerged as one of the most popular resources for code reuse. Therefore, a plethora of work proposed ways to optimally ask questions, search for answers and find relevant code on StackOverflow. However, little work studies the impact of code reuse from StackOverflow.Objective: To better understand the impact of code reuse from StackOverflow, we perform an exploratory study focusing on code reuse from StackOverflow in the context of mobile apps. Specifically, we investigate how much, why, when, and who reuses code. Moreover, to understand the potential implications of code reuse, we examine the percentage of bugs in files that reuse StackOverflow code.Method: We perform our study on 22 open source Android apps. For each project, we mine their source code and use clone detection techniques to identify code that is reused from StackOverflow. We then apply different quantitative and qualitative methods to answer our research questions.Results: Our findings indicate that 1) the amount of reused StackOverflow code varies for different mobile apps, 2) feature additions and enhancements in apps are the main reasons for code reuse from StackOverflow, 3) mid-age and older apps reuse StackOverflow code mostly later on in their project lifetime and 4) that in smaller teams/apps, more experienced developers reuse code, whereas in larger teams/apps, the less experienced developers reuse code the most. Additionally, we found that the percentage of bugs is higher in files after reusing code from StackOverflow.Conclusion: Our results provide insights on the potential impact of code reuse from StackOverflow on mobile apps. Furthermore, these results can benefit the research community in developing new techniques and tools to facilitate and improve code reuse from StackOverflow.

Systematic literature review on agile practices in global software development

Abstract: Context Developing software in distributed development environments exhibits coordination, control and communication challenges. Agile practices, which demand frequent communication and self-organization between remote sites, are increasingly found in global software development (GSD) to mitigate said challenges. Objective We aim to provide detailed insight into what is reported on the successful application of agile practices in GSD from 1999 to 2016 and also identify the most frequently applied agile practices and reported distribution scenarios. We further strive to uncover research opportunities and gaps in the field of agile GSD. Method We build our systematic literature review on top of a previous review, which investigated studies published between 1999 and 2009, and extend the review by years 2010–2016, for which we conduct both a quantitative and a qualitative analysis. Results Our results show that the majority of the cases studied is global and involves complex distribution scenarios with Scrum or combined Scrum/Extreme Programming being the most used agile methods. Key results include that in contrast to 1999–2009, where four Extreme Programming practices were among the ten most frequently used agile practices, in 2010–2016 Scrum is in the center of agile GSD implementations with eight Scrum-based practices in the top ten agile practices used in GSD. Conclusion Agile GSD is a maturing research field with higher quality contributions and a greater variety of publication types and methods from 2010 to 2016 than before from 1999 to 2009. However, researchers need to report full empirical contextual details of their studied cases in order to improve the generalizability of results and allow the future creation of stronger frameworks to drive the implementation of agile practices in GSD.

A benchmark study on the effectiveness of search-based data selection and feature selection for cross project defect prediction

Abstract: Context Previous studies have shown that steered training data or dataset selection can lead to better performance for cross project defect prediction(CPDP). On the other hand, feature selection and data quality are issues to consider in CPDP. Objective We aim at utilizing the Nearest Neighbor (NN)-Filter, embedded in genetic algorithm to produce validation sets for generating evolving training datasets to tackle CPDP while accounting for potential noise in defect labels. We also investigate the impact of using different feature sets. Method We extend our proposed approach, Genetic Instance Selection (GIS), by incorporating feature selection in its setting. We use 41 releases of 11 multi-version projects to assess the performance GIS in comparison with benchmark CPDP (NN-filter and Naive-CPDP) and within project (Cross-Validation(CV) and Previous Releases(PR)). To assess the impact of feature sets, we use two sets of features, SCM+OO+LOC(all) and CK+LOC(ckloc) as well as iterative info-gain subsetting(IG) for feature selection. Results GIS variant with info gain feature selection is significantly better than NN-Filter (all,ckloc,IG) in terms of F1 ( p = v a l u e s ≪ 0.001 , Cohen’s d = { 0.621 , 0.845 , 0.762 } ) and G ( p = v a l u e s ≪ 0.001 , Cohen’s d = { 0.899 , 1.114 , 1.056 } ), and Naive CPDP (all,ckloc,IG) in terms of F1 ( p = v a l u e s ≪ 0.001 , Cohen’s d = { 0.743 , 0.865 , 0.789 } ) and G ( p = v a l u e s ≪ 0.001 , Cohen’s d = { 1.027 , 1.119 , 1.050 } ). Overall, the performance of GIS is comparable to that of within project defect prediction (WPDP) benchmarks, i.e. CV and PR. In terms of multiple comparisons test, all variants of GIS belong to the top ranking group of approaches. Conclusions We conclude that datasets obtained from search based approaches combined with feature selection techniques is a promising way to tackle CPDP. Especially, the performance comparison with the within project scenario encourages further investigation of our approach. However, the performance of GIS is based on high recall in the expense of a loss in precision. Using different optimization goals, utilizing other validation datasets and other feature selection techniques are possible future directions to investigate.

Search-based software library recommendation using multi-objective optimization

Abstract: Context: Software library reuse has significantly increased the productivity of software developers, reduced time-to-market and improved software quality and reusability. However, with the growing number of reusable software libraries in code repositories, finding and adopting a relevant software library becomes a fastidious and complex task for developers.Objective: In this paper, we propose a novel approach called LibFinder to prevent missed reuse opportunities during software maintenance and evolution. The goal is to provide a decision support for developers to easily find "useful" third-party libraries to the implementation of their software systems.Method: To this end, we used the non-dominated sorting genetic algorithm (NSGA-II), a multi-objective search-based algorithm, to find a trade-off between three objectives : 1) maximizing co-usage between a candidate library and the actual libraries used by a given system, 2) maximizing the semantic similarity between a candidate library and the source code of the system, and 3) minimizing the number of recommended libraries.Results: We evaluated our approach on 6083 different libraries from Maven Central super repository that were used by 32,760 client systems obtained from Github super repository. Our results show that our approach outperforms three other existing search techniques and a state-of-the art approach, not based on heuristic search, and succeeds in recommending useful libraries at an accuracy score of 92%, precision of 51% and recall of 68%, while finding the best trade-off between the three considered objectives. Furthermore, we evaluate the usefulness of our approach in practice through an empirical study on two industrial Java systems with developers. Results show that the top 10 recommended libraries was rated by the original developers with an average of 3.25 out of 5.Conclusion: This study suggests that (1) library usage history collected from different client systems and (2) library semantics/content embodied in library identifiers should be balanced together for an efficient library recommendation technique.

A systematic mapping study on game-related methods for software engineering education

Abstract: Context The use of games in software engineering education is not new. However, recent technologies have provided new opportunities for using games and their elements to enhance learning and student engagement. Objective The goal of this paper is twofold. First, we discuss how game-related methods have been used in the context of software engineering education by means of a systematic mapping study. Second, we investigate how these game-related methods support specific knowledge areas from software engineering. By achieving these goals, we aim not only to characterize the state of the art on the use of game-related methods on software engineering education, but also to identify gaps and opportunities for further research. Method We carried out a systematic mapping study to identify primary studies which address the use, proposal or evaluation of games and their elements on software engineering education. We classified primary studies based on type of approaches, learning goals based on software engineering knowledge areas, and specific characteristics of each type of approach. Results We identified 156 primary studies, published between 1974 and June 2016. Most primary studies describe the use of serious games (86) and game development (57) for software engineering education, while Gamification is the least explored method (10). Learning goals of these studies and their development of skills are mostly related to the knowledge areas of “Software Process”, “Software Design”, and “Professional Practices”. Conclusions The use of games in software engineering education is not new. However, there are some knowledge areas where the use of games can still be further explored. Gamification is a new trend and existing research in the field is quite preliminary. We also noted a lack of standardization both in the definition of learning goals and in the classification of game-related methods.

Software teams and their knowledge networks in large-scale software development

Abstract: Empirical findings from ten software teams from two large-scale software development projects in Ericsson and ABB demonstrated that teams receive and share their knowledge with a large number of contacts, including other team members, experts, administrative roles, and support roles.Along with human and organizational capital, social capital and networking are also necessary for participation in large-scale software development, both for novice teams and for mature teams working on complex, unfamiliar, or interdependent tasks.Social capital has the potential to compensate for gaps in human capital (i.e., an individual's knowledge and skills).The team network size and networking behavior depend on the following factors: company experience, employee turnover, team culture, need for networking, and organizational support.Along with investments in training programs, software companies should also cultivate a networking culture to strengthen their social capital and achieve better performance. ContextLarge software development projects involve multiple interconnected teams, often spread around the world, developing complex products for a growing number of customers and users. Succeeding with large-scale software development requires access to an enormous amount of knowledge and skills. Since neither individuals nor teams can possibly possess all the needed expertise, the resource availability in a team's knowledge network, also known as social capital, and effective knowledge coordination become paramount. ObjectiveIn this paper, we explore the role of social capital in terms of knowledge networks and networking behavior in large-scale software development projects. MethodWe conducted a multi-case study in two organizations, Ericsson and ABB, with software development teams as embedded units of analysis. We organized focus groups with ten software teams and surveyed 61 members from these teams to characterize and visualize the teams knowledge networks. To complement the team perspective, we conducted individual interviews with representatives of supporting and coordination roles. Based on survey data, data obtained from focus groups, and individual interviews, we compared the different network characteristics and mechanisms that support knowledge networks. We used social network analysis to construct the team networks, thematic coding to identify network characteristics and context factors, and tabular summaries to identify the trends. ResultsOur findings indicate that social capital and networking are essential for both novice and mature teams when solving complex, unfamiliar, or interdependent tasks. Network size and networking behavior depend on company experience, employee turnover, team culture, need for networking, and organizational support. A number of mechanisms can support the development of knowledge networks and social capital, for example, introduction of formal technical experts, facilitation of communities of practice and adequate communication infrastructure. ConclusionsOur study emphasizes the importance of social capital and knowledge networks. Therefore, we suggest that, along with investments into training programs, software companies should also cultivate a networking culture to strengthen their social capital, a known driver of better performance.

An examination of personality traits and how they impact on software development teams

Abstract: An interactive personality assessment is conducted on an industrial scale.A novel approach is proposed for data visualization.We explore the personality characteristics of software teams.This study indicates that software practitioners become more extroverted. Context: Research has shown that a significant number of software projects fail due to social issues such as team or personality conflicts. However, only a limited number of empirical studies have been undertaken to understand the impact of individuals personalities on software team configurations. These studies suffer from an important limitation as they lack a systematic and rigorous method to relate personality traits of software practitioners and software team structures.Objective: Based on an interactive personality profiling approach, the goal of this study is to reveal the personality traits of software practitioners with an aim to explore effective software team structures.Method: To explore the importance of individuals personalities on software teams, we employed a two-step empirical approach. Firstly, to assess the personality traits of software practitioners, we developed a context-specific survey instrument, which was conducted on 216 participants from a middle-sized software company. Secondly, we propose a novel team personality illustration method to visualize team structures.Results: Study results indicated that effective team structures support teams with higher emotional stability, agreeableness, extroversion, and conscientiousness personality traits.Conclusion: Furthermore, empirical results of the current study show that extroversion trait was more predominant than previously suggested in the literature, which was especially more observable among agile software development teams.

A systematic review on search-based refactoring

Abstract: Context: To find the best sequence of refactorings to be applied in a software artifact is an optimization problem that can be solved using search techniques, in the field called Search-Based Refactoring (SBR). Over the last years, the field has gained importance, and many SBR approaches have appeared, arousing research interest.Objective: The objective of this paper is to provide an overview of existing SBR approaches, by presenting their common characteristics, and to identify trends and research opportunities.Method: A systematic review was conducted following a plan that includes the definition of research questions, selection criteria, a search string, and selection of search engines. 71 primary studies were selected, published in the last sixteen years. They were classified considering dimensions related to the main SBR elements, such as addressed artifacts, encoding, search technique, used metrics, available tools, and conducted evaluation.Results: Some results show that code is the most addressed artifact, and evolutionary algorithms are the most employed search technique. Furthermore, most times, the generated solution is a sequence of refactorings. In this respect, the refactorings considered are usually the ones of the Fowler's Catalog. Some trends and opportunities for future research include the use of models as artifacts, the use of many objectives, the study of the bad smells effect, and the use of hyper-heuristics.Conclusions: We have found many SBR approaches, most of them published recently. The approaches are presented, analyzed, and grouped following a classification scheme. The paper contributes to the SBR field as we identify a range of possibilities that serve as a basis to motivate future researches.

Improved bug localization based on code change histories and bug reports

Abstract: Context Several issues or defects in released software during the maintenance phase are reported to the development team. It is costly and time-consuming for developers to precisely localize bugs. Bug reports and the code change history are frequently used and provide information for identifying fault locations during the software maintenance phase. Objective It is difficult to standardize the style of bug reports written in natural languages to improve the accuracy of bug localization. The objective of this paper is to propose an effective information retrieval-based bug localization method to find suspicious files and methods for resolving bugs. Method In this paper, we propose a novel information retrieval-based bug localization approach, termed Bug Localization using Integrated Analysis (BLIA). Our proposed BLIA integrates analyzed data by utilizing texts, stack traces and comments in bug reports, structured information of source files, and the source code change history. We improved the granularity of bug localization from the file level to the method level by extending previous bug repository data. Results We evaluated the effectiveness of our approach based on experiments using three open-source projects, namely AspectJ, SWT, and ZXing. In terms of the mean average precision, on average our approach improves the metric of BugLocator, BLUiR, BRTracer, AmaLgam and the preliminary version of BLIA by 54%, 42%, 30%, 25% and 15%, respectively, at the file level of bug localization. Conclusion Compared with prior tools, the results showed that BLIA outperforms these other methods. We analyzed the influence of each score of BLIA from various combinations based on the analyzed information. Our proposed enhancement significantly improved the accuracy. To improve the granularity level of bug localization, a new approach at the method level is proposed and its potential is evaluated.

Mutation operators for testing Android apps

Abstract: Context: Due to the widespread use of Android devices, Android applications (apps) have more releases, purchases, and downloads than apps for any other mobile devices. The sheer volume of code in these apps creates significant concerns about the quality of the software. However, testing Android apps is different from testing traditional Java programs due to the unique program structure and new features of apps. Simple testing coverage criteria such as statement coverage are insufficient to assure high quality of Android apps. While researchers show significant interest in finding better Android testing approaches, there is still a lack of effective and usable techniques to evaluate their proposed test selection strategies, and to ensure a reasonable number of effective tests.Objective: As mutation analysis has been found to be an effective way to design tests in other software domains, we hypothesize that it is also a viable solution for Android apps.Method: This paper proposes an innovative mutation analysis approach that is specific for Android apps. We define mutation operators specific to the characteristics of Android apps, such as the extensive use of XML files to specify layout and behavior, the inherent event-driven nature, and the unique Activity lifecycle structure. We also report on an empirical study to evaluate these mutation operators.Results: We have built a tool that uses the novel Android mutation operators to mutate the source code of Android apps, then generates mutants that can be installed and run on Android devices. We evaluated the effectiveness of Android mutation testing through an empirical study on real-world apps. This paper introduces several novel mutation operators based on a fault study of Android apps, presents a significant empirical study with real-world apps, and provides conclusions based on an analysis of the results.Conclusion: The results show that the novel Android mutation operators provide comprehensive testing for Android apps. Additionally, as applying mutation testing to Android apps is still at a preliminary stage, we identify challenges, possibilities, and future research directions to make mutation analysis for mobile apps more effective and efficient.

How developers perceive smells in source code: A replicated study

Abstract: Context. In recent years, smells, also referred to as bad smells, have gained popularity among developers. However, it is still not clear how harmful they are perceived from the developers’ point of view. Many developers talk about them, but only few know what they really are, and even fewer really take care of them in their source code. Objective. The goal of this work is to understand the perceived criticality of code smells both in theory, when reading their description, and in practice. Method. We executed an empirical study as a differentiated external replication of two previous studies. The studies were conducted as surveys involving only highly experienced developers (63 in the first study and 41 in the second one). First the perceived criticality was analyzed by proposing the description of the smells, then different pieces of code infected by the smells were proposed, and finally their ability to identify the smells in the analyzed code was tested. Results. According to our knowledge, this is the largest study so far investigating the perception of code smells with professional software developers. The results show that developers are very concerned about code smells in theory, nearly always considering them as harmful or very harmful (17 out of 23 smells). However, when they were asked to analyze an infected piece of code, only few infected classes were considered harmful and even fewer were considered harmful because of the smell. Conclusions. The results confirm our initial hypotheses that code smells are perceived as more critical in theory but not as critical in practice.

Research patterns and trends in software effort estimation

Abstract: Context Software effort estimation (SEE) is most crucial activity in the field of software engineering. Vast research has been conducted in SEE resulting into a tremendous increase in literature. Thus it is of utmost importance to identify the core research areas and trends in SEE which may lead the researchers to understand and discern the research patterns in large literature dataset. Objective To identify unobserved research patterns through natural language processing from a large set of research articles on SEE published during the period 1996 to 2016. Method A generative statistical method, called Latent Dirichlet Allocation (LDA), applied on a literature dataset of 1178 articles published on SEE. Results As many as twelve core research areas and sixty research trends have been revealed; and the identified research trends have been semantically mapped to associate core research areas. Conclusions This study summarises the research trends in SEE based upon a corpus of 1178 articles. The patterns and trends identified through this research can help in finding the potential research areas.

Analyzing the concept of technical debt in the context of agile software development: A systematic literature review

Abstract: Context Technical debt (TD) is a metaphor that is used to communicate the consequences of poor software development practices to non-technical stakeholders In recent years, it has gained significant attention in agile software development (ASD) Objective The purpose of this study is to analyze and synthesize the state of the art of TD, and its causes, consequences, and management strategies in the context of ASD Research Method Using a systematic literature review (SLR), 38 primary studies, out of 346 studies, were identified and analyzed Results We found five research areas of interest related to the literature of TD in ASD Among those areas, “managing TD in ASD” received the highest attention, followed by “architecture in ASD and its relationship with TD” In addition, eight categories regarding the causes and five categories regarding the consequences of incurring TD in ASD were identified “Focus on quick delivery” and “architectural and design issues” were the most popular causes of incurring TD in ASD “Reduced productivity”, “system degradation” and “increased maintenance cost” were identified as significant consequences of incurring TD in ASD Additionally, we found 12 strategies for managing TD in the context of ASD, out of which “refactoring” and “enhancing the visibility of TD” were the most significant Conclusion The results of this study provide a structured synthesis of TD and its management in the context of ASD as well as potential research areas for further investigation

A risk management framework for distributed agile projects

Abstract: Context Distributed agile development (DAD) approach has been adopted by the software companies for cost and time benefits. However, it causes significant challenges considering the contradicting nature of the agile and distributed development. Objective The objective of this study is to develop a risk management framework that comprises the perceived risks in DAD projects, their causes and the methods used in industry for managing those risks. Method This work is an extension of an exploratory study, wherein, DAD practitioners reported the risks they face in projects and the methods they use for managing those risks. The identified risks were further categorized based on their relevance to different aspects of DAD projects. In this extension, industry practitioners ranked the risks for their impact on DAD projects and rated the methods for the frequency of their use in projects. As the number of risks under each category was large for ranking, they were grouped under the risk areas within each category. The ranking of risk categories, risk areas and risk factors for their impact on DAD projects manifests their importance. The framework includes ranked risks, their causes and the risk management approaches. It was partially implemented in live projects in three different companies and was found to be beneficial. Results The perceived impact of the risk categories, ‘Group Awareness’, ‘External Stakeholder Collaboration’ and ‘Software Development Life Cycle’ on DAD projects has been found to be high and caused by the properties of Distributed Software Development (DSD). The partial validation of the framework in three companies reported the elimination of majority of risk factors and/or reduction in their impact. Conclusion DAD projects provide significant benefits but hold substantial risks due to the contradiction between distributed development and agile practices. The reported framework could effectively minimize the DAD risks in practice.

Model-based security engineering for cyber-physical systems

Abstract: ContextCyber-physical systems (CPSs) have emerged to be the next generation of engineered systems driving the so-called fourth industrial revolution. CPSs are becoming more complex, open and more prone to security threats, which urges security to be engineered systematically into CPSs. Model-Based Security Engineering (MBSE) could be a key means to tackle this challenge via security by design, abstraction, and automation. ObjectiveWe aim at providing an initial assessment of the state of the art in MBSE for CPSs (MBSE4CPS). Specifically, this work focuses on finding out 1) the publication statistics of MBSE4CPS studies; 2) the characteristics of MBSE4CPS studies; and 3) the open issues of MBSE4CPS research. MethodWe conducted a systematic mapping study (SMS) following a rigorous protocol that was developed based on the state-of-the-art SMS and systematic review guidelines. From thousands of relevant publications, we systematically identified 48 primary MBSE4CPS studies for data extraction and synthesis to answer predefined research questions. ResultsSMS results show that for three recent years (2014-2016) the number of primary MBSE4CPS studies has increased significantly. Within the primary studies, the popularity of using Domain-Specific Languages (DSLs) is comparable with the use of the standardised UML modelling notation. Most primary studies do not explicitly address specific security concerns (e.g., confidentiality, integrity) but rather focus on security analyses in general on threats, attacks or vulnerabilities. Few primary studies propose to engineer security solutions for CPSs. Many focus on the early stages of development lifecycle such as security requirement engineering or analysis. ConclusionThe SMS does not only provide the state of the art in MBSE4CPS, but also points out several open issues that would deserve more investigation, e.g., the lack of engineering security solutions for CPSs, limited tool support, too few industrial case studies, and the challenge of bridging DSLs in engineering secure CPSs.

An anomaly detection system based on variable N-gram features and one-class SVM

Abstract: Context: Run-time detection of system anomalies at the host level remains a challenging task. Existing techniques suffer from high rates of false alarms, hindering large-scale deployment of anomaly detection techniques in commercial settings. Objective: To reduce the false alarm rate, we present a new anomaly detection system based on a novel feature extraction technique, which combines the frequency with the temporal information from system call traces, and on one-class support vector machine (OC-SVM) detector.Method: The proposed feature extraction approach starts by segmenting the system call traces into multiple n-grams of variable length and mapping them to fixed-size sparse feature vectors, which are then used to train OC-SVM detectors.Results: The results achieved on a real-world system call dataset show that our feature vectors with up to 6-grams outperform the term vector models (using the most common weighting schemes) proposed in related work. More importantly, our anomaly detection system using OC-SVM with a Gaussian kernel, trained on our feature vectors, achieves a higher-level of detection accuracy (with a lower false alarm rate) than that achieved by Markovian and n-gram based models as well as by the state-of-the-art anomaly detection techniques.Conclusion: The proposed feature extraction approach from traces of events provides new and general data representations that are suitable for training standard one-class machine learning algorithms, while preserving the temporal dependencies among these events.

Software landscape and application visualization for system comprehension with ExplorViz

Abstract: Hierarchical and multi-layer visualization of large software landscapes with ExplorViz, generated from monitoring traces.Reusable design and execution of a controlled experiment comparing a flat landscape visualization to our hierarchical landscape visualization in system comprehension tasks.Controlled experiments for comparing ExplorViz with the Extravis trace visualization approach, for employing physical 3D-printed ExplorViz models, and for exploring 3D ExplorViz models in virtual reality. Display Omitted Context: The number of software applications deployed in organizations is constantly increasing. Those applications often several hundreds form large software landscapes.Objective: The comprehension of such landscapes and their applications is often impeded by, for instance, architectural erosion, personnel turnover, or changing requirements. Therefore, an efficient and effective way to comprehend such software landscapes is required.Method: In our ExplorViz visualization, we introduce hierarchical abstractions aiming at solving system comprehension tasks fast and accurately for large software landscapes. Besides hierarchical visualization on the landscape level, ExplorViz provides multi-level visualization from the landscape to the level of individual applications. The 3D application-level visualization is empirically evaluated with a comparison to the Extravis approach, with physical models and in virtual reality. To evaluate ExplorViz, we conducted four controlled experiments. We provide packages containing all our experimental data to facilitate the verifiability, reproducibility, and further extensibility of our results.Results: We observed a statistical significant increase in task correctness of the hierarchical visualization compared to the flat visualization. The time spent did not show any significant differences. For the comparison with Extravis, we observed that solving program comprehension tasks using ExplorViz leads to a significant increase in correctness and in less or similar time spent. The physical models improved the team-based program comprehension process for specific tasks by initiating gesture-based interaction, but not for all tasks. The participants of our virtual reality experiment with ExplorViz rated the realized gestures for translation, rotation, and selection as highly usable. However, our zooming gesture was less favored.Conclusion: The results backup our claim that our hierarchical and multi-level approach enhances the current state of the art in landscape and application visualization for better software system comprehension, including new forms of interaction with physical models and virtual reality.

Reporting systematic reviews: Some lessons from a tertiary study

Abstract: Context Many of the systematic reviews published in software engineering are related to research or methodological issues and hence are unlikely to be of direct benefit to practitioners or teachers. Those that are relevant to practice and teaching need to be presented in a form that makes their findings usable with minimum interpretation. Objective We have examined a sample of the many systematic reviews that have been published over a period of six years, in order to assess how well these are reported and identify useful lessons about how this might be done. Method We undertook a tertiary study, performing a systematic review of systematic reviews. Our study found 178 systematic reviews published in a set of major software engineering journals over the period 2010–2015. Of these, 37 provided recommendations or conclusions of relevance to education and/or practice and we used the DARE criteria as well as other attributes related to the systematic review process to analyse how well they were reported. Results We have derived a set of 12 ‘lessons’ that could help authors with reporting the outcomes of a systematic review in software engineering. We also provide an associated checklist for use by journal and conference referees. Conclusion There are several areas where better reporting is needed, including quality assessment, synthesis, and the procedures followed by the reviewers. Researchers, practitioners, teachers and journal referees would all benefit from better reporting of systematic reviews, both for clarity and also for establishing the provenance of any findings.

Open source software ecosystems: A Systematic mapping

Abstract: Context: Open source software (OSS) and software ecosystems (SECOs) are two consolidated research areas in software engineering. OSS influences the way organizations develop, acquire, use and commercialize software. SECOs have emerged as a paradigm to understand dynamics and heterogeneity in collaborative software development. For this reason, SECOs appear as a valid instrument to analyze OSS systems. However, there are few studies that blend both topics together. Objective: The purpose of this study is to evaluate the current state of the art in OSS ecosystems (OSSECOs) research, specifically: (a) what the most relevant definitions related to OSSECOs are; (b) what the particularities of this type of SECO are; and (c) how the knowledge about OSSECO is represented. Method: We conducted a systematic mapping following recommended practices. We applied automatic and manual searches on different sources and used a rigorous method to elicit the keywords from the research questions and selection criteria to retrieve the final papers. As a result, 82 papers were selected and evaluated. Threats to validity were identified and mitigated whenever possible. Results: The analysis allowed us to answer the research questions. Most notably, we did the following: (a) identified 64 terms related to the OSSECO and arranged them into a taxonomy; (b) built a genealogical tree to understand the genesis of the OSSECO term from related definitions; (c) analyzed the available definitions of SECO in the context of OSS; and (d) classified the existing modelling and analysis techniques of OSSECOs. Conclusion: As a summary of the systematic mapping, we conclude that existing research on several topics related to OSSECOs is still scarce (e.g., modelling and analysis techniques, quality models, standard definitions, etc.). This situation calls for further investigation efforts on how organizations and OSS communities actually understand OSSECOs.

Who should comment on this pull request? Analyzing attributes for more accurate commenter recommendation in pull-based development

Abstract: Context: The pull-based software development helps developers make contributions flexibly and efficiently. Commenters freely discuss code changes and provide suggestions. Core members make decision of pull requests. Both commenters and core members are reviewers in the evaluation of pull requests. Since some popular projects receive many pull requests, commenters may not notice new pull requests in time, and even ignore appropriate pull requests.Objective: Our objective in this paper is to analyze attributes that affect the precision and recall of commenter prediction, and choose appropriate attributes to build commenter recommendation approach.Method: We collect 19,543 pull requests, 206,664 comments and 4817 commenters from 8 popular projects in GitHub. We build approaches based on different attributes, including activeness, text similarity, file similarity and social relation. We also build composite approaches, including time-based text similarity, time-based file similarity and time-based social relation. The time-based social relation approach is the state-of-the-art approach proposed by Yu etal. Then we compare precision and recall of different approaches.Results: We find that for 8 projects, the activeness based approach achieves the top-3 precision of 0.276, 0.386, 0.389, 0.516, 0.322, 0.572, 0.428, 0.402, and achieves the top-3 recall of 0.475, 0.593, 0.613, 0.66, 0.644, 0.791, 0.714, 0.65, which outperforms approaches based on text similarity, file similarity or social relation by a substantial margin. Moreover, the activeness based approach achieves better precision and recall than composite approaches. In comparison with the state-of-the-art approach, the activeness based approach improves the top-3 precision by 178.788%, 30.41%, 25.08%, 41.76%, 49.07%, 32.71%, 25.15%, 78.67%, and improves the top-3 recall by 196.875%, 36.32%, 29.05%, 46.02%, 43.43%, 27.79%, 25.483%, 79.06% for 8 projects.Conclusion: The activeness is the most important attribute in the commenter prediction. The activeness based approach can be used to improve the commenter recommendation in code review.

Modeling and measuring attributes influencing DevOps implementation in an enterprise using structural equation modeling

Abstract: Context DevOps refer to set of principles that advocate a tight integration between development and operation to achieve higher quality with faster turnaround. It is paramount to assess and measure the DevOps attributes in an enterprise. The literature provides references to these attributes but the detail assessment of these attributes and determination of the maturity of DevOps implementation is still a challenge. Objective This paper provides important insights for practitioners to assess and measure the DevOps attributes using statistical analysis and Two-way assessment. The proposed framework facilitates the detailed assessment of eighteen attributes to identify key independent attributes and measure them to determine the maturity of DevOps implementation in an enterprise. Method The relationship between eighteen attributes was examined; a structural model was established using Exploratory and Confirmatory Factor Analysis, the model was validated using Structural Equation Modelling. Key independent attributes were identified which influences other attributes and overall DevOps implementation. Using Two-way assessment, key independent attributes were measured and the maturity of the DevOps implementation was determined in an enterprise. Results Using Exploratory and Confirmatory Factor Analysis, 18 attributes were categorized under 4 latent variables namely Automation, Source Control, Cohesive Teams and Continuous Delivery. Using Structural Equation Modelling, 10 key independent attributes were determined, that influenced other attributes and overall DevOps implementation. Two-way assessment was applied to measure the key independent attributes and it was found that 4 of these attributes were performing below threshold level. Corrective actions were taken by the management team, and the revised measurement of these attributes demonstrated 40% improvement in the maturity level of DevOps implementation. Conclusion The proposed framework contributes significantly to the field of DevOps by enabling practitioners to conduct the detailed assessment and measurement of DevOps attributes to determine the maturity of DevOps implementation to achieve higher quality.