Showing papers in "Information & Software Technology in 2013"
TL;DR: A systematic review of papers reporting experiences of undertaking SRs and/or discussing techniques that could be used to improve the SR process recommended removing advice to use structured questions to construct search strings and including Advice to use a quasi-gold standard based on a limited manual search to assist the construction of search stings and evaluation of the search process.
Abstract: Context: Many researchers adopting systematic reviews (SRs) have also published papers discussing problems with the SR methodology and suggestions for improving it. Since guidelines for SRs in software engineering (SE) were last updated in 2007, we believe it is time to investigate whether the guidelines need to be amended in the light of recent research. Objective: To identify, evaluate and synthesize research published by software engineering researchers concerning their experiences of performing SRs and their proposals for improving the SR process. Method: We undertook a systematic review of papers reporting experiences of undertaking SRs and/or discussing techniques that could be used to improve the SR process. Studies were classified with respect to the stage in the SR process they addressed, whether they related to education or problems faced by novices and whether they proposed the use of textual analysis tools. Results: We identified 68 papers reporting 63 unique studies published in SE conferences and journals between 2005 and mid-2012. The most common criticisms of SRs were that they take a long time, that SE digital libraries are not appropriate for broad literature searches and that assessing the quality of empirical studies of different types is difficult. Conclusion: We recommend removing advice to use structured questions to construct search strings and including advice to use a quasi-gold standard based on a limited manual search to assist the construction of search stings and evaluation of the search process. Textual analysis tools are likely to be useful for inclusion/exclusion decisions and search string construction but require more stringent evaluation. SE researchers would benefit from tools to manage the SR process but existing tools need independent validation. Quality assessment of studies using a variety of empirical methods remains a major problem.
623 citations
TL;DR: Object-oriented and process metrics have been reported to be more successful in finding faults compared to traditional size and complexity metrics and seem to be better at predicting post-release faults than any static code metrics.
Abstract: ContextSoftware metrics may be used in fault prediction models to improve software quality by predicting fault location. ObjectiveThis paper aims to identify software metrics and to assess their applicability in software fault prediction. We investigated the influence of context on metrics' selection and performance. MethodThis systematic literature review includes 106 papers published between 1991 and 2011. The selected papers are classified according to metrics and context properties. ResultsObject-oriented metrics (49%) were used nearly twice as often compared to traditional source code metrics (27%) or process metrics (24%). Chidamber and Kemerer's (CK) object-oriented metrics were most frequently used. According to the selected studies there are significant differences between the metrics used in fault prediction performance. Object-oriented and process metrics have been reported to be more successful in finding faults compared to traditional size and complexity metrics. Process metrics seem to be better at predicting post-release faults compared to any static code metrics. ConclusionMore studies should be performed on large industrial software systems to find metrics more relevant for the industry and to answer the question as to which metrics should be used in a given context.
437 citations
TL;DR: An extensive systematic literature review of software clones in general and software clone detection in particular calls for an increased awareness of the potential benefits of software clone management, and identifies the need to develop semantic and model clone detection techniques.
Abstract: Context Reusing software by means of copy and paste is a frequent activity in software development. The duplicated code is known as a software clone and the activity is known as code cloning. Software clones may lead to bug propagation and serious maintenance problems. Objective This study reports an extensive systematic literature review of software clones in general and software clone detection in particular. Method We used the standard systematic literature review method based on a comprehensive set of 213 articles from a total of 2039 articles published in 11 leading journals and 37 premier conferences and workshops. Results Existing literature about software clones is classified broadly into different categories. The importance of semantic clone detection and model based clone detection led to different classifications. Empirical evaluation of clone detection tools/techniques is presented. Clone management, its benefits and cross cutting nature is reported. Number of studies pertaining to nine different types of clones is reported. Thirteen intermediate representations and 24 match detection techniques are reported. Conclusion We call for an increased awareness of the potential benefits of software clone management, and identify the need to develop semantic and model clone detection techniques. Recommendations are given for future research.
366 citations
TL;DR: TMS and team tacit knowledge can differentiate between low- and high-performing teams in terms of effectiveness, where more effective teams have a competitive advantage in developing new products and bringing them to market.
Abstract: Context Sharing expert knowledge is a key process in developing software products. Since expert knowledge is mostly tacit, the acquisition and sharing of tacit knowledge along with the development of a transactive memory system (TMS) are significant factors in effective software teams. Objective We seek to enhance our understanding human factors in the software development process and provide support for the agile approach, particularly in its advocacy of social interaction, by answering two questions : How do software development teams acquire and share tacit knowledge ? What roles do tacit knowledge and transactive memory play in successful team performance ? Method A theoretical model describing the process for acquiring and sharing tacit knowledge and development of a TMS through social interaction is presented and a second predictive model addresses the two research questions above. The elements of the predictive model and other demographic variables were incorporated into a larger online survey for software development teams, completed by 46 software SMEs, consisting of 181 individual team members. Results Our results show that team tacit knowledge is acquired and shared directly through good quality social interactions and through the development of a TMS with quality of social interaction playing a greater role than transactive memory. Both TMS and team tacit knowledge predict effectiveness but not efficiency in software teams. Conclusion It is concluded that TMS and team tacit knowledge can differentiate between low- and high-performing teams in terms of effectiveness, where more effective teams have a competitive advantage in developing new products and bringing them to market. As face-to-face social interaction is key, collocated, functionally rich, domain expert teams are advocated rather than distributed teams, though arguably the team manager may be in a separate geographic location provided that there is frequent communication and effective use of issue tracking tools as in agile teams.
154 citations
TL;DR: A novel conceptual framework is developed, using thematic analysis to understand the possible mechanisms behind the main productivity factors perceived by team members through interviews, documentation from retrospectives, and non-participant observation, and the most influential factor was Agile team management.
Abstract: Context: The management of software development productivity is a key issue in software organizations, where the major drivers are lower cost and shorter time-to-market. Agile methods, including Extreme Programming and Scrum, have evolved as ''light'' approaches that simplify the software development process, potentially leading to increased team productivity. However, little empirical research has examined which factors do have an impact on productivity and in what way, when using agile methods. Objective: Our objective is to provide a better understanding of the factors and mediators that impact agile team productivity. Method: We have conducted a multiple-case study for 6months in three large Brazilian companies that have been using agile methods for over 2years. We have focused on the main productivity factors perceived by team members through interviews, documentation from retrospectives, and non-participant observation. Results: We developed a novel conceptual framework, using thematic analysis to understand the possible mechanisms behind such productivity factors. Agile team management was found to be the most influential factor in achieving agile team productivity. At the intra-team level, the main productivity factors were team design (structure and work allocation) and member turnover. At the inter-team level, the main productivity factors were how well teams could be effectively coordinated by proper interfaces and other dependencies and avoiding delays in providing promised software to dependent teams. Conclusion: Teams should be aware of the influence and magnitude of turnover, which has been shown negative for agile team productivity. Team design choices remain an important factor impacting team productivity, even more pronounced on agile teams that rely on teamwork and people factors. The intra-team coordination processes must be adjusted to enable productive work by considering priorities and pace between teams. Finally, the revised conceptual framework for agile team productivity supports further tests through confirmatory studies.
148 citations
TL;DR: This SM provides an overview of existing GUI testing approaches and helps spot areas in the field that require more attention from the research community, and derives the trends in terms of types of articles, sources of information to derive test cases, types of evaluations used in articles, etc.
Abstract: Context: GUI testing is system testing of a software that has a graphical-user interface (GUI) front-end. Because system testing entails that the entire software system, including the user interface, be tested as a whole, during GUI testing, test cases-modeled as sequences of user input events-are developed and executed on the software by exercising the GUI's widgets (e.g., text boxes and clickable buttons). More than 230 articles have appeared in the area of GUI testing since 1991. Objective: In this paper, we study this existing body of knowledge using a systematic mapping (SM). Method: The SM is conducted using the guidelines proposed by Petersen et al. We pose three sets of research questions. We define selection and exclusion criteria. From the initial pool of 230 articles, published in years 1991-2011, our final pool consisted of 136 articles. We systematically develop a classification scheme and map the selected articles to this scheme. Results: We present two types of results. First, we report the demographics and bibliometrics trends in this domain, including: top-cited articles, active researchers, top venues, and active countries in this research area. Moreover, we derive the trends, for instance, in terms of types of articles, sources of information to derive test cases, types of evaluations used in articles, etc. Our second major result is a publicly-accessible repository that contains all our mapping data. We plan to update this repository on a regular basis, making it a ''live'' resource for all researchers. Conclusion: Our SM provides an overview of existing GUI testing approaches and helps spot areas in the field that require more attention from the research community. For example, much work is needed to connect academic model-based techniques with commercially available tools. To this end, studies are needed to compare the state-of-the-art in GUI testing in academic techniques and industrial tools.
140 citations
TL;DR: Combining the power of automated ensembles and locality can lead to competitive results in SEE by analysing such approaches and providing several insights that can be used by future research in the area.
Abstract: ContextEnsembles of learning machines and locality are considered two important topics for the next research frontier on Software Effort Estimation (SEE). ObjectivesWe aim at (1) evaluating whether existing automated ensembles of learning machines generally improve SEEs given by single learning machines and which of them would be more useful; (2) analysing the adequacy of different locality approaches; and getting insight on (3) how to improve SEE and (4) how to evaluate/choose machine learning (ML) models for SEE. MethodA principled experimental framework is used for the analysis and to provide insights that are not based simply on intuition or speculation. A comprehensive experimental study of several automated ensembles, single learning machines and locality approaches, which present features potentially beneficial for SEE, is performed. Additionally, an analysis of feature selection and regression trees (RTs), and an investigation of two tailored forms of combining ensembles and locality are performed to provide further insight on improving SEE. ResultsBagging ensembles of RTs show to perform well, being highly ranked in terms of performance across different data sets, being frequently among the best approaches for each data set and rarely performing considerably worse than the best approach for any data set. They are recommended over other learning machines should an organisation have no resources to perform experiments to chose a model. Even though RTs have been shown to be more reliable locality approaches, other approaches such as k-Means and k-Nearest Neighbours can also perform well, in particular for more heterogeneous data sets. ConclusionCombining the power of automated ensembles and locality can lead to competitive results in SEE. By analysing such approaches, we provide several insights that can be used by future research in the area.
137 citations
TL;DR: This work proposes an approach to predict a product's non-functional properties, based on the product's feature selection, and shows how already little domain knowledge can improve predictions and discusses trade-offs regarding accuracy and the required number of measurements.
Abstract: Context: A software product line is a family of related software products, typically created from a set of common assets Users select features to derive a product that fulfills their needs Users often expect a product to have specific non-functional properties, such as a small footprint or a bounded response time Because a product line may have an exponential number of products with respect to its features, it is usually not feasible to generate and measure non-functional properties for each possible product Objective: Our overall goal is to derive optimal products with respect to non-functional requirements by showing customers which features must be selected Method: We propose an approach to predict a product's non-functional properties based on the product's feature selection We aggregate the influence of each selected feature on a non-functional property to predict a product's properties We generate and measure a small set of products and, by comparing measurements, we approximate each feature's influence on the non-functional property in question As a research method, we conducted controlled experiments and evaluated prediction accuracy for the non-functional properties footprint and main-memory consumption But, in principle, our approach is applicable for all quantifiable non-functional properties Results: With nine software product lines, we demonstrate that our approach predicts the footprint with an average accuracy of 94%, and an accuracy of over 99% on average if feature interactions are known In a further series of experiments, we predicted main memory consumption of six customizable programs and achieved an accuracy of 89% on average Conclusion: Our experiments suggest that, with only few measurements, it is possible to accurately predict non-functional properties of products of a product line Furthermore, we show how already little domain knowledge can improve predictions and discuss trade-offs between accuracy and required number of measurements With this technique, we provide a basis for many reasoning and product-derivation approaches
123 citations
TL;DR: This work investigates the merits of using mixed project data for binary defect prediction and concludes that the extra effort associated with collecting data from other projects is not feasible in terms of practical performance improvement.
Abstract: Context: Defect prediction research mostly focus on optimizing the performance of models that are constructed for isolated projects (i.e. within project (WP)) through retrospective analyses. On the other hand, recent studies try to utilize data across projects (i.e. cross project (CP)) for building defect prediction models for new projects. There are no cases where the combination of within and cross (i.e. mixed) project data are used together. Objective: Our goal is to investigate the merits of using mixed project data for binary defect prediction. Specifically, we want to check whether it is feasible, in terms of defect detection performance, to use data from other projects for the cases (i) when there is an existing within project history and (ii) when there are limited within project data. Method: We use data from 73 versions of 41 projects that are publicly available. We simulate the two above-mentioned cases, and compare the performances of naive Bayes classifiers by using within project data vs. mixed project data. Results: For the first case, we find that the performance of mixed project predictors significantly improves over full within project predictors (p-value<0.001), however the effect size is small (Hedges' g=0.25). For the second case, we found that mixed project predictors are comparable to full within project predictors, using only 10% of available within project data (p-value=0.002, g=0.17). Conclusion: We conclude that the extra effort associated with collecting data from other projects is not feasible in terms of practical performance improvement when there is already an established within project defect predictor using full project history. However, when there is limited project history, e.g. early phases of development, mixed project predictions are justifiable as they perform as good as full within project models.
120 citations
TL;DR: It is argued that there is also an apparent and essential need for evidence-based body of knowledge about different aspects of the adoption of SLRs in SE and the potential impact of such reviews.
Abstract: Background Systematic Literature Reviews (SLRs) have gained significant popularity among Software Engineering (SE) researchers since 2004. Several researchers have also been working on improving the scientific and methodological infrastructure to support SLRs in SE. We argue that there is also an apparent and essential need for evidence-based body of knowledge about different aspects of the adoption of SLRs in SE. Objective The main objective of this research is to empirically investigate the adoption, value, and use of SLRs in SE research from various perspectives. Method We used mixed-methods approach (systematically integrating tertiary literature review, semi-structured interviews and questionnaire-based survey) as it is based on a combination of complementary research methods which are expected to compensate each others’ limitations. Results A large majority of the participants are convinced of the value of using a rigourous and systematic methodology for literature reviews in SE research. However, there are concerns about the required time and resources for SLRs. One of the most important motivators for performing SLRs is new findings and inception of innovative ideas for further research. The reported SLRs are more influential compared to the traditional literature reviews in terms of number of citations. One of the main challenges of conducting SLRs is drawing a balance between methodological rigour and required effort. Conclusions SLR has become a popular research methodology for conducting literature review and evidence aggregation in SE. There is an overall positive perception about this relatively new methodology to SE research. The findings provide interesting insights into different aspects of SLRs. We expect that the findings can provide valuable information to readers about what can be expected from conducting SLRs and the potential impact of such reviews.
114 citations
TL;DR: An Interactive Genetic Algorithm (IGA) is proposed that includes incremental knowledge acquisition and combines it with the existing constraints, such as dependencies and priorities, and outperforms IAHP in terms of effectiveness, efficiency and robustness to decision maker errors.
Abstract: Context: The order in which requirements are implemented affects the delivery of value to the end-user, but it also depends on technical constraints and resource availability. The outcome of requirements prioritization is a total ordering of requirements that best accommodates the various kinds of constraints and priorities. During requirements prioritization, some decisions on the relative importance of requirements or the feasibility of a given implementation order must necessarily resort to a human (e.g., the requirements analyst), possessing the involved knowledge. Objective: In this paper, we propose an Interactive Genetic Algorithm (IGA) that includes incremental knowledge acquisition and combines it with the existing constraints, such as dependencies and priorities. We also assess the performance of the proposed algorithm. Method: The validation of IGA was conducted on a real case study, by comparing the proposed algorithm with the state of the art, interactive prioritization technique Incomplete Analytic Hierarchy Process (IAHP). Results: The proposed method outperforms IAHP in terms of effectiveness, efficiency and robustness to decision maker errors. Conclusion: IGA produces a good approximation of the reference requirements ranking, requiring an acceptable manual effort and tolerating a reasonable human error rate.
TL;DR: A grounded theory of the challenges of using agile methods in traditional enterprise environments is presented based on a Grounded Theory research involving 21 agile practitioners from two large enterprise organizations in the Netherlands, finding that Agile practices can coexist with plan-driven development.
Abstract: Context: While renowned agile methods like XP and Scrum were initially intended for projects with small teams, traditional enterprise environments, i.e. environments where plan-driven development is prevalent, have also become attracted by the promises of a faster time to market through agility. Agile software development methods emphasize lightweight software development. Projects within enterprise environments, however, are typically confronted with a large and complex IT landscape, where mission-critical information is at play whose criticality requires prudence regarding design and development. In many an organization, both approaches are used simultaneously. Objective: Find out which challenges the co-existence of agile methods and plan-driven development brings, and how organizations deal with those challenges. Method: We present a grounded theory of the challenges of using agile methods in traditional enterprise environments, based on a Grounded Theory research involving 21 agile practitioners from two large enterprise organizations in the Netherlands. Results: We organized the challenges under two factors: Increased landscape complexity and Lack of business involvement. For both factors, we identify successful mitigation strategies. These mitigation strategies concern the communication between the agile and traditional part of the organization, and the timing of that communication. Conclusion: Agile practices can coexist with plan-driven development. One should, however, keep in mind the context and take actions to mitigate the challenges incurred.
TL;DR: The results of this systematic mapping study can help researchers to obtain an overview of existing web application testing approaches and indentify areas in the field that require more attention from the research community.
Abstract: Context The Web has had a significant impact on all aspects of our society. As our society relies more and more on the Web, the dependability of web applications has become increasingly important. To make these applications more dependable, for the past decade researchers have proposed various techniques for testing web-based software applications. Our literature search for related studies retrieved 147 papers in the area of web application testing, which have appeared between 2000 and 2011. Objective As this research area matures and the number of related papers increases, it is important to systematically identify, analyze, and classify the publications and provide an overview of the trends in this specialized field. Method We review and structure the body of knowledge related to web application testing through a systematic mapping (SM) study. As part of this study, we pose two sets of research questions, define selection and exclusion criteria, and systematically develop and refine a classification schema. In addition, we conduct a bibliometrics analysis of the papers included in our study. Results Our study includes a set of 79 papers (from the 147 retrieved papers) published in the area of web application testing between 2000 and 2011. We present the results of our systematic mapping study. Our mapping data is available through a publicly-accessible repository. We derive the observed trends, for instance, in terms of types of papers, sources of information to derive test cases, and types of evaluations used in papers. We also report the demographics and bibliometrics trends in this domain, including top-cited papers, active countries and researchers, and top venues in this research area. Conclusion We discuss the emerging trends in web application testing, and discuss the implications for researchers and practitioners in this area. The results of our systematic mapping can help researchers to obtain an overview of existing web application testing approaches and indentify areas in the field that require more attention from the research community.
TL;DR: This work aims at developing a framework that can extend the application of Spectrum-Based Fault Localization to the common situations where test oracles do not exist, and uses a new concept of metamorphic slice resulting from the integration of meetamorphic testing and program slicing.
Abstract: Context: Because of its simplicity and effectiveness, Spectrum-Based Fault Localization (SBFL) has been one of the popular approaches towards fault localization. It utilizes the execution result of failure or pass, and the corresponding coverage information (such as program slice) to estimate the risk of being faulty for each program entity (such as statement). However, all existing SBFL techniques assume the existence of a test oracle to determine the execution result of a test case. But, it is common that test oracles do not exist, and hence the applicability of SBFL has been severely restricted. Objective: We aim at developing a framework that can extend the application of SBFL to the common situations where test oracles do not exist. Method: Our approach uses a new concept of metamorphic slice resulting from the integration of metamorphic testing and program slicing. In SBFL, instead of using the program slice and the result of failure or pass for an individual test case, a metamorphic slice and the result of violation or non-violation of a metamorphic relation are used. Since we need not know the execution result for an individual test case, the existence of a test oracle is no longer a requirement to apply SBFL. Results: An experimental study involving nine programs and three risk evaluation formulas was conducted. The results show that our proposed solution delivers a performance comparable to the performance observed by existing SBFL techniques for the situations where test oracles exist. Conclusion: With respect to the problem that SBFL is only applicable to programs with test oracles, we propose an innovative solution. Our solution is not only intuitively appealing and conceptually feasible, but also practically effective. Consequently, test oracles are no longer mandatory for SBFL, and hence the applicability of SBFL is significantly extended.
TL;DR: Assessing methods for handling variability in quality attributes of service-based systems, collecting evidence about current research that suggests implications for practice, and identifying open problems and areas for improvement are aimed at.
Abstract: Context: Variability is the ability of a software artifact (e.g., a system, component) to be adapted for a specific context, in a preplanned manner. Variability not only affects functionality, but also quality attributes (e.g., security, performance). Service-based software systems consider variability in functionality implicitly by dynamic service composition. However, variability in quality attributes of service-based systems seems insufficiently addressed in current design practices. Objective: We aim at (a) assessing methods for handling variability in quality attributes of service-based systems, (b) collecting evidence about current research that suggests implications for practice, and (c) identifying open problems and areas for improvement. Method: A systematic literature review with an automated search was conducted. The review included studies published between the year 2000 and 2011. We identified 46 relevant studies. Results: Current methods focus on a few quality attributes, in particular performance and availability. Also, most methods use formal techniques. Furthermore, current studies do not provide enough evidence for practitioners to adopt proposed approaches. So far, variability in quality attributes has mainly been studied in laboratory settings rather than in industrial environments. Conclusions: The product line domain as the domain that traditionally deals with variability has only little impact on handling variability in quality attributes. The lack of tool support, the lack of practical research and evidence for the applicability of approaches to handle variability are obstacles for practitioners to adopt methods. Therefore, we suggest studies in industry (e.g., surveys) to collect data on how practitioners handle variability of quality attributes in service-based systems. For example, results of our study help formulate hypotheses and questions for such surveys. Based on needs in practice, new approaches can be proposed.
TL;DR: The role of code smells on the overall system maintainability is relatively minor, thus complementary approaches are needed to achieve more comprehensive assessments of maintainability.
Abstract: Context: Code smells are indicators of poor coding and design choices that can cause problems during software maintenance and evolution. Objective: This study is aimed at a detailed investigation to which extent problems in maintenance projects can be predicted by the detection of currently known code smells. Method: A multiple case study was conducted, in which the problems faced by six developers working on four different Java systems were registered on a daily basis, for a period up to four weeks. Where applicable, the files associated to the problems were registered. Code smells were detected in the pre-maintenance version of the systems, using the tools Borland Together and InCode. In-depth examination of quantitative and qualitative data was conducted to determine if the observed problems could be explained by the detected smells. Results: From the total set of problems, roughly 30% percent were related to files containing code smells. In addition, interaction effects were observed amongst code smells, and between code smells and other code characteristics, and these effects led to severe problems during maintenance. Code smell interactions were observed between collocated smells (i.e., in the same file), and between coupled smells (i.e., spread over multiple files that were coupled). Conclusions: The role of code smells on the overall system maintainability is relatively minor, thus complementary approaches are needed to achieve more comprehensive assessments of maintainability. Moreover, to improve the explanatory power of code smells, interaction effects amongst collocated smells and coupled smells should be taken into account during analysis.
TL;DR: There is a need for more software robustness research on real-world, industrial systems and on software development phases other than testing and design, in particular on requirements engineering.
Abstract: Context: With the increased use of software for running key functions in modern society it is of utmost importance to understand software robustness and how to support it. Although there have been many contributions to the field there is a lack of a coherent and summary view. Objective: To address this issue, we have conducted a literature review in the field of robustness. Method: This review has been conducted by following guidelines for systematic literature reviews. Systematic reviews are used to find and classify all existing and available literature in a certain field. Results: From 9193 initial papers found in three well-known research databases, the 144 relevant papers were extracted through a multi-step filtering process with independent validation in each step. These papers were then further analyzed and categorized based on their development phase, domain, research, contribution and evaluation type. The results indicate that most existing results on software robustness focus on verification and validation of Commercial of the shelf (COTS) or operating systems or propose design solutions for robustness while there is a lack of results on how to elicit and specify robustness requirements. The research is typically solution proposals with little to no evaluation and when there is some evaluation it is primarily done with small, toy/academic example systems. Conclusion: We conclude that there is a need for more software robustness research on real-world, industrial systems and on software development phases other than testing and design, in particular on requirements engineering.
TL;DR: This study investigates Software Evolution Visualization approaches, collecting evidence about how SEV research is structured, synthesizing current evidence on the goals of the proposed approaches and identifying key challenges for its use in practice.
Abstract: Background: Software evolution is an important topic in software engineering. It generally deals with large amounts of data, as one must look at whole project histories as opposed to their current snapshot. Software visualization is the field of software engineering that aims to help people to understand software through the use of visual resources. It can be effectively used to analyze and understand the large amount of data produced during software evolution. Objective: This study investigates Software Evolution Visualization (SEV) approaches, collecting evidence about how SEV research is structured, synthesizing current evidence on the goals of the proposed approaches and identifying key challenges for its use in practice. Methods: A mapping study was conducted to analyze how the SEV area is structured. Selected primary studies were classified and analyzed with respect to nine research questions. Results: SEV has been used for many different purposes, especially for change comprehension, change prediction and contribution analysis. The analysis identified gaps in the studies with respect to their goals, strategies and approaches. It also pointed out to a widespread lack of empirical studies in the area. Conclusion: Researchers have proposed many SEV approaches during the past years, but some have failed to clearly state their goals, tie them back to concrete problems, or formally validate their usefulness. The identified gaps indicate that there still are many opportunities to be explored in the area.
TL;DR: This work empirically studies the relationship between internal class quality attributes (size, cohesion, and coupling) and an external quality attribute (class maintainability) and constructs models based on the selected internal attributes to predict class maintainability.
Abstract: Context: Class maintainability is the likelihood that a class can be easily modified. Before releasing an object-oriented software system, it is impossible to know with certainty when, where, how, and how often a class will be modified. At that stage, this likelihood can be estimated using the internal quality attributes of a class, which include cohesion, coupling, and size. To reduce the future class maintenance efforts and cost, developers are encouraged to carefully test and well document low maintainability classes before releasing the object-oriented system. Objective: We empirically study the relationship between internal class quality attributes (size, cohesion, and coupling) and an external quality attribute (class maintainability). Using statistical techniques, we also construct models based on the selected internal attributes to predict class maintainability. Method: We consider classes of three open-source systems. For each class, we account for two actual maintainability indicators, the number of revised lines of code and the number of revisions in which the class was involved. Using 19 internal quality measures, we empirically explore the impact of size, cohesion, and coupling on class maintainability. We also empirically investigate the abilities of the measures, considered both individually and combined, to estimate class maintainability. Statistically based prediction models are constructed and validated. Results: Our results demonstrate that classes with better qualities (i.e., higher cohesion values and lower size and coupling values) have better maintainability (i.e., are more likely to be easily modified) than those of worse qualities. Most of the considered measures are shown to be predictors of the considered maintainability indicators to some degree. The abilities of the considered internal quality measures to predict class maintainability are improved when the measures are combined using optimized multivariate statistical models. Conclusion: The prediction models can help software engineers locate classes with low maintainability. These classes must be carefully tested and well documented.
TL;DR: A number of knowledge-based approaches, including knowledge capture and representation, reuse, sharing, recovery, and reasoning, have been employed in a spectrum of architecting activities, among which embedded software has received the most attention.
Abstract: Context: Knowledge management technologies have been employed across software engineering activities for more than two decades. Knowledge-based approaches can be used to facilitate software architecting activities (e.g., architectural evaluation). However, there is no comprehensive understanding on how various knowledge-based approaches (e.g., knowledge reuse) are employed in software architecture. Objective: This work aims to collect studies on the application of knowledge-based approaches in software architecture and make a classification and thematic analysis on these studies, in order to identify the gaps in the existing application of knowledge-based approaches to various architecting activities, and promising research directions. Method: A systematic mapping study is conducted for identifying and analyzing the application of knowledge-based approaches in software architecture, covering the papers from major databases, journals, conferences, and workshops, published between January 2000 and March 2011. Results: Fifty-five studies were selected and classified according to the architecting activities they contribute to and the knowledge-based approaches employed. Knowledge capture and representation (e.g., using an ontology to describe architectural elements and their relationships) is the most popular approach employed in architecting activities. Knowledge recovery (e.g., documenting past architectural design decisions) is an ignored approach that is seldom used in software architecture. Knowledge-based approaches are mostly used in architectural evaluation, while receive the least attention in architecture impact analysis and architectural implementation. Conclusions: The study results show an increased interest in the application of knowledge-based approaches in software architecture in recent years. A number of knowledge-based approaches, including knowledge capture and representation, reuse, sharing, recovery, and reasoning, have been employed in a spectrum of architecting activities. Knowledge-based approaches have been applied to a wide range of application domains, among which ''Embedded software'' has received the most attention.
TL;DR: A set of automated analysis mechanisms to support the requirements engineers to detect and analyze modelling errors in contextual requirements models to avoid developing unusable and unwanted functionalities and functionalities which lead to conflicts when they operate together.
Abstract: Context: The environment in which the system operates, its context, is variable. The autonomous ability of a software to adapt to context has to be planned since the requirements analysis stage as a strong mutual influence between requirements and context does exist. On the one hand, context is a main factor to decide whether to activate a requirement, the applicable alternatives to meet an activated requirement as well as their qualities. On the other hand, the system actions to reach requirements could cause changes in the context. Objectives: Modelling the relationship between requirements and context is a complex task and developing error-free models is hard to achieve without an automated support. The main objective of this paper is to develop a set of automated analysis mechanisms to support the requirements engineers to detect and analyze modelling errors in contextual requirements models. Method: We study the analysis of the contextual goal model which is a requirements model that weaves together the variability of both context and requirements. Goal models are used during the early stages of software development and, thus, our analysis detects errors early in the development process. We develop two analysis mechanisms to detect two kinds of modelling errors. The first mechanism concerns the detection of inconsistent specification of contexts in a goal model. The second concerns the detection of conflicting context changes that arise as a consequence of the actions performed by the system to meet different requirements simultaneously. We support our analysis with a CASE tool and provide a systematic process that guides the construction and analysis of contextual goal models. We illustrate and evaluate our framework via a case study on a smart-home system for supporting the life of people having dementia problems. Results: The evaluation showed a significant ability of our analysis mechanisms to detect errors which were not notable by requirements engineers. Moreover, the evaluation showed acceptable performance of these mechanisms when processing up to medium-sized contextual goal models. The modelling constructs which we proposed as an input to enable the analysis were found easy to understand and capture. Conclusions: Our developed analysis for the detection of inconsistency and conflicts in contextual goal models is an essential step for the entire system correctness. It avoids us developing unusable and unwanted functionalities and functionalities which lead to conflicts when they operate together. Further research to improve our analysis to scale with large-sized models and to consider other kinds of errors is still needed.
TL;DR: This paper proposes a model-based approach that enables software engineers to assess their design solutions for software product lines in the early stages of development and discusses how the analysis time can be surprisingly reduced by applying parametric model checking instead of classic model checking.
Abstract: Evaluating quality attributes of a design model in the early stages of development can significantly reduce the cost and risks of developing a low quality product. To make this possible, software designers should be able to predict quality attributes by reasoning on a model of the system under development. Although there exists a variety of quality-driven analysis techniques for software systems, only a few work address software product lines. This paper describes how probabilistic model checking techniques and tools can be used to verify non-functional properties of different configurations of a software product line. We propose a model-based approach that enables software engineers to assess their design solutions for software product lines in the early stages of development. Furthermore, we discuss how the analysis time can be surprisingly reduced by applying parametric model checking instead of classic model checking. The results show that the parametric approach is able to substantially alleviate the verification time and effort required to analyze non-functional properties of software product lines.
TL;DR: The findings indicated that carefully selecting team member for software teams is likely to positively influence the projects in which these teams participate, and it seems that the type of development method used can moderate this influence.
Abstract: Context The internal composition of a work team is an important antecedent of team performance and the criteria used to select team members play an important role in determining team composition. However, there are only a handful of empirical studies about the use of team building criteria in the software industry. Objective The goal of this article is to identify criteria used in industrial practice to select members of a software project team, and to look for relationships between the use of these criteria and project success. In addition, we expect to contribute with findings about the use of replication in empirical studies involving human factors in software engineering. Method Our research was based on an iterative mix-method, replication strategy. In the first iteration, we used qualitative research to identify team-building criteria interviewing software project managers from industry. Then, we performed a cross-sectional survey to assess the correlations of the use of these criteria and project success. In the second iteration, we used the results of a systematic mapping study to complement the set of team building criteria. Finally, we performed a replication of the survey research with variations to verify and improve the results. Results Our results showed that the consistent use team building criteria correlated significantly with project success, and the criteria related to human factors, such as personality and behavior, presented the strongest correlations. The results of the replication did not reproduce the results of the original survey with respect to the correlations between criteria and success goals. Nevertheless, the variations in the design and the difference in the sample of projects allowed us to conclude that the two results were compatible, increasing our confidence on the existence of the correlations. Conclusion Our findings indicated that carefully selecting team member for software teams is likely to positively influence the projects in which these teams participate. Besides, it seems that the type of development method used can moderate (increase or decrease) this influence. In addition, our study showed that the choice of sampling technique is not straightforward given the many interacting factors affecting this type of investigation.
TL;DR: In this paper, a systematic review of the literature in automated requirements elicitation is performed, which aims to capture the current state of the art and derive future research directions by identifying gaps in the existing body of knowledge and through relating existing works to each other.
Abstract: Context: In large software development projects a huge number of unstructured text documents from various stakeholders becomes available and needs to be analyzed and transformed into structured requirements. This elicitation process is known to be time-consuming and error-prone when performed manually by a requirements engineer. Consequently, substantial research has been done to automate the process through a plethora of tools and technologies. Objective: This paper aims to capture the current state of automated requirements elicitation and derive future research directions by identifying gaps in the existing body of knowledge and through relating existing works to each other. More specifically, we are investigating the following research question: What is the state of the art in research covering tool support for automated requirements elicitation from natural language documents? Method: A systematic review of the literature in automated requirements elicitation is performed. Identified works are categorized using an analysis framework comprising tool categories, technological concepts and evaluation approaches. Furthermore, the identified papers are related to each other through citation analysis to trace the development of the research field. Results: We identified, categorized and related 36 relevant publications. Summarizing the observations we made, we propose future research to (1) investigate alternative elicitation paradigms going beyond a pure automation approach (2) compare the effects of different types of knowledge on elicitation results (3) apply comparative evaluation methods and multi-dimensional evaluation measures and (4) strive for a closer integration of research activities across the sub-fields of automatic requirements elicitation. Conclusion: Through the results of our paper, we intend to contribute to the Requirements Engineering body of knowledge by (1) conceptualizing an analysis framework for works in the area of automated requirements elicitation, going beyond former classifications (2) providing an extensive overview and categorization of existing works in this area (3) formulating concise directions for future research.
TL;DR: Assessing the contribution of agile methodology, agile-specific team methods, and general team factors in the performance of software teams found that the client and team foci of the XP method are its critical active ingredients.
Abstract: ContextDeveloping a theory of agile technology, in combination with empirical work, must include assessing its performance effects, and whether all or some of its key ingredients account for any performance advantage over traditional methods. Given the focus on teamwork, is the agile technology what really matters, or do general team factors, such as cohesion, primarily account for a team's success? Perhaps the more specific software engineering team factors, for example the agile development method's collective ownership and code management, are decisive. ObjectiveTo assess the contribution of agile methodology, agile-specific team methods, and general team factors in the performance of software teams. MethodWe studied 40 small-scale software development teams which used Extreme Programming (XP). We measured (1) the teams' adherence to XP methods, (2) their use of XP-specific team practices, and (3) standard team attributes, as well as the quality of the project's outcomes. We used Williams et al.'s (2004a) 33] Shodan measures of XP methods, and regression analysis. ResultsAll three types of variables are associated with the project's performance. Teamworking is important but it is the XP-specific team factor (continuous integration, coding standards, and collective code ownership) that is significant. Only customer planning (release planning/planning game, customer access, short releases, and stand-up meeting) is positively related to performance. A negative relationship between foundations (automated unit tests, customer acceptance tests, test-first design, pair programming, and refactoring) is found and is moderated by craftsmanship (sustainable pace, simple design, and metaphor/system of names). Of the general team factors only cooperation is related to performance. Cooperation mediates the relationship between the XP-specific team factor and performance. ConclusionClient and team foci of the XP method are its critical active ingredients.
TL;DR: There is evidence to indicate that the repair based algorithm introduced in this paper is a suitable technique for extending previous work on requirements optimization to handle the requirement interaction constraints inherent in requirement interactions arising from dependencies, and, or, precedence, cost- and value-based constraints.
Abstract: Context: Requirements optimization has been widely studied in the Search Based Software Engineering (SBSE) literature. However, previous approaches have not handled requirement interactions, such as the dependencies that may exist between requirements, and, or, precedence, cost- and value-based constraints. Objective: To introduce and evaluate a Multi-Objective Search Based Requirements Selection technique, using chromosome repair and to evaluate it on both synthetic and real world data sets, in order to assess its effectiveness and scalability. The paper extends and improves upon our previous conference paper on requirements interaction management. Method: The popular multi-objective evolutionary algorithm NSGA-II was used to produce baseline data for each data set in order to determine how many solutions on the Pareto front fail to meet five different requirement interaction constraints. The results for this baseline data are compared to those obtained using the archive based approach previously studied and the repair based approach introduced in this paper. Results: The repair based approach was found to produce more solutions on the Pareto front and better convergence and diversity of results than the previously studied NSGA-II and archive-based NSGA-II approaches based on Kruskal-Wallis test in most cases. The repair based approach was also found to scale almost as well as the previous approach. Conclusion: There is evidence to indicate that the repair based algorithm introduced in this paper is a suitable technique for extending previous work on requirements optimization to handle the requirement interaction constraints inherent in requirement interactions arising from dependencies, and, or, precedence, cost- and value-based constraints.
TL;DR: Overall, it is found that the evidence-based process as presented in this study helps in technology transfer of research results to industry, but at the same time some challenges lie ahead.
Abstract: Context: Evidence-based software engineering (EBSE) provides a process for solving practical problems based on a rigorous research approach. The primary focus so far was on mapping and aggregating evidence through systematic reviews. Objectives: We extend existing work on evidence-based software engineering by using the EBSE process in an industrial case to help an organization to improve its automotive testing process. With this we contribute in (1) providing experiences on using evidence based processes to analyze a real world automotive test process and (2) provide evidence of challenges and related solutions for automotive software testing processes. Methods: In this study we perform an in-depth investigation of an automotive test process using an extended EBSE process including case study research (gain an understanding of practical questions to define a research scope), systematic literature review (identify solutions through systematic literature), and value stream mapping (map out an improved automotive test process based on the current situation and improvement suggestions identified). These are followed by reflections on the EBSE process used. Results: In the first step of the EBSE process we identified 10 challenge areas with a total of 26 individual challenges. For 15 out of those 26 challenges our domain specific systematic literature review identified solutions. Based on the input from the challenges and the solutions, we created a value stream map of the current and future process. Conclusions: Overall, we found that the evidence-based process as presented in this study helps in technology transfer of research results to industry, but at the same time some challenges lie ahead (e.g. koping systematic reviews to focus more on concrete industry problems, and understanding strategies of conducting EBSE with respect to effort and quality of the evidence). (C) 2013 Elsevier B.V. All rights reserved. (Less)
TL;DR: This paper proposes a novel approach to aid suppliers in creating the evidence necessary for certification according to standards based on Model-Driven Engineering and addresses the challenges of using certification standards while providing assistance with compliance.
Abstract: Context: Many safety-critical systems are subject to safety certification as a way to provide assurance that these systems cannot unduly harm people, property or the environment. Creating the requisite evidence for certification can be a challenging task due to the sheer size of the textual standards based on which certification is performed and the amenability of these standards to subjective interpretation. Objective: This paper proposes a novel approach to aid suppliers in creating the evidence necessary for certification according to standards. The approach is based on Model-Driven Engineering (MDE) and addresses the challenges of using certification standards while providing assistance with compliance. Method: Given a safety standard, a conceptual model is built that provides a succinct and explicit interpretation of the standard. This model is then used to create a UML profile that helps system suppliers in relating the concepts of the safety standard to those of the application domain, in turn enabling the suppliers to demonstrate how their system development artifacts comply with the standard. Results: We provide a generalizable and tool-supported solution to support the verification of compliance to safety standards. Empirical validation of the work is presented via an industrial case study that shows how the concepts of a sub-sea production control system can be aligned with the evidence requirements of the IEC61508 standard. A subsequent survey examines the perceptions of practitioners about the solution. Conclusion: The case study indicates that the supplier company where the study was performed found the approach useful in helping them prepare for certification of their software. The survey indicates that practitioners found our approach easy to understand and that they would be willing to adopt it in practice. Since the IEC61508 standard applies to multiple domains, these results suggest wider applicability and usefulness of our work.
TL;DR: This paper aims to provide an alternative or complementary solution to existing taint analyzers by proposing static code attributes that can be used to predict specific program statements, rather than software components, which are likely to be vulnerable to SQLI or XSS.
Abstract: Context: SQL injection (SQLI) and cross site scripting (XSS) are the two most common and serious web application vulnerabilities for the past decade. To mitigate these two security threats, many vulnerability detection approaches based on static and dynamic taint analysis techniques have been proposed. Alternatively, there are also vulnerability prediction approaches based on machine learning techniques, which showed that static code attributes such as code complexity measures are cheap and useful predictors. However, current prediction approaches target general vulnerabilities. And most of these approaches locate vulnerable code only at software component or file levels. Some approaches also involve process attributes that are often difficult to measure. Objective: This paper aims to provide an alternative or complementary solution to existing taint analyzers by proposing static code attributes that can be used to predict specific program statements, rather than software components, which are likely to be vulnerable to SQLI or XSS. Method: From the observations of input sanitization code that are commonly implemented in web applications to avoid SQLI and XSS vulnerabilities, in this paper, we propose a set of static code attributes that characterize such code patterns. We then build vulnerability prediction models from the historical information that reflect proposed static attributes and known vulnerability data to predict SQLI and XSS vulnerabilities. Results: We developed a prototype tool called PhpMinerI for data collection and used it to evaluate our models on eight open source web applications. Our best model achieved an averaged result of 93% recall and 11% false alarm rate in predicting SQLI vulnerabilities, and 78% recall and 6% false alarm rate in predicting XSS vulnerabilities. Conclusion: The experiment results show that our proposed vulnerability predictors are useful and effective at predicting SQLI and XSS vulnerabilities.