Showing papers in "Information & Software Technology in 2009"

Systematic literature reviews in software engineering - A systematic literature review

Abstract: Background: In 2004 the concept of evidence-based software engineering (EBSE) was introduced at the ICSE04 conference. Aims: This study assesses the impact of systematic literature reviews (SLRs) which are the recommended EBSE method for aggregating evidence. Method: We used the standard systematic literature review method employing a manual search of 10 journals and 4 conference proceedings. Results: Of 20 relevant studies, eight addressed research trends rather than technique evaluation. Seven SLRs addressed cost estimation. The quality of SLRs was fair with only three scoring less than 2 out of 4. Conclusions: Currently, the topic areas covered by SLRs are limited. European researchers, particularly those at the Simula Laboratory appear to be the leading exponents of systematic literature reviews. The series of cost estimation SLRs demonstrate the potential value of EBSE for synthesising evidence and making it available to practitioners.

A systematic review of search-based testing for non-functional system properties

Abstract: Search-based software testing is the application of metaheuristic search techniques to generate software tests. The test adequacy criterion is transformed into a fitness function and a set of solutions in the search space are evaluated with respect to the fitness function using a metaheuristic search technique. The application of metaheuristic search techniques for testing is promising due to the fact that exhaustive testing is infeasible considering the size and complexity of software under test. Search-based software testing has been applied across the spectrum of test case design methods; this includes white-box (structural), black-box (functional) and grey-box (combination of structural and functional) testing. In addition, metaheuristic search techniques have also been applied to test non-functional properties. The overall objective of undertaking this systematic review is to examine existing work into non-functional search-based software testing (NFSBST). We are interested in types of non-functional testing targeted using metaheuristic search techniques, different fitness functions used in different types of search-based non-functional testing and challenges in the application of these techniques. The systematic review is based on a comprehensive set of 35 articles obtained after a multi-stage selection process and have been published in the time span 1996-2007. The results of the review show that metaheuristic search techniques have been applied for non-functional testing of execution time, quality of service, security, usability and safety. A variety of metaheuristic search techniques are found to be applicable for non-functional testing including simulated annealing, tabu search, genetic algorithms, ant colony methods, grammatical evolution, genetic programming (and its variants including linear genetic programming) and swarm intelligence methods. The review reports on different fitness functions used to guide the search for each of the categories of execution time, safety, usability, quality of service and security; along with a discussion of possible challenges in the application of metaheuristic search techniques.

Evaluating and selecting software packages: A review

Abstract: Evaluating and selecting software packages that meet an organization's requirements is a difficult software engineering process. Selection of a wrong software package can turn out to be costly and adversely affect business processes. The aim of this paper is to provide a basis to improve the process of evaluation and selection of the software packages. This paper reports a systematic review of papers published in journals and conference proceedings. The review investigates methodologies for selecting software packages, software evaluation techniques, software evaluation criteria, and systems that support decision makers in evaluating software packages. The key findings of the review are: (1) analytic hierarchy process has been widely used for evaluation of the software packages, (2) there is lack of a common list of generic software evaluation criteria and its meaning, and (3) there is need to develop a framework comprising of software selection methodology, evaluation technique, evaluation criteria, and system to assist decision makers in software selection.

Supporting user-oriented analysis for multi-view domain-specific visual languages

Abstract: The integration of usable and flexible analysis support in modelling environments is a key success factor in Model-Driven Development In this paradigm, models are the core asset from which code is automatically generated, and thus ensuring model correctness is a fundamental quality control activity For this purpose, a common approach is to transform the system models into formal semantic domains for verification However, if the analysis results are not shown in a proper way to the end-user (eg in terms of the original language) they may become useless In this paper we present a novel DSVL called BaVeL that facilitates the flexible annotation of verification results obtained in semantic domains to different formats, including the context of the original language BaVeL is used in combination with a consistency framework, providing support for all steps in a verification process: acquisition of additional input data, transformation of the system models into semantic domains, verification, and flexible annotation of analysis results The approach has been validated analytically by the cognitive dimensions framework, and empirically by its implementation and application to several DSVLs Here we present a case study of a notation in the area of Digital Libraries, where the analysis is performed by transformations into Petri nets and a process algebra

How do personality, team processes and task characteristics relate to job satisfaction and software quality?

Abstract: This article analyses the relationships between personality, team processes, task characteristics, product quality and satisfaction in software development teams. The data analysed here were gathered from a sample of 35 teams of students (105 participants). These teams applied an adaptation of an agile methodology, eXtreme Programming (XP), to develop a software product. We found that the teams with the highest job satisfaction are precisely the ones whose members score highest for the personality factors agreeableness and conscientiousness. The satisfaction levels are also higher when the members can decide how to develop and organize their work. On the other hand, the level of satisfaction and cohesion drops the more conflict there is between the team members. Finally, the teams exhibit a significant positive correlation between the personality factor extraversion and software product quality.

The effectiveness of pair programming: A meta-analysis

Abstract: Several experiments on the effects of pair versus solo programming have been reported in the literature. We present a meta-analysis of these studies. The analysis shows a small significant positive overall effect of pair programming on quality, a medium significant positive overall effect on duration, and a medium significant negative overall effect on effort. However, between-study variance is significant, and there are signs of publication bias among published studies on pair programming. A more detailed examination of the evidence suggests that pair programming is faster than solo programming when programming task complexity is low and yields code solutions of higher quality when task complexity is high. The higher quality for complex tasks comes at a price of considerably greater effort, while the reduced completion time for the simpler tasks comes at a price of noticeably lower quality. We conclude that greater attention should be given to moderating factors on the effects of pair programming.

A systematic literature review to identify and classify software requirement errors

Abstract: Most software quality research has focused on identifying faults (i.e., information is incorrectly recorded in an artifact). Because software still exhibits incorrect behavior, a different approach is needed. This paper presents a systematic literature review to develop taxonomy of errors (i.e., the sources of faults) that may occur during the requirements phase of software lifecycle. This taxonomy is designed to aid developers during the requirement inspection process and to improve overall software quality. The review identified 149 papers from the software engineering, psychology and human cognition literature that provide information about the sources of requirements faults. A major result of this paper is a categorization of the sources of faults into a formal taxonomy that provides a starting point for future research into error-based approaches to improving software quality.

Customizing ISO 9126 quality model for evaluation of B2B applications

Abstract: A software quality model acts as a framework for the evaluation of attributes of an application that contribute to the software quality. In this paper, a quality model is presented for evaluation of B2B applications. First, the most well-known quality models are studied, and reasons for using ISO 9126 quality model as the basis are discussed. This model, then, is customized in accordance with special characteristics of B2B applications. The customization is done by extracting the quality factors from web applications and B2B e-commerce applications, weighting these factors from the viewpoints of both developers and end users, and adding them to the model. Finally, as a case study, ISACO portal is evaluated by the proposed model.

A systematic review of UML model consistency management

Abstract: Information System (IS) development has been beset by consistency problems since its infancy. These problems are greater still in UML software development, and are principally caused by the existence of multiple views (models) for the same system, and may involve potentially contradictory system specifications. Since a considerable amount of work takes place within the scope of model consistency management, this paper presents a systematic literature review (SLR) which was carried out to discover the various current model consistency conceptions, proposals, problems and solutions provided. To do this, a total of 907 papers related to UML model consistency published in literature and extracted from the most relevant scientific sources (IEEE Computer Society, ACM Digital Library, Google Scholar, ScienceDirect, and the SCOPUS Database) were considered, of which 42 papers were eventually analyzed. This systematic literature review resulted in the identification of the current state-of-the-art with regard to UML model consistency management research along with open issues, trends and future research within this scope. A formal approach for the handling of inconsistency problems which fulfils the identified limitations is also briefly presented.

Models of motivation in software engineering

Abstract: Motivation in software engineering is recognized as a key success factor for software projects, but although there are many papers written about motivation in software engineering, the field lacks a comprehensive overview of the area. In particular, several models of motivation have been proposed, but they either rely heavily on one particular model (the job characteristics model), or are quite disparate and difficult to combine. Using the results from our previous systematic literature review (SLR), we constructed a new model of motivation in software engineering. We then compared this new model with existing models and refined it based on this comparison. This paper summarises the SLR results, presents the important existing models found in the literature and explains the development of our new model of motivation in software engineering.

Empirical investigation of refactoring effect on software quality

Abstract: Developers and designers always strive for quality software. Quality software tends to be robust, reliable and easy to maintain, and thus reduces the cost of software development and maintenance. Several methods have been applied to improve software quality. Refactoring is one of those methods. The goal of this paper is to validate/invalidate the claims that refactoring improves software quality. We focused this study on different external quality attributes, which are adaptability, maintainability, understandability, reusability, and testability. We found that refactoring does not necessarily improve these quality attributes.

A framework for understanding creativity in requirements engineering

Abstract: Creativity is important in the discovery and analysis of user and business requirements to achieve innovative uses of information and communication technologies. This paper builds a theoretical framework for understanding creativity in requirements engineering. The framework provides a systematic means of understanding creativity in requirements engineering and comprises five elements (product, process, domain, people and socio-organisational context). The framework provides researchers with a sound basis for exploring how the five elements of creativity can be incorporated within RE methods and techniques to support creative requirements engineering. It provides practitioners with a systematic means of creating environments that nurture and develop creative people, cognitive and collaborative processes and products.

Automating regression test selection based on UML designs

Abstract: This paper presents a methodology and tool to support test selection from regression test suites based on change analysis in object-oriented designs. We assume that designs are represented using the Unified Modeling Language (UML) 2.0 and we propose a formal mapping between design changes and a classification of regression test cases into three categories: Reusable, Retestable, and Obsolete. We provide evidence of the feasibility of the methodology and its usefulness by using our prototype tool on an industrial case study and two student projects.

Towards the development of privacy-aware systems

Abstract: Privacy and data protection are pivotal issues in nowadays society. They concern the right to prevent the dissemination of sensitive or confidential information of individuals. Many studies have been proposed on this topic from various perspectives, namely sociological, economic, legal, and technological. We have recognized the legal perspective as being the basis of all other perspectives. Actually, data protection regulations set the legal principles and requirements that must be met by organizations when processing personal data. The objective of this work is to provide a reference base for the development of methodologies tailored to design privacy-aware systems to be compliant with data protection regulations.

On the generation of requirements specifications from software engineering models: A systematic literature review

Abstract: System and software requirements documents play a crucial role in software engineering in that they must both communicate requirements to clients in an understandable manner and define requirements in precise detail for system developers. The benefits of both lists of textual requirements (usually written in natural language) and software engineering models (usually specified in graphical form) can be brought together by combining the two approaches in the specification of system and software requirements documents. If, moreover, textual requirements are generated from models in an automatic or closely monitored form, the effort of specifying those requirements is reduced and the completeness of the specification and the management of the requirements traceability are improved. This paper presents a systematic review of the literature related to the generation of textual requirements specifications from software engineering models.

Definitions and approaches to model quality in model-based software development - A review of literature

Abstract: More attention is paid to the quality of models along with the growing importance of modelling in software development. We performed a systematic review of studies discussing model quality published since 2000 to identify what model quality means and how it can be improved. From forty studies covered in the review, six model quality goals were identified; i.e., correctness, completeness, consistency, comprehensibility, confinement and changeability. We further present six practices proposed for developing high-quality models together with examples of empirical evidence. The contributions of the article are identifying and classifying definitions of model quality and identifying gaps for future research.

Experimental comparison of attack trees and misuse cases for security threat identification

Abstract: A number of methods have been proposed or adapted to include security in the requirements analysis stage, but the industrial take-up has been limited and there are few empirical and comparative evaluations. This paper reports on a pair of controlled experiments that compared two methods for early elicitation of security threats, namely attack trees and misuse cases. The 28 and 35 participants in the two experiments solved two threat identification tasks individually by means of the two techniques, using a Latin-Squares design to control for technique and task order. The dependent variables were effectiveness of the techniques measured as the number of threats found, coverage of the techniques measured in terms of the types of threats found and perceptions of the techniques measured through a post-task questionnaire based on the Technology Acceptance Model. The only difference was that, in the second experiment, the participants were given a pre-drawn use-case diagram to use as a starting point for solving the tasks. In the first experiment, no pre-drawn use-case diagram was provided. The main finding was that attack trees were more effective for finding threats, in particular when there was no pre-drawn use-case diagram. However, the participants had similar opinions of the two techniques, and perception of a technique was not correlated with performance with that technique. The study underlines the need for further comparisons in a broader range of settings involving additional techniques, and it suggests several concrete experiments and other paths for further work.

VxBPEL: Supporting variability for Web services in BPEL

Abstract: Web services provide a way to facilitate the business integration over the Internet. Flexibility is an important and desirable property of Web service-based systems due to dynamic business environments. The flexibility can be provided or addressed by incorporating variability into a system. In this study, we investigate how variability can be incorporated into service-based systems. We propose a language, VxBPEL, which is an adaptation of an existing language, BPEL, and able to capture variability in these systems. We develop a prototype to interpret this language. Finally, we illustrate our method by using it to handle variability of an example.

Automated analysis of security-design models

Abstract: We have previously proposed SecureUML, an expressive UML-based language for constructing security-design models, which are models that combine design specifications for distributed systems with specifications of their security policies. Here, we show how to automate the analysis of such models in a semantically precise and meaningful way. In our approach, models are formalized together with scenarios that represent possible run-time instances. Queries about properties of the security policy modeled are expressed as formulas in UML's Object Constraint Language. The policy may include both declarative aspects, i.e., static access-control information such as the assignment of users and permissions to roles, and programmatic aspects, which depend on dynamic information, namely the satisfaction of authorization constraints in a given scenario. We show how such properties can be evaluated, completely automatically, in the context of the metamodel of the security-design language. We demonstrate, through examples, that this approach can be used to formalize and check non-trivial security properties. The approach has been implemented in the SecureMOVA tool and all of the examples presented have been checked using this tool.

Complexity metrics for Workflow nets

Abstract: Process modeling languages such as EPCs, BPMN, flow charts, UML activity diagrams, Petri nets, etc., are used to model business processes and to configure process-aware information systems. It is known that users have problems understanding these diagrams. In fact, even process engineers and system analysts have difficulties in grasping the dynamics implied by a process model. Recent empirical studies show that people make numerous errors when modeling complex business processes, e.g., about 20% of the EPCs in the SAP reference model have design flaws resulting in potential deadlocks, livelocks, etc. It seems obvious that the complexity of the model contributes to design errors and a lack of understanding. It is not easy to measure complexity, however. This paper presents three complexity metrics that have been implemented in the process analysis tool ProM. The metrics are defined for a subclass of Petri nets named Workflow nets, but the results can easily be applied to other languages. To demonstrate the applicability of these metrics, we have applied our approach and tool to 262 relatively complex Protos models made in the context of various student projects. This allows us to validate and compare the different metrics. It turns out that our new metric focusing on the structuredness outperforms existing metrics.

On the secure software development process: CLASP, SDL and Touchpoints compared

Abstract: Development processes for software construction are common knowledge and mainstream practice in most development organizations. Unfortunately, these processes offer little support in order to meet security requirements. Over the years, research efforts have been invested in specific methodologies and techniques for secure software engineering, yet dedicated processes have been proposed only recently. In this paper, three high-profile processes for the development of secure software, namely OWASP's CLASP, Microsoft's SDL and McGraw's Touchpoints, are evaluated and compared in detail. The paper identifies the commonalities, discusses the specificity of each approach, and proposes suggestions for improvement.

Tool-supported requirements prioritization: Comparing the AHP and CBRank methods

Abstract: Requirements prioritization aims at identifying the most important requirements for a software system, a crucial step when planning for system releases and deciding which requirements to implement in each release. Several prioritization methods and supporting tools have been proposed so far. How to evaluate their properties, with the aim of supporting the selection of the most appropriate method for a specific project, is considered a relevant question. In this paper, we present an empirical study aiming at evaluating two state-of-the art tool-supported requirements prioritization methods, AHP and CBRank. We focus on three measures: the ease of use, the time-consumption and the accuracy. The experiment has been conducted with 23 experienced subjects on a set of 20 requirements from a real project. Results indicate that for the first two characteristics CBRank overcomes AHP, while for the accuracy AHP performs better than CBRank, even if the resulting ranks from the two methods are very similar. The majority of the users found CBRank the ''overall best'' method.

A systematic review of quasi-experiments in software engineering

Abstract: Background:: Experiments in which study units are assigned to experimental groups nonrandomly are called quasi-experiments. They allow investigations of cause-effect relations in settings in which randomization is inappropriate, impractical, or too costly. Problem outline:: The procedure by which the nonrandom assignments are made might result in selection bias and other related internal validity problems. Selection bias is a systematic (not happening by chance) pre-experimental difference between the groups that could influence the results. By detecting the cause of the selection bias, and designing and analyzing the experiments accordingly, the effect of the bias may be reduced or eliminated. Research method:: To investigate how quasi-experiments are performed in software engineering (SE), we conducted a systematic review of the experiments published in nine major SE journals and three conference proceedings in the decade 1993-2002. Results:: Among the 113 experiments detected, 35% were quasi-experiments. In addition to field experiments, we found several applications for quasi-experiments in SE. However, there seems to be little awareness of the precise nature of quasi-experiments and the potential for selection bias in them. The term ''quasi-experiment'' was used in only 10% of the articles reporting quasi-experiments; only half of the quasi-experiments measured a pretest score to control for selection bias, and only 8% reported a threat of selection bias. On average, larger effect sizes were seen in randomized than in quasi-experiments, which might be due to selection bias in the quasi-experiments. Conclusion:: We conclude that quasi-experimentation is useful in many settings in SE, but their design and analysis must be improved (in ways described in this paper), to ensure that inferences made from this kind of experiment are valid.

Factor oriented requirement coverage based system test case prioritization of new and regression test cases

Abstract: Test case prioritization involves scheduling test cases in an order that increases the effectiveness in achieving some performance goals. One of the most important performance goals is the rate of fault detection. Test cases should run in an order that increases the possibility of fault detection and also that detects the most severe faults at the earliest in its testing life cycle. In this paper, we propose to put forth a model for system level test case prioritization (TCP) from software requirement specification to improve user satisfaction with quality software that can also be cost effective and to improve the rate of severe fault detection. The proposed model prioritizes the system test cases based on the six factors: customer priority, changes in requirement, implementation complexity, completeness, traceability and fault impact. The proposed prioritization technique is validated with two different validation techniques and is experimented in three phases with student projects and two sets of industrial projects and the results show convincingly that the proposed prioritization technique improves the rate of severe fault detection.

The repertory grid technique: Its place in empirical software engineering research

Abstract: Personal construct theory (applied via the repertory grid technique) supports interpretivist research in a structured manner and, as such, has relevance for researchers conducting studies focused on the human and organisational aspects of software engineering. Personal construct theory (which underpins the repertory grid technique) is introduced, and the technique and its administration is discussed. Research studies from the literature are reviewed to provide illustrative examples of its application within a software engineering context. Since any research approach needs to answer questions about its reliability and validity within a particular study, these issues are considered for repertory grid investigations and criteria are offered that can be used to judge these issues within a planned, and/or reported, study.

On automated prepared statement generation to remove SQL injection vulnerabilities

Abstract: Since 2002, over 10% of total cyber vulnerabilities were SQL injection vulnerabilities (SQLIVs). This paper presents an algorithm of prepared statement replacement for removing SQLIVs by replacing SQL statements with prepared statements. Prepared statements have a static structure, which prevents SQL injection attacks from changing the logical structure of a prepared statement. We created a prepared statement replacement algorithm and a corresponding tool for automated fix generation. We conducted four case studies of open source projects to evaluate the capability of the algorithm and its automation. The empirical results show that prepared statement code correctly replaced 94% of the SQLIVs in these projects.

An aspect-oriented methodology for designing secure applications

Abstract: We propose a methodology, based on aspect-oriented modeling (AOM), for incorporating security mechanisms in an application. The functionality of the application is described using the primary model and the attacks are specified using aspects. The attack aspect is composed with the primary model to obtain the misuse model. The misuse model describes how much the application can be compromised. If the results are unacceptable, then some security mechanism must be incorporated into the application. The security mechanism, modeled as security aspect, is composed with the primary model to obtain the security-treated model. The security-treated model is analyzed to give assurance that it is resilient to the attack.

A framework for developing measurement systems and its industrial evaluation

Abstract: As in every engineering discipline, metrics play an important role in software development, with the difference that almost all software projects need the customization of metrics used. In other engineering disciplines, the notion of a measurement system (i.e. a tool used to collect, calculate, and report quantitative data) is well known and defined, whereas it is not as widely used in software engineering. In this paper we present a framework for developing custom measurement systems and its industrial evaluation in a software development unit within Ericsson. The results include the framework for designing measurement systems and its evaluation in real life projects at the company. The results show that with the help of ISO/IEC standards, measurement systems can be effectively used in software industry and that the presented framework improves the way of working with metrics. This paper contributes with the presentation of how automation of metrics collection and processing can be successfully introduced into a large organization and shows the benefits of it: increased efficiency of metrics collection, increased adoption of metrics in the organization, independence from individuals and standardized nomenclature for metrics in the organization.

Analysis of test suite reduction with enhanced tie-breaking techniques

Abstract: Test suite minimization techniques try to remove redundant test cases of a test suite. However, reducing the size of a test suite might reduce its ability to reveal faults. In this paper, we present a novel approach for test suite reduction that uses an additional testing criterion to break the ties in the minimization process. We integrated the proposed approach with two existing algorithms and conducted experiments for evaluation. The experiment results show that our approach can improve the fault detection effectiveness of reduced suites with a negligible increase in the size of the suites. Besides, under specific conditions, the proposed approach can also accelerate the process of minimization.

Empirical investigation towards the effectiveness of Test First programming

Abstract: The Test First (TF) programming, which is based on an iterative process of ''setting up test cases, implementing the functionality, and having all test cases passed'', has been put forward for decades, however knowledge of the evidence of the Test First programming's success is limited. This paper describes a controlled experiment that investigated the distinctions between the effectiveness of Test First and that of Test Last (TL) (the traditional approach). The experimental results showed that Test First teams spent a larger percentage of time on testing. The achievable minimum external quality of delivered software applications increased with the percentage of time spent on testing regardless of the testing strategy (TF or TL) applied, although there does not exist a linear correlation between them. With four years' data, it is also found that a strong linear correlation between the amount of effort spent on testing and coding in Test First teams, while this phenomenon was not observed in Test Last teams.