BACKGROUND: A software engineering systematic map is a defined method to build a classification scheme and structure a software engineering field of interest. The analysis of results focuses on frequencies of publications for categories within the scheme. Thereby, the coverage of the research field can be determined. Different facets of the scheme can also be combined to answer more specific research questions.

OBJECTIVE: We describe how to conduct a systematic mapping study in software engineering and provide guidelines. We also compare systematic maps and systematic reviews to clarify how to chose between them. This comparison leads to a set of guidelines for systematic maps.

METHOD: We have defined a systematic mapping process and applied it to complete a systematic mapping study. Furthermore, we compare systematic maps with systematic reviews by systematically analyzing existing systematic reviews.

RESULTS: We describe a process for software engineering systematic mapping studies and compare it to systematic reviews. Based on this, guidelines for conducting systematic maps are defined.

CONCLUSIONS: Systematic maps and reviews are different in terms of goals, breadth, validity issues and implications. Thus, they should be used complementarily and require different methods (e.g., for analysis).

/pdf/systematic-mapping-studies-in-software-engineering-4h4l7vgsw3.pdf

Systematic mapping studies in software engineering

Like other sciences and engineering disciplines, software engineering requires a cycle of model building, experimentation, and learning. Experiments are valuable tools for all software engineers who are involved in evaluating and choosing between different methods, techniques, languages and tools. The purpose of Experimentation in Software Engineering is to introduce students, teachers, researchers, and practitioners to empirical studies in software engineering, using controlled experiments. The introduction to experimentation is provided through a process perspective, and the focus is on the steps that we have to go through to perform an experiment. The book is divided into three parts. The first part provides a background of theories and methods used in experimentation. Part II then devotes one chapter to each of the five experiment steps: scoping, planning, execution, analysis, and result presentation. Part III completes the presentation with two examples. Assignments and statistical material are provided in appendixes. Overall the book provides indispensable information regarding empirical studies in particular for experiments, but also for case studies, systematic literature reviews, and surveys. It is a revision of the authors book, which was published in 2000. In addition, substantial new material, e.g. concerning systematic literature reviews and case study research, is introduced. The book is self-contained and it is suitable as a course book in undergraduate or graduate studies where the need for empirical studies in software engineering is stressed. Exercises and assignments are included to combine the more theoretical material with practical aspects. Researchers will also benefit from the book, learning more about how to conduct empirical studies, and likewise practitioners may use it as a cookbook when evaluating new methods or techniques before implementing them in their organization.

Experimentation in Software Engineering

Lessons from applying the systematic literature review process within the software engineering domain

With the ever-increasing development of technology and its integration into users' private and professional life, a decision regarding its acceptance or rejection still remains an open question. A respectable amount of work dealing with the technology acceptance model (TAM), from its first appearance more than a quarter of a century ago, clearly indicates a popularity of the model in the field of technology acceptance. Originated in the psychological theory of reasoned action and theory of planned behavior, TAM has evolved to become a key model in understanding predictors of human behavior toward potential acceptance or rejection of the technology. The main aim of the paper is to provide an up-to-date, well-researched resource of past and current references to TAM-related literature and to identify possible directions for future TAM research. The paper presents a comprehensive concept-centric literature review of the TAM, from 1986 onwards. According to a designed methodology, 85 scientific publications have been selected and classified according to their aim and content into three categories such as (i) TAM literature reviews, (ii) development and extension of TAM, and (iii) modification and application of TAM. Despite a continuous progress in revealing new factors with significant influence on TAM's core variables, there are still many unexplored areas of model potential application that could contribute to its predictive validity. Consequently, four possible future directions for TAM research based on the conducted literature review and analysis are identified and presented.

Technology acceptance model: a literature review from 1986 to 2013

https://cyberleninka.org/article/n/463350.pdf

Factors influencing acceptance of technology for aging in place: A systematic review

Context: The technology acceptance model (TAM) was proposed in 1989 as a means of predicting technology usage. However, it is usually validated by using a measure of behavioural intention to use (BI) rather than actual usage. Objective: This review examines the evidence that the TAM predicts actual usage using both subjective and objective measures of actual usage. Method: We performed a systematic literature review based on a search of six digital libraries, along with vote-counting meta-analysis to analyse the overall results. Results: The search identified 79 relevant empirical studies in 73 articles. The results show that BI is likely to be correlated with actual usage. However, the TAM variables perceived ease of use (PEU) and perceived usefulness (PU) are less likely to be correlated with actual usage. Conclusion: Care should be taken using the TAM outside the context in which it has been validated.

/pdf/does-the-technology-acceptance-model-predict-actual-use-a-58ax9mg1kr.pdf

Does the technology acceptance model predict actual use? A systematic literature review

There have been many changes in statistical theory in the past 30 years, including increased evidence that non-robust methods may fail to detect important results. The statistical advice available to software engineering researchers needs to be updated to address these issues. This paper aims both to explain the new results in the area of robust analysis methods and to provide a large-scale worked example of the new methods. We summarise the results of analyses of the Type 1 error efficiency and power of standard parametric and non-parametric statistical tests when applied to non-normal data sets. We identify parametric and non-parametric methods that are robust to non-normality. We present an analysis of a large-scale software engineering experiment to illustrate their use. We illustrate the use of kernel density plots, and parametric and non-parametric methods using four different software engineering data sets. We explain why the methods are necessary and the rationale for selecting a specific analysis. We suggest using kernel density plots rather than box plots to visualise data distributions. For parametric analysis, we recommend trimmed means, which can support reliable tests of the differences between the central location of two or more samples. When the distribution of the data differs among groups, or we have ordinal scale data, we recommend non-parametric methods such as Cliff's ź or a robust rank-based ANOVA-like method.

/pdf/robust-statistical-methods-for-empirical-software-3osxgs3iqt.pdf

Robust Statistical Methods for Empirical Software Engineering

When conducting a systematic literature review, researchers usually determine the relevance of primary studies on the basis of the title and abstract. However, experience indicates that the abstracts for many software engineering papers are of too poor a quality to be used for this purpose. A solution adopted in other domains is to employ structured abstracts to improve the quality of information provided. This study consists of a formal experiment to investigate whether structured abstracts are more complete and easier to understand than non-structured abstracts for papers that describe software engineering experiments. We constructed structured versions of the abstracts for a random selection of 25 papers describing software engineering experiments. The 64 participants were each presented with one abstract in its original unstructured form and one in a structured form, and for each one were asked to assess its clarity (measured on a scale of 1 to 10) and completeness (measured with a questionnaire that used 18 items). Based on a regression analysis that adjusted for participant, abstract, type of abstract seen first, knowledge of structured abstracts, software engineering role, and preference for conventional or structured abstracts, the use of structured abstracts increased the completeness score by 6.65 (SE 0.37, p?<?0.001) and the clarity score by 2.98 (SE 0.23, p?<?0.001). 57 participants reported their preferences regarding structured abstracts: 13 (23%) had no preference; 40 (70%) preferred structured abstracts; four preferred conventional abstracts. Many conventional software engineering abstracts omit important information. Our study is consistent with studies from other disciplines and confirms that structured abstracts can improve both information content and readability. Although care must be taken to develop appropriate structures for different types of article, we recommend that Software Engineering journals and conferences adopt structured abstracts.

Presenting software engineering results using structured abstracts: a randomised experiment

The problem of trying to view and comprehend large amounts of data is a well-known one. A specialised variant of this problem is the visualisation of software code and components for the purposes of understanding, decision-making, reuse and even integration. In particular the visualisation of software components, at a much higher level than source code, has received very little research. Visualisation is a powerful tool in situations such as this. This paper presents the application of real world metaphor based visualisations that address this problem. The application of visualisation to selecting software components is especially novel. It seeks to decrease the effort required by system integrators when locating suitable components in what is an increasingly crowded marketplace. Accurate information and understanding are vital if correct and informed decisions and judgements are to be made.

Visualisation for informed decision making; from code to components

Context: Secondary studies, such as systematic literature reviews and mapping studies, are an essential element of the evidence-based paradigm. A critical part of the review process is the identification of all relevant research. As such, any researcher intending to conduct a secondary review should be aware of the strengths and weakness of the search engines available. Objectives: Analyse the overlap between search engine results for software engineering studies. Methods: Three independent studies were conducted to evaluate the overlap between multiple search engines for different search areas. Results: The findings indicate that very little overlap was found between the search engines. Conclusions of the study: To complete a systematic review, researchers must use multiple search terms and search engines. The lack of overlap might also be caused by inconsistent keyword selection amongst authors.

Stuart Charters

Papers

Does the technology acceptance model predict actual use? A systematic literature review

Robust Statistical Methods for Empirical Software Engineering

Presenting software engineering results using structured abstracts: a randomised experiment

Visualisation for informed decision making; from code to components

Search Engine Overlaps: Do they agree or disagree?