Context: Software product lines (SPL) are used in industry to achieve more efficient software development. However, the testing side of SPL is underdeveloped. Objective: This study aims at surveying existing research on SPL testing in order to identify useful approaches and needs for future research. Method: A systematic mapping study is launched to find as much literature as possible, and the 64 papers found are classified with respect to focus, research type and contribution type. Results: A majority of the papers are of proposal research types (64%). System testing is the largest group with respect to research focus (40%), followed by management (23%). Method contributions are in majority. Conclusions: More validation and evaluation research is needed to provide a better foundation for SPL testing.

/pdf/software-product-line-testing-a-systematic-mapping-study-27nvr0okop.pdf

Software product line testing - A systematic mapping study

ContextIn software development, Testing is an important mechanism both to identify defects and assure that completed products work as specified. This is a common practice in single-system development, and continues to hold in Software Product Lines (SPL). Even though extensive research has been done in the SPL Testing field, it is necessary to assess the current state of research and practice, in order to provide practitioners with evidence that enable fostering its further development. ObjectiveThis paper focuses on Testing in SPL and has the following goals: investigate state-of-the-art testing practices, synthesize available evidence, and identify gaps between required techniques and existing approaches, available in the literature. MethodA systematic mapping study was conducted with a set of nine research questions, in which 120 studies, dated from 1993 to 2009, were evaluated. ResultsAlthough several aspects regarding testing have been covered by single-system development approaches, many cannot be directly applied in the SPL context due to specific issues. In addition, particular aspects regarding SPL are not covered by the existing SPL approaches, and when the aspects are covered, the literature just gives brief overviews. This scenario indicates that additional investigation, empirical and practical, should be performed. ConclusionThe results can help to understand the needs in SPL Testing, by identifying points that still require additional investigation, since important aspects regarding particular points of software product lines have not been addressed yet.

A systematic mapping study of software product lines testing

Since software systems need to be continuously available under varying conditions, their ability to evolve at runtime is increasingly seen as one key issue. Modern programming frameworks already provide support for dynamic adaptations. However the high-variability of features in Dynamic Adaptive Systems (DAS) introduces an explosion of possible runtime system configurations (often called modes) and mode transitions. Designing these configurations and their transitions is tedious and error-prone, making the system feature evolution difficult. While Aspect-Oriented Modeling (AOM) was introduced to improve the modularity of software, this paper presents how an AOM approach can be used to tame the combinatorial explosion of DAS modes. Using AOM techniques, we derive a wide range of modes by weaving aspects into an explicit model reflecting the runtime system. We use these generated modes to automatically adapt the system. We validate our approach on an adaptive middleware for home-automation currently deployed in Rennes metropolis.

https://www.irisa.fr/triskell/publis/2009/Morin09a.pdf

Taming Dynamically Adaptive Systems using models and aspects

https://lup.lub.lu.se/search/ws/files/3391741/3971740.pdf

On the Reliability of Mapping Studies in Software Engineering

Context: Since the introduction of evidence-based software engineering in 2004, systematic literature review (SLR) has been increasingly used as a method for conducting secondary studies in software engineering. Two tertiary studies, published in 2009 and 2010, identified and analysed 54 SLRs published in journals and conferences in the period between 1st January 2004 and 30th June 2008. Objective: In this article, our goal was to extend and update the two previous tertiary studies to cover the period between 1st July 2008 and 31st December 2009. We analysed the quality, coverage of software engineering topics, and potential impact of published SLRs for education and practice. Method: We performed automatic and manual searches for SLRs published in journals and conference proceedings, analysed the relevant studies, and compared and integrated our findings with the two previous tertiary studies. Results: We found 67 new SLRs addressing 24 software engineering topics. Among these studies, 15 were considered relevant to the undergraduate educational curriculum, and 40 appeared of possible interest to practitioners. We found that the number of SLRs in software engineering is increasing, the overall quality of the studies is improving, and the number of researchers and research organisations worldwide that are conducting SLRs is also increasing and spreading. Conclusion: Our findings suggest that the software engineering research community is starting to adopt SLRs consistently as a research method. However, the majority of the SLRs did not evaluate the quality of primary studies and fail to provide guidelines for practitioners, thus decreasing their potential impact on software engineering practice.

Six years of systematic literature reviews in software engineering: An updated tertiary study

Software product lines constitute a new paradigm where industrial production techniques are adapted and applied to software development. Reuse and the maintenance of traceability between the different artefacts in the line are fundamental requirements in this paradigm, articulating the best practices for software development in an environment that is perfectly controlled by software engineering methods. This article presents a systematic review of the literature which deals with testing in software product lines. The objective is to analyse the existing approaches to testing in software product lines, discussing the significant issues related to this area of knowledge and providing an up-to-date state of the art which can serve as a basis for innovative research activities. The paper also analyses how SPL research can contribute and dynamise the research in software

SOFTWARE PRODUCT LINE TESTING - A Systematic Review

Diverse models currently exist in the field of Software Engineering which help organizations to apply recommended practices in order to support ther multiple needs in the areas of software development, maintenance and operation, security, IT government, etc. Examples of such models are CMMI, ISO 9001, ISO 12207, ISO 27001, COBIT, ITIL. Nevertheless, many differences exist between these models, since each model defines its own structure, terminology, definitions and quality systems, amongst other aspects. This issue increases the complexity when an organization is required to apply two or more models in order to satisfy its needs. Organizations must, therefore, define the most appropriate means of choosing and implementing multi-models, and harmonization may be one solution. This paper presents a systematic literature review with the aim of analyzing the state of the art with regard to inititatives concerning the harmonization of multiple reference models. As a result, it has been concluded that there is currently a lack of guidelines with which to help organizations to implement the harmonization of multiple models, and of a unified terminology with which to homogenize the diversity of the structure of the different models and the harmonization techniques which can be applied. In order to address these issues, a framework to support the harmonization of multiple models is outlined.

Trends in Harmonization of Multiple Reference Models

This paper describes an approach for model-based testing which is completely based on standards, what constitutes an essential factor for its implantation in industrial software tools. The inputs of the process are models described in UML 2.0, whereas the outputs consist of artefacts of the UML Testing Profile. The transformation process is based on QVT (Query-View-Transformation) scripts, a transformation language also adopted by the OMG.

Automated model-based testing using the UML testing profile and QVT

informático colombiano a mejorar sus procesos de desarrollo de software, con el objetivo de lograr un nivel de madurez en sus procesos que garantice su competitividad internacional, es necesario adecuar a sus propias características los modelos internacionalmente reconocidos de mejoramiento, evaluación y calidad. Estos modelos difícilmente pueden ser aplicados a empresas pequeñas debido a su gran inversión en dinero, tiempo y recursos, además de la complejidad de las recomendaciones y un retorno de la inversión a largo plazo. El objetivo de este trabajo es presentar, a Light MECPDS, un modelo ligero de evaluación de la calidad de procesos de desarrollo de software basado las normas ISO/IEC 12207:2002 e ISO/IEC 15504:2003 aplicable a las micro, pequeñas y medianas empresas, de manera fácil y económica, con pocos recursos y en poco tiempo.

Adaptación de las Normas ISO/IEC 12207:2002 e ISO/IEC 15504:2003 para la Evaluación de la Madurez de Procesos Software en Países en Desarrollo

Information Technologies (IT) play a crucial role in the development of activities concerning banking organizations. To achieve IT Governance, while at the same time giving special consideration to the attainment of business objectives, is of essential interest to any banking organization. This article presents a model for IT Governance which is applicable to the banking sector, which begins with the identification of COBIT processes that support the fulfilment of the principles of operative risk defined by BASEL II. It also integrates best practices oriented to risk and investment management for IT, information security and service lifecycle administration, described in models like: RISK IT, VAL IT, ISO 27002 and ITIL respectively. This model allows a banking organization to govern, evaluate and monitor its IT, thus fitting it to its strategic objectives. It also permits it to manage the operative risk from the perspective of normative fulfilment in banking.

Mario Piattini Velthius

Papers

SOFTWARE PRODUCT LINE TESTING - A Systematic Review

Trends in Harmonization of Multiple Reference Models

Automated model-based testing using the UML testing profile and QVT

Adaptación de las Normas ISO/IEC 12207:2002 e ISO/IEC 15504:2003 para la Evaluación de la Madurez de Procesos Software en Países en Desarrollo

Towards a model for information technology governance applicable to the banking sector