Investigating software testing and maintenance reports: Case study
TL;DR: Fine-grained problem handling reports provide a wider scope of possible measures to assess the relevant development processes and provide significant enhancement in evaluation and refinement of software development processes as well as in reliability prediction.
Abstract: Context Although many papers have been published on software development and defect prediction techniques, problem reports in real projects quite often differ from those described in the literature. Hence, there is still a need for deeper exploration of case studies from industry. Objective The aim of this study is to present the impact of fine-grained problem reports on improving evaluation of testing and maintenance processes. It is targeted at projects involving several releases and complex schemes of problem handling. This is based on our experience gained while monitoring several commercial projects. Method Extracting certain features from detailed problem reports, we derive various measures and present analysis models which characterize and visualize the effectiveness of testing and problem resolution processes. The considered reports describe types of problems (e.g. defects), their locations in project versions and software modules, ways of their resolution, etc. The performed analysis is related to eleven projects developed in the same company. This study is an exploratory research with some explanatory features. Moreover, having identified some drawbacks, we present extensions of problem reports and their analysis which have been verified in another industrial case study project. Results Fine-grained (accurate) problem handling reports provide a wider scope of possible measures to assess the relevant development processes. This is helpful in controlling single projects (local perspective) as well as in managing these processes in the whole company (global perspective). Conclusion Detailed problem handling reports extend the space and quality of statistical analysis, they provide significant enhancement in evaluation and refinement of software development processes as well as in reliability prediction.
Citations
More filters
01 Jan 2017
TL;DR: Strategies to Manage Enterprise Information Technology Projects by as mentioned in this paper ) is a strategy to manage enterprise information technology projects in the enterprise. But it is not suitable for large-scale projects.
Abstract: Strategies to Manage Enterprise Information Technology Projects by
10 citations
Cites methods from "Investigating software testing and ..."
...Janczarek and Sosnowski (2015) referred to the quantitative method as theoretical testing consisting of statistical measurements of human problems....
[...]
...Qualitative research was more suitable to identify the cause of a problem instead of seeking to explain variables (see Janczarek & Sosnowski, 2015)....
[...]
01 May 2020
TL;DR: This paper seeks to model this way of working by extending the Jelinski–Moranda model to a “stack” of feature-specific models, assuming that the bugs are labeled with the features they belong to, and presents the results in predicting the reliability of software for agile testing environments.
Abstract: It is known that quantitative measures for the reliability of software systems can be derived from software reliability models, and, as such, support the product development process. Over the past four decades, research activities in this area have been performed. As a result, many software reliability models have been proposed. It was shown that, once these models reach a certain level of convergence, it can enable the developer to release the software and stop software testing accordingly. Criteria to determine the optimal testing time include the number of remaining errors, failure rate, reliability requirements, or total system cost. In this paper, we present our results in predicting the reliability of software for agile testing environments. We seek to model this way of working by extending the Jelinski-Moranda model to a "stack" of feature-specific models, assuming that the bugs are labeled with the features they belong to. In order to demonstrate the extended model, two use cases are presented. The questions to be answered in these two cases are: how many software bugs remain in the software and should one decide to stop testing the software?.
9 citations
TL;DR: Detailed problem handling reports extend the space and quality of statistical analysis, the presented graph model with proposed metrics provides a useful tool to evaluate and refine problem resolution schemes.
Abstract: Context Although many papers have been published on software development and maintenance processes, there is still a need for deeper exploration of software repositories related to real projects to evaluate these processes. Objective The aim of this study is to present and evaluate different schemes of handling problems (bugs) during software development and maintenance. It is based on exploring information comprised in various software repositories. Method Having analysed the contents of software repositories we have introduced a graph model describing problem handling processes and a scheme of analysing the effectiveness of these processes. This analysis is targeted at structural, throughput and timing aspects. The presented methodology has been verified on real data comprised in open source and commercial projects. Moreover, having identified some drawbacks and anomalies, we point out possible improvements in problem reports and their management. Results Detailed software repositories provide a wider scope of possible measures to assess the relevant problem handling processes. This is helpful in controlling single projects (local perspective) as well as in managing these processes in the whole company (global perspective). Conclusion Detailed problem handling reports extend the space and quality of statistical analysis, the presented graph model with proposed metrics provides a useful tool to evaluate and refine problem resolution schemes.
8 citations
27 Jun 2016
TL;DR: A special program is developed which extracts information from typical bug repositories and generates various useful statistics, based on problem handling graphs (PHGs) and various metrics, which confirm the practical significance of these data.
Abstract: The paper deals with the problem of monitoring software development and maintenance processes. In particular, we concentrate on data reported in software bug repositories. These data characterize the progress and effectiveness of the above mentioned processes. To analyze these data we have developed a special program which extracts information from typical bug repositories (e.g. Bugzilla) and generates various useful statistics. The analysis methodology is based on problem handling graphs (PHGs) and various metrics. The usefulness of this approach has been illustrated for some real open source and commercial projects. We present the measurement methodology and interpretation of the obtained results, which confirmed their practical significance.
4 citations
Dissertation•
08 Jun 2018
TL;DR: In this article, the authors have developed a theoretical framework appropriate to reduce ethical problems and decrease high attrition rate within the digital service sector (DSS) in India, which is based on an extensive review of the literature and testing several factors pertinent to the Indian context.
Abstract: This research aims to test the hypothesis that ethical problems are causing high staff attrition rates within the digital service sector (DSS) in India. It is also shown that ethical problems affect other factors in DSS, such as trust, decision-making process, stability, and day to day activities of the individuals in the workplace. This research focuses on the ethical issues in India’s DSS as it exhibits a high staff attrition rate. India is the second largest populated country in the world and produces millions of multitasking, knowledgeable, talented and digitally skilled engineers annually. The rapid growth of Indian’s economy in the DSS is contributing largely to the national Gross Domestic Product (GDP). Unfortunately, with the increasing and leading development of Indian DSS, organisations have started facing unethical conducts and unfair treatment towards their employees. In general ethical issues are well managed in developed and advanced countries, but their implementation in developing countries such as India is still problematic.
This study has developed a theoretical framework appropriate to reduce ethical problems and decrease high attrition rate in the Indian DSS. It also focuses on identifying and reducing the ethical problems and decrease their effects on the major issue of a high rate of employee attrition. This research is based on an extensive review of the literature and testing several factors pertinent to the Indian context. The researcher has adopted a novel mixed method approach to accomplish aim and objectives by securing the perceptions of Indian DSS employees. Additionally, the researcher conducted a quantitative survey and qualitative interviews to gather information via a mixed research method strategy. The quantitative information collected from 64 Indian digitally skilled employees through surveys and a total 16 interviews were conducted to analyse the data. The survey questionnaires were analysed quantitatively by descriptive and statistical methods using SPSS and interviews were examined using the NVivo software.
Furthermore, the researcher has also investigated the existence of guidelines, policies and code of ethics within the industry and whether it is properly followed and implemented by the management. Using a specialised code of ethics and professional conduct that have proved to be successful, the researcher created, develop and enhanced the existing code of ethics with regards to the employee, with an attempt to reduce the ethical problems and reduce high staff attrition rate across India’s DSS. The framework is useful for policymakers, the commercial and digital service sectors and to the government bodies to create change in India. The result of the study produces a new theory to solve the problem with the set of recommendations and proposals. Hence, the research contributes to knowledge about ethical issues and their effects on attrition. The development of this research study is considered to be the first of its kind and thus lays the ground for further research in this very sensitive area of the subject.
4 citations
Cites background from "Investigating software testing and ..."
...It is a requirement of a deep exploration of the collected information and case studies from the industries (Janczarek and Sosnowski, 2015)....
[...]
References
More filters
Book•
16 Jun 2012
TL;DR: The purpose of Experimentation in Software Engineering is to introduce students, teachers, researchers, and practitioners to empirical studies in software engineering, using controlled experiments, and provides indispensable information regarding empirical Studies in particular for experiments, but also for case studies, systematic literature reviews, and surveys.
Abstract: Like other sciences and engineering disciplines, software engineering requires a cycle of model building, experimentation, and learning. Experiments are valuable tools for all software engineers who are involved in evaluating and choosing between different methods, techniques, languages and tools. The purpose of Experimentation in Software Engineering is to introduce students, teachers, researchers, and practitioners to empirical studies in software engineering, using controlled experiments. The introduction to experimentation is provided through a process perspective, and the focus is on the steps that we have to go through to perform an experiment. The book is divided into three parts. The first part provides a background of theories and methods used in experimentation. Part II then devotes one chapter to each of the five experiment steps: scoping, planning, execution, analysis, and result presentation. Part III completes the presentation with two examples. Assignments and statistical material are provided in appendixes. Overall the book provides indispensable information regarding empirical studies in particular for experiments, but also for case studies, systematic literature reviews, and surveys. It is a revision of the authors book, which was published in 2000. In addition, substantial new material, e.g. concerning systematic literature reviews and case study research, is introduced. The book is self-contained and it is suitable as a course book in undergraduate or graduate studies where the need for empirical studies in software engineering is stressed. Exercises and assignments are included to combine the more theoretical material with practical aspects. Researchers will also benefit from the book, learning more about how to conduct empirical studies, and likewise practitioners may use it as a cookbook when evaluating new methods or techniques before implementing them in their organization.
2,079 citations
Book•
01 Dec 1994
TL;DR: Stephen H. Kan is responsible for IBM Rochester's software quality strategy and plans, quality assessment, software measurements, and statistical analysis.
Abstract: From the Publisher:
Author Biography:
Dr. Stephen H. Kan, an ASQC Certified Quality Engineer, is responsible for IBM Rochester's software quality strategy and plans, quality assessment, software measurements, and statistical analysis. He has been the software quality focal point for the software system of the AS/400 computer since its initial release in 1988.
1,075 citations
Book•
07 Mar 2012
TL;DR: This is the first software engineering specific book on the case study research method and presents detailed practical guidelines on the preparation, conduct, design and reporting of case studies of software engineering.
Abstract: Based on their own experiences of in-depth case studies of software projects in international corporations, in this bookthe authors present detailed practical guidelines on the preparation, conduct, design and reporting of case studies of software engineering. This is the first software engineering specific book on thecase study research method.
1,001 citations
Book•
[...]
01 Mar 1999
TL;DR: In this paper, the authors present detailed analytical models, state-of-the-art techniques, methodologies, and tools used to assess the reliability of software systems, and present a detailed analytical model for software systems.
Abstract: From the Publisher:
this book presents detailed analytical models, state-of-the-art techniques, methodologies, and tools used to assess the reliability of software systems.
752 citations
TL;DR: Object-oriented and process metrics have been reported to be more successful in finding faults compared to traditional size and complexity metrics and seem to be better at predicting post-release faults than any static code metrics.
Abstract: ContextSoftware metrics may be used in fault prediction models to improve software quality by predicting fault location. ObjectiveThis paper aims to identify software metrics and to assess their applicability in software fault prediction. We investigated the influence of context on metrics' selection and performance. MethodThis systematic literature review includes 106 papers published between 1991 and 2011. The selected papers are classified according to metrics and context properties. ResultsObject-oriented metrics (49%) were used nearly twice as often compared to traditional source code metrics (27%) or process metrics (24%). Chidamber and Kemerer's (CK) object-oriented metrics were most frequently used. According to the selected studies there are significant differences between the metrics used in fault prediction performance. Object-oriented and process metrics have been reported to be more successful in finding faults compared to traditional size and complexity metrics. Process metrics seem to be better at predicting post-release faults compared to any static code metrics. ConclusionMore studies should be performed on large industrial software systems to find metrics more relevant for the industry and to answer the question as to which metrics should be used in a given context.
437 citations