Showing papers on "Acceptance testing published in 2016"

An Innovative Approach to Investigate Various Software Testing Techniques and Strategies

Abstract: Software testing is a way of finding errors from the system. It helps us to identify and debug mistakes, errors, faults and failures of a system. There are many techniques and strategies emerged since the concept of software development emerged. The aim of testing is to make the quality of software as efficient as possible.in this paper we discuss most widely used techniques and strategies. Where they can be used and how they can be used. How they work and how they differ (from each other).They are the following. Techniques: Black Box Testing, White Box Testing, And Grey Box Testing. Strategies: Unit Testing, System Testing, And Acceptance Testing.

Challenges and strategies for motivating software testing personnel

Abstract: ContextSoftware testing is the key to ensuring a successful and reliable software product or service, yet testing is often considered uninteresting work compared to design or coding. As any human-based activity, the outcome of the final software product is dependent of human factors and an essential challenge for software development organizations is to find effective ways to enhance the motivation and job-satisfaction of their testers. ObjectiveOur study aims to cast light on how professional software testers can be motivated and we explore the policies and rules conceptualized and implemented inside software development projects. MethodThis paper presents the results of an empirical study that collected data through semi-structured and in-depth interviews with 36 practitioners from 12 companies in Norway. The data collection was performed over a two years period and investigates the strategies applied by the companies for stimulating their testers, while considering the motivational and de-motivational factors influencing the testing personnel. ResultsOur results provide a set of motivational and de-motivational factors for software testing personnel and present the strategies deployed by the companies for stimulating their testing staff. ConclusionsThe study shows that combining testing responsibilities with development and ensuring a variety of engaging, challenging tasks and products does increase the satisfaction of testing personnel. However, despite the systematic and sincere effort invested in recognizing the importance of testing and motivating the testers, heavy emphasis is laid on minimizing project costs and duration. The results could help the companies in organizing and managing processes and stimulate their testing personnel, which will lead to better job satisfaction and productivity.

Testing the untestable: model testing of complex software-intensive systems

Abstract: Increasingly, we are faced with systems that are untestable, meaning that traditional testing methods are expensive, time-consuming or infeasible to apply due to factors such as the systems' continuous interactions with the environment and the deep intertwining of software with hardware. In this paper we outline our vision to enable testing of untestable systems. Our key idea is to frame testing on models rather than operational systems. We refer to such testing as model testing. Our goal is to raise the level of abstraction of testing from operational systems to models of their behaviors and properties. The models that underlie model testing are executable representations of the relevant aspects of a system and its environment, alongside the risks of system failures. Such models necessarily have uncertainties due to complex, dynamic environment behaviors and the unknowns about the system. This makes it crucial for model testing to be uncertainty-aware. We propose to synergistically combine metaheuristic search, increasingly used in traditional software testing, with system and risk models to drive the search for faults that entail the most risk. We expect model testing to bring early and cost-effective automation to the testing of many critical systems that defy existing automation techniques, thus significantly improving the dependability of such systems.

Assessing the Verification and Validation of Building Fire Evacuation Models

Abstract: To date there is no International standard on the verification and validation (V&V) of building fire evacuation models, i.e., model testers adopt inconsistent procedures or tests designed for other model uses. For instance, the tests presented within the MSC/Circ.1238 Guidelines for evacuation analysis for new and existing passenger ships provided by the International Maritime Organization are often employed for the V&V of models outside their original context of use (building fires instead of maritime applications). This paper presents a list of verification tests for component testing and the analysis of emergent behaviours together with examples of experimental data-sets suitable for the analysis of different core components. The capabilities of building fire evacuation models are evaluated by studying their five main core components, namely (1) pre-evacuation time, (2) movement and navigation, (3) exit usage, (4) route availability and (5) flow constraints. This paper discusses the tests which are included in a freely available Technical Note developed at the National Institute of Standards and Technology. This work is intended to open a discussion on the main issues associated with the definition of a standard procedure for the V&V of building fire evacuation models, including the definition of the acceptance criteria of a standard V&V protocol.

Improving the delivery cycle

Abstract: Context: Software companies seek to gain benefit from agile development approaches in order to meet evolving market needs without losing their innovative edge. Agile practices emphasize frequent releases with the help of an automated toolchain from code to delivery.Objective: We investigate, which tools are used in software delivery, what are the reasons omitting certain parts of the toolchain and what implications toolchains have on how rapidly software gets delivered to customers.Method: We present a multiple-case study of the toolchains currently in use in Finnish software-intensive organizations interested in improving their delivery frequency. We conducted qualitative semi-structured interviews in 18 case organizations from various software domains. The interviewees were key representatives of their organization, considering delivery activities.Results: Commodity tools, such as version control and continuous integration, were used in almost every organization. Modestly used tools, such as UI testing and performance testing, were more distinctly missing from some organizations. Uncommon tools, such as artifact repository and acceptance testing, were used only in a minority of the organizations. Tool usage is affected by the state of current workflows, manual work and relevancy of tools. Organizations whose toolchains were more automated and contained fewer manual steps were able to deploy software more rapidly.Conclusions: There is variety in the need for tool support in different development steps as there are domain-specific differences in the goals of the case organizations. Still, a well-founded toolchain supports speedy delivery of new software.

Loading Protocols for ASCE 41 Backbone Curves

Abstract: ASCE 41 is the industry standard for seismic evaluation and retrofit of existing buildings. It allows for alternative component modeling and acceptance criteria based on a backbone curve constructe...

Recent Advances in Regression Testing Techniques

Abstract: Software systems and their environment change are continuous. They are enhanced, corrected, and ported to new platforms. These changes can affect a system adversely, thus software engineers perform regression testing to ensure the quality of the modified systems. Regression testing is an integral part of most major software projects, but as projects grow larger and the number of tests increases, performing regression testing becomes more costly. To address this problem, many researchers and practitioners have proposed and empirically evaluated various regression testing techniques, such as regression test selection, test case prioritization, and test suite minimization. Recent surveys on these techniques indicate that this research area continues to grow, heuristics and the types of data utilized become diverse, and wider application domains have been considered. This chapter presents the current status and the trends of three regression testing techniques and discusses recent advances of each technique.

A Controlled Experiment in Testing of Safety-Critical Embedded Software

Abstract: In engineering of safety critical systems, regulatory standards often put requirements on both traceable specification-based testing, and structural coverage on program units. Automated test generation techniques can be used to generate inputs to cover the structural aspects of a program. However, there is no conclusive evidence on how automated test generation compares to manual test design, or how testing based on the program implementation relates to specification-based testing. In this paper, we investigate specification -- and implementation-based testing of embedded software written in the IEC 61131-3 language, a programming standard used in many embedded safety critical software systems. Further, we measure the efficiency and effectiveness in terms of fault detection. For this purpose, a controlled experiment was conducted, comparing tests created by a total of twenty-three software engineering master students. The participants worked individually on manually designing and automatically generating tests for two IEC 61131-3 programs. Tests created by the participants in the experiment were collected and analyzed in terms of mutation score, decision coverage, number of tests, and testing duration. We found that, when compared to implementation-based testing, specification-based testing yields significantly more effective tests in terms of the number of faults detected. Specifically, specification-based tests more effectively detect comparison and value replacement type of faults, compared to implementation-based tests. On the other hand, implementation-based automated test generation leads to fewer tests (up to 85% improvement) created in shorter time than the ones manually created based on the specification.

An Exploratory Study on Handling Requirements and Acceptance Test Documentation in Industry

Abstract: With the emergence and spread of agile processes, the practices of writing and maintaining documentation have drastically changed in the last decade. In this work, we performed a qualitative study to explore the current practices for managing two related types of software documentation: requirements and acceptance tests. We interviewed twenty practitioners from seventeen business units in fifteen companies to investigate the companies' practices for writing, maintaining and linking requirements and acceptance test documentation. The study yields interesting and partially unexpected results. For example, we had expected that tests would be more extensively documented than requirements, while we found a strong linear correlation between the number of requirements and tests in our sample. We also found that technical people are usually not involved in the requirements engineering activities, which often results in misunderstood or underestimated requirements. Acceptance tests are written, in many cases, based on requirements that are not necessarily detailed enough. Also, acceptance tests are not regularly maintained, which occasionally results in confusing features and bugs.

Advances in unit testing: theory and practice

Abstract: Parameterized unit testing, recent advances in unit testing, is a new methodology extending the previous industry practice based on traditional unit tests without parameters. A parameterized unit test (PUT) is simply a test method that takes parameters, calls the code under test, and states assertions. Parameterized unit testing allows the separation of two testing concerns or tasks: the specification of external, black-box behavior (i.e., assertions or specifications) by developers and the generation and selection of internal, white-box test inputs (i.e., high-code-covering test inputs) by tools. PUTs have been supported by various testing frameworks. Various open source and industrial testing tools also exist to generate test inputs for PUTs. This technical briefing presents latest research on principles and techniques, as well as practical considerations to apply parameterized unit testing on real-world programs, highlighting success stories, research and education achievements, and future research directions in developer testing.

Validation of ASCE 41-13 Modeling Parameters and Acceptance Criteria for Rocking Shallow Foundations

Abstract: The standard ASCE 41-13 Seismic Evaluation and Retrofit of Existing Buildings includes new provisions for linear and non-linear modeling parameters and acceptance criteria for rocking shallow found...

Characterization and acceptance testing of fully depleted thick CCDs for the Large Synoptic Survey Telescope

Abstract: The Large Synoptic Survey Telescope (LSST) camera will be made as a mosaic assembled of 189 large format Charge Coupled Devices (CCD). They are n-channel, 100 micron thick devices operated in the over depleted regime. There are 16 segments, 1 million pixels each, that are read out through separate amplifiers. The image quality and readout speed expected from LSST camera translates into strict acceptance requirements for individual sensors. Prototype sensors and preproduction CCDs were delivered by vendors and they have been used for developing test procedures and protocols. Building upon this experience, two test stands were designed and commissioned at Brookhaven National Laboratory for production electro-optical testing. In this article, the sensor acceptance criteria are outlined and discussed, the test stand design and used equipment are presented and the results from commissioning sensor runs are shown.

Reliability Modeling and Prediction of Systems With Mixture of Units

Abstract: Traditional reliability analysis and prediction are performed by utilizing observed failure or degradation data of test units or field observations. Reliability testing is usually performed to predict reliability or performed as acceptance testing or reliability demonstration test. Moreover, in many cases, reliability tests are performed repeatedly during the entire life of the system by testing different samples with different characteristics in the system. At the end of each test, available data only show the number and combination of failed units which are then used for reliability estimation and prediction. This paper investigates several effective approaches to obtain expressions for system reliability metrics under different scenarios, considering the mixture characteristics of the units. The proposed approaches apply to general cases when the population size, as well as the mixture of units, increase over time. A simulation model is utilized to validate the proposed models.

Guided semi-automatic system testing in factory automation

Abstract: Automated production systems (APS) in factory automation possess a high quality standard and complexity that poses high demands on quality assurance. Especially in later phases of system development, functional tests regarding the integrated behavior of software and hardware, so called system tests, are performed. As methods and tools for system testing are lacking or not applicable in this domain, manual testing currently is the prevalent approach for testing factory automation systems. Due to this and scarce resources during testing, this results in varying test quality as well as minimal documentation. Reproducibility and traceability of performed tests is low, prohibiting assessments of test quality and transparency for customers and the engineering company. In the approach presented in this paper, we aim to increase reproducibility, transparency and measurability of system testing in factory automation by including a human operator in a guided, semi-automatic system testing process via a human machine interface. Through this, the flexibility of the human operator is combined with the precision and resource efficiency of automated testing, not requiring costly simulation models or specialized testing personnel. The approach was developed in accordance with industrial requirements and was qualitatively evaluated with a group of expert using a real industrial automation system.

Impact of CS programs on the quality of test cases generation: an empirical study

Abstract: Background: Although most Computer Science (CS) programs offered by higher education institutions usually include a software engineering course, some works report a lack of formal training in software testing. Aim: With the aim of studying the possible impact of knowledge acquired from CS programs on software testing, this paper reports an investigation composed of four experiments. The experiments conducted in Spain, Mexico and Ecuador examine the quality of test cases (TC) generated using black-box and white-box methods. The subjects of the experiments were undergraduate and graduate students who were exposed to different levels of CS knowledge. Method: We pool together the data from the four experiments and apply logistic regression to investigate possible relations of the quality of test cases with students' level of exposure to CS knowledge. Results: The quality of test cases generated by students depend significantly on the amount of CS program studied. The odds of generating test cases that reveal failures against those that do not reveal decrease when students are exposed to a low level of CS knowledge. Conclusions: Software testing plays a key role in what is an increasingly complex process of developing and maintaining software products today. The results of our empirical study provide evidence in favor of greater formal training in software testing as part of CS programs.

Automation Testing In Software Organization

Abstract: In software testing, automation testing plays a great role for improving test efficiency of the software testing team. Sometimes manual testing may not be effective due to its inconsistency, lack of coverage and none repeating in nature. To overcome this Test automation is used in software industry. In this paper we will discuss about Test Automation its pre-requisites, working steps, when to use automation testing, benefits over manual testing and selection of test cases to automate Thus there are a number of testing tools available in the market out of which we will also discuss about Selenium automation tool

A preliminary analysis of various testing techniques in Agile development - a systematic literature review

Abstract: There are numerous software development and testing methods, tools and techniques have emerged over the past few decades with the main objective are to enhance the software quality. In Small and Medium size Enterprises (SME's), Agile methods have been gaining acceptance but quality of the product it produced is remains as the major concern because the tesing methods and standards are not improved. In Agile development process, though there are many testing techniques are exists, SME's cannot afford to follow the traditional methods of testing process, because of high cost and it is a time consuming process. In this research paper, only the issues surfaced in Regression Testing and Automated Testing are analysed and proposed that exploratory testing (ET) methods either can be used as an alternative or can be integrate along with the existing testing methods. However, there is not enough research work is conducted on Exploratory testing and emphasized the importance to focus more research work to be carried out in ET.

Applying agile practices to avoid chaos in User Acceptance Testing: A case study

Abstract: Agile practices have gained increasing popularity in Information Technology (IT), Education, Marketing, and Advertising industry, as it brings quality products into the market faster. Scrum, Lean Development, and Extreme Programming are the most commonly considered processes under the Agile umbrella. Scrum or scrum variants form a high performance, collaborative team to handle projects that are more complex. We examine the applicability of the scrum framework to a large-scale revenue management system for User Acceptance Testing (UAT). Industry believes integration and acceptance testing is not easy to perform within the scrum framework. Nevertheless, very little is explored about the acceptance testing in Agile practices. We fill this gap by empirically evaluating UAT of a complex, large-scale system (in a public sector organization) to showcase the applicability of scrum framework. While the initial UAT team consisted of 100 domain experts, no process was defined for the UAT. This made it easier to streamline the UAT into the scrum framework. Once the scrum framework was introduced significant improvements in the UAT team was experienced with improved morale, productivity, efficiency, and time to market while having a smooth flow.

Acceptance Probability (P a) Analysis for Process Validation Lifecycle Stages.

Abstract: This paper introduces an innovative statistical approach towards understanding how variation impacts the acceptance criteria of quality attributes. Because of more complex stage-wise acceptance criteria, traditional process capability measures are inadequate for general application in the pharmaceutical industry. The probability of acceptance concept provides a clear measure, derived from specific acceptance criteria for each quality attribute. In line with the 2011 FDA Guidance, this approach systematically evaluates data and scientifically establishes evidence that a process is capable of consistently delivering quality product. The probability of acceptance provides a direct and readily understandable indication of product risk. As with traditional capability indices, the acceptance probability approach assumes that underlying data distributions are normal. The computational solutions for dosage uniformity and dissolution acceptance criteria are readily applicable. For dosage uniformity, the expected AV range may be determined using the slo and shi values along with the worst case estimates of the mean. This approach permits a risk-based assessment of future batch performance of the critical quality attributes. The concept is also readily applicable to sterile/non sterile liquid dose products. Quality attributes such as deliverable volume and assay per spray have stage-wise acceptance that can be converted into an acceptance probability. Accepted statistical guidelines indicate processes with Cpk > 1.33 as performing well within statistical control and those with Cpk 1.33 is associated with a centered process that will statistically produce less than 63 defective units per million. This is equivalent to an acceptance probability of >99.99%.

Analysis of reliability characteristics in the acceptance sampling tests

Abstract: Until now, various acceptance reliability sampling plans have been developed based on different life tests of items. However, the statistical effect of the acceptance sampling tests on the reliability characteristic of the lots accepted in the test has not been appropriately addressed. In this paper, we deal with an acceptance reliability sampling plan under a ‘general framework’ and discuss the corresponding statistical effect of the acceptance sampling tests. The lifetime of the population before the acceptance test and that of population ‘conditional on the acceptance’ in the sampling test are stochastically compared. The improvement of reliability characteristics of the population conditional on the acceptance in the sampling test is precisely analyzed.

Sequential Secret Sharing Scheme Based on Level Ordered Access Structure

Abstract: In Software Industry an application can be released to production only after it has gone through Unit testing, followed by Integration testing, then System testing and finally Acceptance testing. Note here that without the completion of unit testing, integration testing cannot be started and similarly without the completion of integration testing, system testing cannot be started and so on. That is the ordering is important. To realize this or similar kind of activity we need a hierarchial access structure that has in built ordering among the levels. Existing access structures fail to realize this scenario as they are short of enforcing the required ordering. The purpose of this pa- per is to propose an access structure that caters to this kind of scenarios and come up with schemes that realize this access structure. We call this new access structure as Level Ordered Access Structure (LOAS) and the schemes that realize this access structure as Level Ordered Secret Sharing (LOSS) schemes.