Showing papers on "Acceptance testing published in 2000"

Testing: a roadmap

Abstract: Testing is an important process that is performed to support quality assurance. Testing activities support quality assurance by gathering information about the nature of the software being studied. These activities consist of designing test cases, executing the software with those test cases, and examining the results produced by those executions. Studies indicate that more than fty percent of the cost of software development is devoted to testing, with the percentage for testing critical software being even higher. As software becomes more pervasive and is used more often to perform critical tasks, it will be required to be of higher quality. Unless we can nd ecient ways to perform eective testing, the percentage of development costs devoted to testing will increase signicantly. This report briefly assesses the state of the art in software testing, outlines some future directions in software testing, and gives some pointers to software testing resources.

What is software testing? And why is it so hard?

Abstract: The author sheds some light on why testing today's software products is so challenging, and he identifies several solid approaches that all testers should be able to thoughtfully apply. The effective tester has a rich toolkit of fundamental testing techniques, understands how the product will be used in its operating environment, and has a nose for where subtle bugs might lurk in the product and a bag of tricks for flushing them out. The methods described can help testers provide a sensible answer to the question of what they really mean when they say they have finished testing a software system.

Experience with performance testing of software systems: issues, an approach, and case study

Abstract: An approach to software performance testing is discussed. A case study describing the experience of using this approach for testing the performance of a system used as a gateway in a large industrial client/server transaction processing application is presented.

Comparing functional and structural tests

Abstract: This paper describes an experimental study to understand issues and requirements for structural-based testing using low cost testers, compared to functional-based testing using expensive testers. Several studies have been directed at the effectiveness of various test methods, but none explicitly addressed issues involved in attempting to replace functional vectors with scan vectors and none carried the experiment further by placing defective chips into systems and running system tests. This paper describes the results of such an experiment and offers insight into necessary requirements for reduction or elimination of functional tests.

Mutation testing applied to Estelle specifications

Abstract: Many researchers have pursued the establishment of a low-cost, effective testing and validation strategy at the program level as well as at the specification level Mutation Testing is an error-based approach, originally introduced for program testing, that provides testers a systematic way to evaluate how good a given tester is Some studies have also investigated its use to generate testers In this article the application of Mutation Testing for validating Estelle specifications is proposed A mutation operator set for Estelle-one of the crucial points for effectively applying Mutation Testing-is defined, addressing: the validation of the behavior of the modules, the communication among modules and the architecture of the specification In this scope, these operators can be taken as a fault model Considering this context, a strategy for validating Estelle-based specification is proposed and exemplified using the alternating-bit protocol

Testing Applications on the Web: Test Planning for Internet-Based Systems

Abstract: Welcome to Web Testing. Web Testing versus Traditional Testing. METHODOLOGY AND TECHNOLOGY. Software Testing Basics. Networking Basics. Web Application Components. Test Planning Fundamentals. Sample Application. Sample Test Plan. TESTING PRACTICES. User Interface Tests. Functional Tests. Database Tests. Help Tests. Installation Tests. Configuration and Compatibility Tests. Web Security Concerns. Performance, Load, and Stress Tests. Web Testing Tools. Finding Additional Information. Appendices. Index.

Quality assurance for a treatment planning system in scanned ion beam therapy

Abstract: Conformal radiation therapy using dynamic beam delivery systems like scanned ion beams requires concise quality assurance procedures for the complete treatment planning process. For the heavy ion therapy facility at GSI, Darmstadt, a quality assurance program for the treatment planning system (TPS) has been developed. It covers the development and updating of software, data protection and safety, and the application of soft- and hardware. The tests also apply to the geometrical precision of imaging devices and the geometrical and dosimetrical verification of dose distributions in different phantoms. The quality assurance program addresses acceptance and constancy tests of the treatment planning program. Results of the acceptance tests served as a basis for its governmental approval. Two main results of the acceptance tests are representative for the overall performance of the system. (1) The geometrical uncertainty that could be achieved for the target point definition, setup accuracy, field contouring, and field alignment is typically 1.5 mm. The uncertainty for the setup verification using digitally reconstructed radiographs (DRR’s) is limited to 2 mm. (2) The mean deviations between measured and planned dose values is 3% for standardized cases in a water phantom and up to 6% for more complicated treatment configurations.

Risk metric for testing software

Abstract: A computer-implemented system and method for improving the efficiency of testing a process by focusing the testing in the highest risk areas of the process. One application of this computer-implemented system and method is in software testing to improve the effectiveness of software testing by focusing the testing efforts on the parts of the software that have a highest risk of failure and also by focusing the testing on the types of tests needed to reduce the risk.

Quality management of medical physics issues at the German heavy ion therapy project.

Abstract: For the commissioning and operation of the German Heavy Ion Therapy Project a quality assurance program was developed and successfully applied. The complete radiotherapy process using heavy ions was carefully analyzed and divided into three areas related to beam delivery and control, safety-interlock system and medical physics issues. In this paper, the medical physics issues are addressed. Since the irradiation with heavy ions is a nonstandard modality, new concepts and ion specific tests were developed. As far as possible, national and international standard specifications for radiotherapy were adopted. For each aspect, a performance characteristic and a corresponding acceptance test were introduced. In addition, test characteristics for the constancy tests were established. For all tests, intervention thresholds and test frequencies were specified. Using the described protocol of acceptance tests, the commissioning was passed successfully. The heavy ion irradiation facility was approved by the governmental authorities on the basis of these test results. During clinical operation, constancy tests are performed at the beginning of each treatment period, in order to maintain the quality found during the acceptance tests. Up to now, 48 patients have been treated within 6 treatment periods of 4 weeks each. The concepts used and the tests developed for the quality assurance program may serve as an example of how to introduce systematically a quality assurance program for a new treatment modality.

Platform-independent and tool-neutral test descriptions for automated software testing

Abstract: Current automatic test execution techniques are sensitive to changes in program implementation. Moreover, different test descriptions are required by different testing tools. As a result, it is difficult to maintain or port test descriptions. To address this problem, we developed TestTalk, a comprehensive testing language. TestTalk test descriptions are platform-independent and tool-neutral. The same software test in TestTalk can be automatically executed by different testing tools on different platforms. The goal of TestTalk is to make software test descriptions, which represent a significant portion of a software project, last as long as the software project.

Reliability and performance testing of photovoltaic modules

Abstract: In recent years there has been a paradigm shift in photovoltaic module testing from reliability to qualification testing. This has raised several questions as to whether or not reliability testing is still required. This paper discusses why reliability testing is still a necessary and integral part of product development and deployment. It also discusses new test procedures that have been developed or are currently under development to improve and enhance reliability testing.

Improving software quality using statistical testing techniques

Abstract: Cleanroom usage-based statistical testing techniques have been incorporated into the software development process for a program in the Electronic Systems business of Raytheon Company. Cost-effectively improving the quality of software delivered into systems integration was a driving criterion for the program. Usage-based statistical testing provided the capability to increase the number of test cases executed on the software and to focus the testing on expected usage scenarios. The techniques provide quantitative methods for measuring and reporting testing progress, and support managing the testing process. This article discusses the motivation and approach for adopting usage-based statistical testing techniques, and experiences using these testing techniques.

Successful implementation of structured testing

Abstract: This paper describes techniques developed to successfully implement structured testing into the manufacturing flow. Analysis and discussion of the main techniques will be presented to show how these techniques are optimized through systematic learning.

ILLISIM program for end-result specification development

Abstract: A new computer simulation program, called ILLISIM, is presented. ILLISIM allows the user to conduct a comprehensive assessment of agency and contractor risks over a wide variety of end-result specification (ERS) parameters. ILLISIM was created to assist the Illinois Department of Transportation (IDOT) in developing an ERS for asphalt pavement construction. The focus is on the use of ILLISIM to explore the possibility of using the nuclear density gauge as a basis for acceptance testing of as-constructed pavement density. Preliminary simulation results from ILLISIM suggest that it may be feasible to use the nuclear gauge for acceptance if stratified random sampling techniques and adequate test replications are used. However, this is true only if test bias is kept under control, for example, if a representative nuclear gauge-to-core correlation is maintained throughout the project. ILLISIM can be used to study the effects of changing device bias, which should be investigated in future work. The results also indicate that a larger sample size will be necessary to increase accuracy and minimize disputes when marginal quality levels are detected.

Measuring software product quality during testing

Abstract: Quality requirements of software products are often described in vague and broad terms. As a consequence it makes it difficult for software engineers to determine how quality influences their assignment and it is almost impossible for test engineers to evaluate the quality of the software product as no concrete and quantitative reference, of what quality in that context means, exists. This paper describes a possible way to define and measure software product quality. The authors have applied this method during the development of copier/printer controller software at Oce Technologies B.V., a Dutch developer and manufacturer of copying and printing equipment.

SPRAE: A Framework for Teaching Software Testing in the Undergraduate Curriculum

Abstract: For the past three years undergraduate students at Florida A&M University have been exposed to the practice of software testing, both as part of the senior projects course, and in an elective course in software testing. Because these courses provide a mere introduction to software testing, the students are presented a framework for understanding the theory, management and practice of testing and quality assurance. This paper defines this framework, SPRAE (Specification-PremeditationRepeatability-Accountability-Economy). We show that SPRAE provides a simple methodology for teaching testing, and it provides an evaluative model for assessing organizational practice of SQA/testing. Finally, we show how this framework is being used to establish the software testing laboratory (TestLab) environment in which students learn the art and science of software testing

Developing cone calorimeter acceptance criteria for materials used in high speed craft

Abstract: This paper presents an overview of a research programme to develop reaction-to-fire acceptance criteria for materials tested in the cone calorimeter. This work, sponsored by the U.S. Coast Guard, includes the testing of eight composite materials and one textile wall covering in several standard test methods. Materials were tested in the ISO 9705 room corner test, the cone calorimeter, the International Maritime Organization's surface flammability test apparatus, the smoke chamber, and in real-scale configurations as furniture items and overhead luggage racks. Summary data from these tests are presented, with particular emphasis on correlations between the room corner test and the cone calorimeter. Data from this research programme were used to develop cone calorimeter acceptance criteria for materials used in furniture and other room contents in high-speed craft. Future work will include more data analysis and fire growth modelling in order to further develop acceptance criteria for other materials, especially compartment Linings. This programme has served as an example of how well-planned and coordinated research can be effective tool in the evaluation of existing regulations and the development of new safety standards, helping to ensure public safety through technically justified standards.

Software Testing & Diagnostics: Theory & Practice

Abstract: The introduction into the field of software testing, automated software testing and diagnostics will be given together with explanation of fundamental terminology. The viewpoint of quality theory will be stressed. Presented state of the art basic concepts of software testing, design of tests, their execution and methods of test evaluation will be selected according to their practical usage. The methodology will be demonstrated on case studies developed during practical software testing and diagnostics projects for large international companies in the field of industrial automation and medical instrumentation. The paper will be concluded with a summary of practical experience.

Improving a product with usage-based testing

Abstract: This paper presents the introduction and application of a usage-based testing approach for the acceptance of an aircraft traffic flow management system by the FAA. The FAA needed an approach to provide more effective testing than that typically achieved in their requirements-based acceptance testing. Testing in a manner more consistent with operational usage was desired. A testing technology that was new to the organization was introduced and applied successfully. The approach resulted in a greatly improved product being released to the field, a cost-effective approach for testing, and a commitment to apply the technology across the organization.

Computerized test development software. A comparative review updated.

Abstract: This article provides a comparative review of nine commercially available software programs: A+ Test Manager and Test Taker, CATSoftware, Diploma 97, Examaker for Windows, FastTEST, Microtest Pro, Pedagogue, Question Mark, and Test Construction Set. Issues for using test development software are covered including security, pedagogical uses, and test design and administration. Nurse educators are encouraged to consider the unique needs of each setting when selecting a software package to support test development and administration.

Developing and testing algorithms for stopping testing, screening, run-in of large systems or programs

Abstract: When large hardware-software systems are run-in or an acceptance testing is made, a problem is when to stop the test and deliver/accept the system. The same problem exists when a large software program is tested with simulated operations data. Based on two theses from the Technical University of Denmark, the paper describes and evaluates 7 possible algorithms. Of these algorithms, the three most promising are tested with simulated data. 27 different systems are simulated, and 50 Monte Carlo simulations made on each system. The stop times generated by the algorithm is compared with the known perfect stop time. Of the three algorithms two is selected as good. These two algorithms are then tested on 10 sets of real data. The algorithms are tested with three different levels of confidence. The number of correct and wrong stop decisions are counted. The conclusion is that the Weibull algorithm with 90% confidence level takes the right decision in every one of the 10 cases.