Showing papers on "Acceptance testing published in 2004"

Toward preprototype user acceptance testing of new information systems: implications for software project management

Abstract: Errors in requirements specifications have been identified as a major contributor to costly software project failures. It would be highly beneficial if information systems developers could verify requirements by predicting workplace acceptance of a new system based on user evaluations of its specifications measured during the earliest stages of the development project, ideally before building a working prototype. However, conventional wisdom among system developers asserts that prospective users must have direct hands-on experience with at least a working prototype of a new system before they can provide assessments that accurately reflect future usage behavior after workplace implementation. The present research demonstrates that this assumption is only partially true. Specifically, it is true that stable and predictive assessments of a system's perceived ease of use should be based on direct behavioral experience using the system. However, stable and behaviorally predictive measures of perceived usefulness can be captured from target users who have received information about a system's functionality, but have not had direct hands-on usage experience. This distinction is key because, compared to ease of use, usefulness is generally much more strongly linked to future usage intentions and behaviors in the workplace. Two longitudinal field experiments show that preprototype usefulness measures can closely approximate hands-on based usefulness measures, and are significantly predictive of usage intentions and behavior up to six months after workplace implementation. The present findings open the door toward research on how user acceptance testing may be done much earlier in the system development process than has traditionally been the case. Such preprototype user acceptance tests have greater informational value than their postprototype counterparts because they are captured when only a relatively small proportion of project costs have been incurred and there is greater flexibility to modify a new system's design attributes. Implications are discussed for future research to confirm the robustness of the present findings and to better understand the practical potential and limitations of preprototype user acceptance testing.

Reviewing 25 Years of Testing Technique Experiments

Abstract: Mature knowledge allows engineering disciplines the achievement of predictable results. Unfortunately, the type of knowledge used in software engineering can be considered to be of a relatively low maturity, and developers are guided by intuition, fashion or market-speak rather than by facts or undisputed statements proper to an engineering discipline. Testing techniques determine different criteria for selecting the test cases that will be used as input to the system under examination, which means that an effective and efficient selection of test cases conditions the success of the tests. The knowledge for selecting testing techniques should come from studies that empirically justify the benefits and application conditions of the different techniques. This paper analyzes the maturity level of the knowledge about testing techniques by examining existing empirical studies about these techniques. We have analyzed their results, and obtained a testing technique knowledge classification based on their factuality and objectivity, according to four parameters.

Reliability verification using proof pile load tests

Abstract: Proof pile load tests are an important means to cope with uncertainties in the design and construction of pile foundations. In this paper, a systematic method to incorporate the results of proof load tests not conducted to failure into the design of pile foundations is developed. In addition, illustrative acceptance criteria for driven piles based on proof load tests are proposed for use in a reliability-based design. Finally, modifications to conventional proof test procedures are studied so that the value derived from proof tests can be maximized. Whether or not a proof test is conducted to failure, its results can be used to update the probability distribution of the pile capacity using the method proposed in this paper. Hence, contributions of the proof test can be included in foundation design in a logical manner by considering several load test parameters such as the number of tests, the test load, the factor of safety, and test results. This adds value to proof load tests and warrants improvements in the procedures for acceptance of pile foundations using proof load tests. A larger test load for proof tests, say 1.5 times the predicted pile capacity, is recommended since it will yield more information about the capacity statistics and thus allow for more economical designs.

Software Testing and Continuous Quality Improvement, Second Edition

Abstract: Invokes a Continuous Quality Improvement Approach to Promote Effective Testing Methods It is often assumed that software testing is based on clearly defined requirements and software development standards. However, testing is typically performed against changing, and sometimes inaccurate, requirements. The third edition of a bestseller, Software Testing and Continuous Quality Improvement, Third Edition provides a continuous quality framework for the software testing process within traditionally structured and unstructured environments. This framework aids in creating meaningful test cases for systems with evolving requirements. This completely revised reference provides a comprehensive look at software testing as part of the project management process, emphasizing testing and quality goals early on in development. Building on the success of previous editions, the text explains testing in a Service Orientated Architecture (SOA) environment, the building blocks of a Testing Center of Excellence (COE), and how to test in an agile development. Fully updated, the sections on test effort estimation provide greater emphasis on testing metrics. The book also examines all aspects of functional testing and looks at the relation between changing business strategies and changes to applications in development. Includes New Chapters on Process, Application, and Organizational Metrics All IT organizations face software testing issues, but most are unprepared to manage them. Software Testing and Continuous Quality Improvement, Third Edition is enhanced with an up-to-date listing of free software tools and a question-and-answer checklist for choosing the best tools for your organization. It equips you with everything you need to effectively address testing issues in the most beneficial way for your business.

Empirical studies of software testing techniques: challenges, practical strategies, and future research

Abstract: This position paper aims at discussing a number of issues that typically arise when performing empirical studies with software testing techniques. Though some problems are general to all empirical disciplines, software testing studies face a number of specific challenges. Some of the main ones are discussed in sequence below.

Impacts of the Organizational Model on Testing: Three Industrial Cases

Abstract: Quality problems in delivered software systems are causing huge costs to vendors and customers alike. Proper testing is crucial for removing defects and assuring the required reliability of software, but testing tends to be a weak spot in the development process. The problems of testing are often supposed to be mainly technical. However, in our three case studies, the organizational structure of the software engineering organization has shown very significant effects on the testing process and its results. We describe three different organizational models and compare their advantages and disadvantages for testing, according to empirical data.

Towards Generating Acceptance Tests for Product Lines

Abstract: Software product-line engineering aims at improving the efficiency and effectiveness of software development by exploiting the product line members’ commonalities and by controlling their variabilities. The duality of commonalities and variabilities holds for all kinds of assets ranging from requirements specifications over design documents to test cases. A decision model controls the way a product can be distinguished from the rest of the family and is used to extract product-specific information (e.g., product requirements) from the family specifications. Though we traditionally employ decision models for generating code, we aim on capitalizing on the investment for designing the decision model by leveraging it to generate test cases. In this paper we focus on acceptance testing of functions and features, and introduce our approach of using the decision model concept to maintain and generate acceptance test cases for one of our major product lines. Preliminary evaluation of this method demonstrates very promising savings of space and effort as compared to conventional methods.

Acceptance Criteria for Special Precast Concrete Structural Walls Based on Validation Testing

Abstract: The 2003 Edition of the NEHRP Provisions permits, with validation testing, the use of structural (shear) walls of jointed construction in regions of high seismic risk. Acceptance criteria for such validation testing, accompanied by background and commentary, are presented in this article. Lastly, the future direction of non-emulative design of precast concrete walls is discussed.

Suitability of FIT User Acceptance Tests for Specifying Functional Requirements: Developer Perspective

Abstract: The paper outlines an experiment conducted in two different academic environments, in which FIT tests were used as a functional requirements specification. Common challenges for functional requirements specifications are identified, and a comparison is made between how well prose and FIT user acceptance tests are suited to overcoming these challenges from the developer’s perspective. Experimental data and participant feedback are examined to evaluate whether developers can use requirements in the form of FIT tests to create a design and implementation.

Risk Based Acceptance Criteria for Joints Subject to Fatigue Deterioration

Abstract: Different approaches to determine the acceptance criteria for fatigue induced failure of structural systems and components are discussed and compared. The considered approaches take basis in either optimization (societal cost-benefit analysis) or are derived from past and actual practice or codes (revealed preferences). The system acceptance criteria are expressed in terms of the maximal acceptable annual probability of collapse due to fatigue failure. Acceptance criteria for the individual fatigue failure modes are then derived using a simplified system reliability model. The consequence of fatigue failure of the individual joints is related to the overall system by evaluating the change in system reliability given fatigue failure. This is facilitated by the use of a simple indicator, the Residual Influence Factor. The acceptance criteria is thus formulated as a function of the system redundancy and complexity. In addition, the effect of dependencies in the structure on the acceptance criteria are investigated. Finally an example is presented where the optimal allocation of the risk to different welded joints in a jacket structure is performed by consideration of the necessary maintenance efforts.

A case study on regression test suite maintenance in system evolution

Abstract: When a system is maintained, its automated test suites must also be maintained to keep the tests up to date. Even though practice indicates that test suite maintenance can be very costly we have seen few studies considering the actual efforts for maintenance of test-ware. We conducted a case study on an evolving system with three updated versions, changed with three different change strategies. Test suites for automated unit and functional tests were used for regression testing the extended applications. With one change strategy more changes were made in the tests code than in the system that was tested, and with another strategy no changes were needed for the unit tests to work.

The Video Store Revisited Yet Again: Adventures in GUI Acceptance Testing

Abstract: Acceptance testing for graphical user interfaces has long been recognised as a hard problem. At the same time, a full suite of acceptance tests written by the Onsite Customer has been a key principle of XP since it began [1]. It seems, however, that practice has lagged behind theory, with many practitioners still reporting weak or no acceptance testing. At XP2003, we presented our successes with text-based acceptance testing of a batch program [2]. In the past year we have extended this approach to apply to a user interface. We have developed an approach based on simulation of user actions via a record/replay layer between the application and the GUI library, generating a high-level script that functions as a use-case scenario, and using our text-based approach for verification of correctness. We believe this is an approach to GUI acceptance testing which is both customer- and developer-friendly.

7 – Affective Testing

Abstract: Publisher Summary This chapter discusses a sensory evaluation model for measuring product acceptance. Sensory acceptance represents the third and final phase of test resources along with discrimination and descriptive analysis tests. As a resource, it provides continuity between the controlled laboratory environment and the typical product use situation, such as home, restaurant, and so forth. The types of sensory acceptance tests include the laboratory, central location, and home-use. All of these test types follow accepted sensory evaluation procedures and practices. The sensory acceptance test is a very cost-effective resource and plays a major role in the development of successful products. The chapter discusses two methods to measure preference and acceptance—the paired-comparison test and the nine-point hedonic scale. The subjects participating in a sensory acceptance test should be qualified based on typical demographic and usage criteria or preference scores from survey data. The chapter also discusses the relationship between the sensory acceptance and the larger-scale marketing research models, and the role of the sensory acceptance model and its responsibilities in the product evaluation.

Testing object-oriented software

Abstract: The best approach to testing object-oriented software depends on many factors: the application-under-test, the development approach, the organization of the development and quality assurance teams, the criticality of the application, the development environment and the implementation language(s), the use of design and language features, project timing and resource constraints. Nonetheless, we can outline a general approach that works in stages from independent consideration of classes and their features to consideration of their interactions. A coherent strategy would include three main phases: intraclass, interclass, and system and acceptance testing.

Generating regression tests via model checking

Abstract: During software maintenance, regression testing is an expensive but critical activity to ensure that modified versions of the system do not "regress". As software evolves, chances are that not only the implementation changes, but that the specification of the system changes too. We argue that guiding regression testing by the system specifications generated early in the software lifecycle will be more accurate than extracting specification from source code, and more cost effective. Model checking is used to reason about the truth of statements about the system specification. We use a model checker as part of a highly automated regression test generation tool, providing a practical approach to specification based regression testing. Features of our approach that support different specification description languages and different test coverage criteria are also presented.

Agile Methods in Software Engineering Education

Abstract: Agile methods such as extreme programming (XP) are becoming increasingly important for the rapid development of software applications. However, there is a tradeoff in using agile methods. Often they lack in providing a systematic guidance (i.e., a sound description of roles, artifacts, and activities), and thus, require disciplined and experienced developers. Are the promised benefits of agile methods still valid if they are applied by novice (student) developers? To gain some experience, we performed a study on teaching students agile software development with XP. Students performed a small software development project at the University of Kaiserslautern to collect some lessons learned. One result is that although agile approaches are easy to learn and quickly produce results, they are not the best starting point in training software development. The quality of the resulting system, at least in our experience, heavily depends on the discipline of the developers and their background and experience in software development.

Online testing of Web-based applications

Abstract: Testing of Web-based applications is difficult. Online testing of Web-based applications is more difficult. We argue the motivation of online testing, especially those of Web-based applications. We explain what online testing is, what it concerns, and the difference between online testing and offline testing. We introduce some key problems, such as how to reduce the interference to the normal services of system, how to improve the efficiency of online testing, etc. In the end, current research works including our primary research work were introduced.

Model-Checking Plus Testing: From Software Architecture Analysis to Code Testing

Abstract: Software Model-Checking and Testing are some of the most used techniques to analyze software systems and identify hidden faults. While software model-checking allows for an exhaustive and automatic analysis of the system expressed through a model, software testing is based on a clever selection of “relevant” test cases, which may be manually or automatically run over the system.

Software Reuse and the Test Development Process: A Combined Approach

Abstract: Testing is the most time consuming activity in the software development process. The effectiveness of software testing is primarily determined by the quality of the testing process. Software reuse, when effectively applied, has shown to increase the productivity of a software process and enhance the quality of software by the use of components already tested on a large scale. While reusability of testing material and tests has a strong potential, few if any approaches have been proposed that combine these two aspects. Reusability of testing materials is desired, when test development is complex and time-consuming. This is the case, for example, in testing with test-specific languages, such as the TTCN-3. To meet these needs, this paper suggests a test development process model that takes software reuse techniques and activities into account. This paper shows further that in order to produce reusable test material, the software entities must be expressed in terms of features, in which the test materials are attached to. Also, the software components must be designed with reuse in mind when reusable test material is desired. The scope of the proposed test development approach is on the unit and integration testing, because the outcome of higher levels of testing is typically dependent on the tester’s subjective judgment.

Acceptance Testing vs. Unit Testing: A Developer’s Perspective

Abstract: Acceptance testing is one of the most important XP practices and yet it is often neglected or perceived as “too hard”. But what if acceptance tests were like unit tests? This paper provides distilled practical advice in a context familiar to XP developers describing how you can start getting acceptance test-infected on your project.

Acceptance Test Driven Planning

Abstract: The experience of XP planning for many is not a successful one. We have found that by making acceptance tests not only central to the definition of a story but central to our process itself, they can be used to drive the entire development. This paper describes an adaptation, or evolution to XP style planning based around acceptance testing which takes the existing planning practices (with some additions) and organises them in a way that we believe can lead to better planning and more predictable results.

Generative Acceptance Testing for Difficult-to-Test Software

Abstract: While there are many excellent acceptance testing tools and frameworks available today, this paper presents an alternative approach, involving generating code from tests specified in a declarative tabular format within Excel spreadsheets. While this is a general approach, it is most applicable to difficult-to-test situations. Two such situations are presented: one involving complex fixture setup, and another involving complex application workflow concerns.

Review of computer-based test interpretation software for the MMPI-2.

Abstract: (2004). Review of Computer-Based Test Interpretation Software for the MMPI-2. Journal of Personality Assessment: Vol. 83, No. 1, pp. 78-83.

Gas Turbine Test Parameters Corrections Including Operation With Water Injection

Abstract: Methods for correcting data from gas turbine acceptance testing are discussed, focusing on matters which are not sufficiently covered by existing standards. First a brief outline is presented of the reasoning on which correction curves are based. Typical performance correction curves are shown together with the method of calculating mass flow, rate and turbine inlet temperature from test data. A procedure for verifying guarantee data at a specific operating point is then given. Operation with water injection is then considered. Ways of correcting performance data are proposed, and the reasoning of following such a procedure is discussed. Corrections for water amount as well as power and efficiency are discussed. Data from actual gas turbine testing are used to demonstrate how the proposed procedure can be applied in actual cases of acceptance testing.

Formal Approaches to Software Testing

Abstract: Object-oriented frameworks are designed to provide functionality common to a variety of applications. Developers use these frameworks in building their own specialized applications, often without having the source code of the original framework. Unfortunately, the interactions between the framework components and the new application code can lead to behaviors that could not be predicted even if valid black-box specifications were provided for the framework components. What is needed are grey-box specifications that include information about sequences of method calls made by the original framework code. Our focus is on how to test frameworks against such specifications, which requires the ability to monitor such method calls made by the framework during testing. The problem is that without the source code of the framework, we cannot resort to code instrumentation to track these calls. We develop an approach that allows us to do this, and demonstrate it on a simple case study.