Artificial neural networks (ANNs) constitute a class of flexible nonlinear models designed to mimic biological neural systems. In this entry, we introduce ANN using familiar econometric terminology and provide an overview of ANN modeling approach and its implementation methods. † Correspondence: Chung-Ming Kuan, Institute of Economics, Academia Sinica, 128 Academia Road, Sec. 2, Taipei 115, Taiwan; ckuan@econ.sinica.edu.tw. †† I would like to express my sincere gratitude to the editor, Professor Steven Durlauf, for his patience and constructive comments on early drafts of this entry. I also thank Shih-Hsun Hsu and Yu-Lieh Huang for very helpful suggestions. The remaining errors are all mine.

Artificial neural networks

Mutation Testing is a fault-based software testing technique that has been widely studied for over three decades The literature on Mutation Testing has contributed a set of approaches, tools, developments, and empirical results This paper provides a comprehensive analysis and survey of Mutation Testing The paper also presents the results of several development trend analyses These analyses provide evidence that Mutation Testing techniques and tools are reaching a state of maturity and applicability, while the topic of Mutation Testing itself is the subject of increasing interest

An Analysis and Survey of the Development of Mutation Testing

Regression testing is a testing activity that is performed to provide confidence that changes do not harm the existing behaviour of the software. Test suites tend to grow in size as software evolves, often making it too costly to execute entire test suites. A number of different approaches have been studied to maximize the value of the accrued test suite: minimization, selection and prioritization. Test suite minimization seeks to eliminate redundant test cases in order to reduce the number of tests to run. Test case selection seeks to identify the test cases that are relevant to some set of recent changes. Test case prioritization seeks to order test cases in such a way that early fault detection is maximized. This paper surveys each area of minimization, selection and prioritization technique and discusses open problems and potential directions for future research. Copyright © 2010 John Wiley & Sons, Ltd.

Regression testing minimization, selection and prioritization: a survey

Where the creation, understanding, and assessment of software testing and regression testing techniques are concerned, controlled experimentation is an indispensable research methodology. Obtaining the infrastructure necessary to support such experimentation, however, is difficult and expensive. As a result, progress in experimentation with testing techniques has been slow, and empirical data on the costs and effectiveness of techniques remains relatively scarce. To help address this problem, we have been designing and constructing infrastructure to support controlled experimentation with testing and regression testing techniques. This paper reports on the challenges faced by researchers experimenting with testing techniques, including those that inform the design of our infrastructure. The paper then describes the infrastructure that we are creating in response to these challenges, and that we are now making available to other researchers, and discusses the impact that this infrastructure has had and can be expected to have.

/pdf/supporting-controlled-experimentation-with-testing-pe7sdxpo8h.pdf

Supporting Controlled Experimentation with Testing Techniques: An Infrastructure and its Potential Impact

Empirical studies in software testing research may not be comparable, reproducible, or characteristic of practice. One reason is that real bugs are too infrequently used in software testing research. Extracting and reproducing real bugs is challenging and as a result hand-seeded faults or mutants are commonly used as a substitute. This paper presents Defects4J, a database and extensible framework providing real bugs to enable reproducible studies in software testing research. The initial version of Defects4J contains 357 real bugs from 5 real-world open source pro- grams. Each real bug is accompanied by a comprehensive test suite that can expose (demonstrate) that bug. Defects4J is extensible and builds on top of each program’s version con- trol system. Once a program is configured in Defects4J, new bugs can be added to the database with little or no effort. Defects4J features a framework to easily access faulty and fixed program versions and corresponding test suites. This framework also provides a high-level interface to common tasks in software testing research, making it easy to con- duct and reproduce empirical studies. Defects4J is publicly available at http://defects4j.org.

/pdf/defects4j-a-database-of-existing-faults-to-enable-controlled-1x92gx1c9e.pdf

Defects4J: a database of existing faults to enable controlled testing studies for Java programs

The use of relevant metrics of software systems could improve various software engineering tasks, but identifying relationships among metrics is not simple and can be very time consuming. Recommender systems can help with this decision-making process, many applications have utilized these systems to improve the performance of their applications. To investigate the potential benefits of recommender systems in regression testing, we implemented an item-based collaborative filtering recommender system that uses user interaction data and application change history information to develop a test case prioritization technique. To evaluate our approach, we performed an empirical study using three web applications with multiple versions and compared four control techniques. Our results indicate that our recommender system can help improve the effectiveness of test prioritization.

A Collaborative Filtering Recommender System for Test Case Prioritization in Web Applications

Web applications undergo frequent changes. These changes can be due to the addition of new features or the modification of existing features to support customer requests or to patch faults in the system. Given that Web applications have a large surface area subject to attack, changes often include security fixes either in response to malicious attacks or to forestall such attacks. Effective regression testing should ensure that any change does not disable existing features or compromise security. Executing the entire regression test suite takes time and consumes many resources. One approach is to focus regression test efforts only on code paths that were modified in the new version. Such code paths can be identified using tools such as PHP Analysis and Regression Testing Engine (PARTE). In this paper, we extend this approach to test selection where a subset of existing tests that cover the modified code paths can be detected. To further reduce the amount of regression testing needed, we used PARTE’s reusable constraint value information to identify tests that can be reused against the new version without having to modify the input test values. We performed an empirical study to determine whether test selection data combined with reusable constraint values would further improve the turnaround time for regression tests. Results from the experiment conducted on four Hypertext Preprocessor (PHP) web applications demonstrate that this approach is effective in reducing the cost of regression testing of frequently patched Web applications.

An efficient regression testing approach for PHP Web applications using test selection and reusable constraints

Regression testing is a necessary maintenance activity that can ensure high quality of the modified software system, and a great deal of research on regression testing has been performed. Most of the studies performed to date, however, have evaluated regression testing techniques under the limited context, such as a short-term assessment, which do not fully account for system evolution or industrial circumstances. One important issue associated with a system lifetime view that we have overlooked in past years is the effects of residual defects -- defects that persist undetected -- across several releases of a system. Depending on an organization's business goals and the type of system being built, residual defects might affect the level of success of the software products. In this paper, we conducted an empirical study to investigate whether regression testing techniques are effective in reducing the occurrence and persistence of residual defects across a system's lifetime, in particular, considering test case prioritization techniques. Our results show that heuristics can be effective in reducing both the occurrence of residual defects and their age. Our results also indicate that residual defects and their age have a strong impact on the cost-benefits of test case prioritization techniques.

The Effectiveness of Regression Testing Techniques in Reducing the Occurrence of Residual Defects

The use of risk information can help software engineers identify software components that are likely vulnerable or require extra attention when testing. Some studies have shown that the requirements risk-based approaches can be effective in improving the effectiveness of regression testing techniques. However, the risk estimation processes used in such approaches can be subjective, time-consuming, and costly. In this research, we introduce a fuzzy expert system that emulates human thinking to address the subjectivity related issues in the risk estimation process in a systematic and an efficient way and thus further improve the effectiveness of test case prioritization. Further, the required data for our approach was gathered by employing a semi-automated process that made the risk estimation process less subjective. The empirical results indicate that the new prioritization approach can improve the rate of fault detection over several existing test case prioritization techniques, while reducing threats to subjective risk estimation.

A Systematic Requirements and Risks-Based Test Case Prioritization Using a Fuzzy Expert System

Pervasive telemetry in modern applications is providing new possibilities in the application of regression testing techniques. Similar to how research in bioinformatics is leading to personalized medicine, tailored to individuals, usage telemetry in modern software allows for custom regression testing, tailored to the usage patterns of an installation. By customizing regression testing based on software usage, the effectiveness of regression testing techniques can be greatly improved, leading to reduced testing costs and enhanced detection of defects that are most important to that customer. In this research, we introduce the concept of fingerprinting software usage patterns through telemetry. Weprovide various algorithms tocompute fingerprints and conduct an empirical study that shows that fingerprints are effective in identifying distinct usage patterns. Further, we discuss how usage fingerprints can be used to improve regression test prioritization run time by over 30 percent compared to traditional prioritization techniques.

Hyunsook Do

Papers

A Collaborative Filtering Recommender System for Test Case Prioritization in Web Applications

An efficient regression testing approach for PHP Web applications using test selection and reusable constraints

The Effectiveness of Regression Testing Techniques in Reducing the Occurrence of Residual Defects

A Systematic Requirements and Risks-Based Test Case Prioritization Using a Fuzzy Expert System

Customized Regression Testing Using Telemetry Usage Patterns