Showing papers on "White-box testing published in 2022"

Automatic Test Case and Test Oracle Generation Based on Functional Scenarios in Formal Specifications for Conformance Testing

Abstract: Testing a program to confirm whether it consistently implements its requirements specification is a necessary but time-consuming activity in software development. Automatic testing based on specifications can significantly alleviate the workload and cost, but faces a challenge of how to ensure that both the user’s concerns in the specification and possible execution paths in the program are all covered. In this paper, we describe a new method, called “Vibration-Method” or simply “V-Method”, for automatic generation of test cases and test oracle from model-based formal specifications, aiming to address this challenge. The proposed method is suitable for testing information systems in which rich data types are used. Supporting the principle of “divide and conquer”, the method provides a specific technique for generating test cases based on functional scenarios defined in the specification, test case generation criteria, automatic test case generation algorithms, and a well-defined mechanism for deriving test oracle. We elaborate on the method by discussing how initial test cases can be automatically generated, how additional necessary test cases are produced using the “vibration” technique, and how a test oracle can be automatically derived for a group of test cases. We also describe a controlled experiment to evaluate the effectiveness of the method and discuss the important issues in relation to the performance and applicability of the method.

A Case in Point: Verification and Testing of a EULYNX Interface

Abstract: We present a case study on the application of formal methods in the railway domain. The case study is part of the FormaSig project, which aims to support the development of EULYNX — a European standard defining generic interfaces for railway equipment — using formal methods. We translate the semi-formal SysML models created within EULYNX to formal mCRL2 models. By adopting a model-centric approach in which a formal model is used both for analyzing the quality of the EULYNX specification and for automated compliance testing, a high degree of traceability is achieved. The target of our case study is the EULYNX Point subsystem interface. We present a detailed catalog of the safety requirements, and provide counterexamples that show that some of them do not hold without specific fairness assumptions. We also use the mCRL2 model to generate both random and guided tests, which we apply to a third-party software simulator. We share metrics on the coverage and execution time of the tests, which show that guided testing outperforms random testing. The test results indicate several discrepancies between the model and the simulator. One of these discrepancies is caused by a fault in the simulator, the others are caused by false positives, i.e. an over-approximation of fail verdicts by our test setup.

Prevention of SQL Injection Attack by Using Black Box Testing

Abstract: SQL injection attacks are diverse and have fast variations, making it convenient to handle them. An SQL attack can affect the entire system to a great extent, adversely affecting the website, as well as data loss and leaking. In the past, enormous efforts have been put forth into handling the attack. This attack consists of a variety of heterogeneous components, which has always made research a challenging attempt. A web application injection attack allows hackers to gain control of a web application by entering invalid input. In the past, researchers found that many applications were vulnerable to such attacks as a result of less efficient entry validation. We propose an automated black box testing program for automating the task of SQL Injection Vulnerability to eradicate SQL injection attacks. The SQLIV is automatically assessed when such attacks are carried out. Also, it uses object-oriented methodology to develop the SQLIV Scanner so that false +ve and -ve results can be reduced. The algorithm’s performance can be compared with existing algorithms and an experimental analysis can be conducted to test and validate its results. A new algorithm is expected to provide more accurate results.

Aplikasi Pengaduan dan Maintenance Elektronik Berbasis Web dengan Metode First in First Out (FIFO) di Universitas Nasional

Abstract: The BRT unit is a part of the National University that manages electronic equipment including repairing or maintenance in the event of a breakdown. Usually, BRT unit technicians carry out the process of repairing electronic equipment based on complaints received from staff/employees. However, so far the complaint process is still manual and recorded using simple tools such as excel, word and ledgers. This causes the process of electronic complaints and maintenance to be slow. Therefore, in this study, a web-based electronic complaint and maintenance application will be developed at the National University. The application was developed based on the waterfall model using the PHP programming language and using a MySQL database using the First in First Out (FIFO) method. The results of this study are the availability of applications that can help the processing of electronic complaints and maintenance data to be more effective and efficient. Testing of the application was carried out using the White-box and Black-box methods and testing on 3 flowcharts including the Staff/Employee Logic flowchart, Admin Logic flowchart and Technician Logic flowchart with white-box testing calculated cyclomatic complacity and resulted in Cyclomotic Complexity = 8, Region = 8 with path Independent Path = 8 where the result shows that all the logic and functionality of the application has been running well.

Evaluating System-Level Test Generation for Industrial Software: A Comparison between Manual, Combinatorial and Model-Based Testing

Abstract: Adequate testing of safety-critical systems is vital to ensure correct functional and non-functional operations. Previous research has shown that testing such systems requires a lot of effort, thus automated testing techniques have found a certain degree of success. However, automated testing has not replaced the need for manual testing, rather a common industrial practice exhibits a balance between automated and manual testing. In this respect, comparing manual testing with automated testing techniques continues to be an interesting topic to investigate. The need for this investigation is most apparent at system-level testing of industrial systems, where there is a lack of results on how different testing techniques perform concerning both structural and system-level metrics such as Modified Condition/Decision Coverage (MC/DC) and requirement coverage. In addition to the coverage, the cost of these techniques will also determine their efficiency and thus practical viability. In this paper, we have developed cost models for efficiency measurement and performed an experimental evaluation of manual testing, model-based testing (MBT), and combinatorial testing (CT) in terms of MC/DC and requirement coverage. The evaluation is done in an industrial context of a safety-critical system that controls several functions on-board the passenger trains. We have reported the dominant conditions of MC/DC affected by each technique while generating MC/DC adequate test suites. Moreover, we investigated differences and overlaps of test cases generated by each of the three techniques. The results showed that all test suites achieved 100% requirement coverage except the test suite generated by the pairwise testing strategy. However, MBT-generated test suites were more MC/DC adequate and provided a higher number of both similar and unique test cases. Moreover, unique test cases generated by MBT had an observable effect on MC/DC, which will complement manual testing to increase MC/DC coverage. The least dominant MC/DC condition fulfilled by the generated test cases by all three techniques is the ‘independent effect of a condition on the outcomes of a decision’. Lastly, the evaluation also showed CT as the most efficient testing technique amongst the three in terms of time required for test generation and execution, but with an added cost parameter of manual identification of expected outcomes. CCS CONCEPTS •Computer systems organization →Embedded software; • Software and its engineering →Software verification and validation.

Testing Learning Media for English Learning Applications Using BlackBox Testing Based on Equivalence Partitions

Abstract: Some testings or testings of an application are very important. So that an application can appear to be running properly or there are still errors in its use. Which later can be improved optimally. In this test, the black box testing technique was used as a test. Black box testing is a system for testing an application by looking at the menu whether it is following what is desired or still needs improvement. The application that will be tested is the Duolingo application which on the play store gets a high rating and is used by more than 12 million smartphone users. The type of testing of the method used, namely the black box, varies, but one of the methods used here is Equivalence Partitions. The Equivalence Partitions technique is a test based on entering data on each form of the system or application used. In each menu, input data on each form will be tested and carried out based on its function, whether it is valid or not. So that by doing this testing the application is getting better and the application must be of good quality according to their respective functions. So that many smartphone users who want to use the application make it easy to use it

A Novel Real Coded Genetic Algorithm for Software Mutation Testing

Abstract: Information Technology has rapidly developed in recent years and software systems can play a critical role in the symmetry of the technology. Regarding the field of software testing, white-box unit-level testing constitutes the backbone of all other testing techniques, as testing can be entirely implemented by considering the source code of each System Under Test (SUT). In unit-level white-box testing, mutants can be used; these mutants are artificially generated faults seeded in each SUT that behave similarly to the realistic ones. Executing test cases against mutants results in the adequacy (mutation) score of each test case. Efficient Genetic Algorithm (GA)-based methods have been proposed to address different problems in white-box unit testing and, in particular, issues of mutation testing techniques. In this research paper, a new approach, which integrates the path coverage-based testing method with the novel idea of tracing a Fault Detection Matrix (FDM) to achieve maximum mutation coverage, is proposed. The proposed real coded GA for mutation testing is designed to achieve the highest Mutation Score, and it is thus named RGA-MS. The approach is implemented in two phases: path coverage-based test data are initially generated and stored in an optimized test suite. In the next phase, the test suite is executed to kill the mutants present in the SUT. The proposed method aims to achieve the minimum test dataset, having at the same time the highest Mutation Score by removing duplicate test data covering the same mutants. The proposed approach is implemented on the same SUTs as these have been used for path testing. We proved that the RGA-MS approach can cover maximum mutants with a minimum number of test cases. Furthermore, the proposed method can generate a maximum path coverage-based test suite with minimum test data generation compared to other algorithms. In addition, all mutants in the SUT can be covered by less number of test data with no duplicates. Ultimately, the generated optimal test suite is trained to achieve the highest Mutation Score. GA is used to find the maximum mutation coverage as well as to delete the redundant test cases.

An Enhanced Parallel Automation Testing Architecture for Test Case Execution

Abstract: Customer requests for incorporating complicated business logic into software applications are common. As a result, as business requirements expand, the demand on the testing team to deliver a high-quality product in a short amount of time grows. Software testing guarantees that customers receive high-quality software. Manual testing becomes difficult since it is time-consuming. Besides, its cost rises as test suite sizes grow. In addition, human mistakes can slip into a system, resulting in losses for the company. Therefore, automation testing is best suited in situations when requirements change frequently, and a large volume of regression testing is required. Automation testing enhances accuracy while also saving the tester's time and the organization's money. The aim of this work is to propose an improved parallel automation testing architecture to significantly decrease testing time. The main problem with parallel testing is the existence of idle nodes that causes an increase in execution time. The proposed architecture solved this problem by running test cases in parallel using Selenium, Docker containers and implementing a dispatcher to ease and faster the process of distributing test cases between the network's nodes. As a result, the proposed architecture decreases testing process time, and no idle nodes exists anymore.

Pengujian Aplikasi Sistem Informasi Akademik Berbasis Web Menggunakan Metode Black Box Testing Equivalence Partitioning

Abstract: To check the specifications of a software, it is necessary to have a test and validation is something that is needed when testing the software. The inappropriateness of the data input process is caused by the validation process that is not optimal, one of the consequences is that the data addition process fails. Therefore, the application can cause harm to the user. Therefore, improving the quality of accurate validation is very necessary. When the testing process finds an error that was not previously known, then the test can be said to be a good test. One of them is the Black Box testing method, in this test the author chose the iEquivalence Partition testing method. Equivalence Partition is a technique used to reduce the number of test cases that exist at the time of testing. Based on the tests that have been carried out, the authors conclude that this software can be used properly, after repairing the errors that have been found. However, this test is not perfect, because it is only done by using a few samples. From the results of this test, it is expected that the application can be used according to user needs.

Machine Learning Model to Predict Automated Testing Adoption

Abstract: Software testing is an activity conducted to test the software under test. It has two approaches: manual testing and automation testing. Automation testing is an approach of software testing in which programming scripts are written to automate the process of testing. There are some software development projects under development phase for which automated testing is suitable to use and other requires manual testing. It depends on factors like project requirements nature, team which is working on the project, technology on which software is developing and intended audience that may influence the suitability of automated testing for certain software development project. In this paper we have developed machine learning model for prediction of automated testing adoption. We have used chi-square test for finding factors’ correlation and PART classifier for model development. Accuracy of our proposed model is 93.1624%.

The Implementation of the Black Box Method for Testing Smart Hajj Application Ministry of Religion

Abstract: This research was conducted to assist the Ministry of Religious Affairs in testing the quality and accuracy of the Haji Pintar application. The main purpose of testing is to design tests that systematically uncover types of errors that have not been detected before. The Haji Pintar application will be tested using the Black Box method thoroughly with respect to application functionality, interfaces , data structures or external database access, performance, initiation and termination errors, usage, benefits, and results obtained from the use of the application. In testing the Haji Smart application, there are 5 types of testing from the Black Box method. The value of the overall effectiveness of the Haji Pintar application is 83.75%. With this value, the Haji Pintar application has been running according to the expected functionality but there are still errors in the application that must be updated immediately regarding the cancellation information menu and the payment information menu, which experienced problems when checking the status of the cancellation process that the congregation submitted and checking the congregation portion. Hajj includes the right to pay off or not.

Use of genetic algorithms in software testing models

Abstract: Testing software is a highly essential part of the software industry. Effective testing techniques must be used in order to make sure that the software can be used efficiently by the end user. In order for the testing techniques to be effective, the right test cases must be generated. This requires huge manpower and time which is not always available. So, evolutionary algorithms like genetic algorithms (GA) may be used to reduce the burden. In this chapter, we present the applications of GAs in two major software testing models, that is, white box testing and black box testing.

Architecture and Methodology of Unit Testing Embedding Pair-Wise Mode for Small Team

Abstract: In this paper, the new organization for unit testing embedding pair-wise mode is proposed with the core thought focused on the cooperation of programmer and tester by “cross-testing”. The typical content of unit testing for the new organizing mode should have three aspects, including self-checking, cross-testing and independent-testing. For cross-testing, executing “pair-wise” mode, mainly tackles data testing, function testing and state testing, which function testing must be done by details and state testing must be considered for completeness. With the specializing of independent-testing, it should be taken as more rigid testing without arbitrariness. Consequently, strategy and measure are addressed for data testing focusing on boundary testing and function/state testing. And organizing method of procedure and key points of tackling unit testing are investigated for the new organizing mode. In order to assess the validity of our study and approach, a series of actual examples are demonstrated for GUI software. The result indicates that the execution of unit testing for the new organizing mode is effective and applicable.

Towards using coupling measures to guide black‐box integration testing in component‐based systems

Abstract: In component‐based software development, integration testing is a crucial step in verifying the composite behaviour of a system. However, very few formally or empirically validated approaches are available for systematically testing if components have been successfully integrated. In practice, integration testing of component‐based systems is usually performed in a time‐ and resource‐limited context, which further increases the demand for effective test selection strategies. In this work, we therefore analyse the relationship between different component and interface coupling measures found in literature and the distribution of failures found during integration testing of an automotive system. By investigating the correlation for each measure at two architectural levels, we discuss its usefulness to guide integration testing at the software component level as well as for the hardware component level where coupling is measured among multiple electronic control units (ECUs) of a vehicle. Our results indicate that there is a positive correlation between coupling measures and failure‐proneness at both architectural level for all tested measures. However, at the hardware component level, all measures achieved a significantly higher correlation when compared to the software‐level correlation. Consequently, we conclude that prioritizing testing of highly coupled components and interfaces is a valid approach for systematic integration testing, as coupling proved to be a valid indicator for failure‐proneness.

Pengujian Black Box menggunakan teknik Equivalence Partitions pada Aplikasi Sistem Pemilihan Karyawan Terbaik

Abstract: In an application, a test is needed, to find out whether the application has errors or needs to be developed to be better. The Black Box Testing technique has been widely applied by researchers to test an application program that is appropriate or not. The software that will be tested is the best employee selection system application, and consists of a menu that will be inputted about employee data, as well as employee performance while in the company. The best employee selection system application is tested by Black Box testing. The purpose of testing this application is to see if the program performs every function we expect without looking at the source code of the software. The type of black box testing method used in this research is Equivalence Partitions. The Equivalence Partitions method is a test based on filling out each form in the best employee selection system, on each input menu will be tested and made into groups based on whether its function is valid or invalid.

Teknik Equivalence Partitions untuk Pengujian Aplikasi Manajemen Kas dan Inventaris Berbasis Web

Abstract: Performing checks on software specifications requires testing and validation. The discrepancy in the data entry process was caused by the validation process not being optimized. One of the consequences is the process of failure when inputting data. As a result, the application can be dangerous for future system users. Therefore, it is necessary to improve the quality and accurate verification. When the testing process finds errors that were previously unknown, then the test is called a good test. One of the testing method is the black box testing method with the Equivalence Partition technique. Equivalence Partition is a technique used to reduce the number of test cases that exist during testing, by entering data on each form that is in the application in each menu, then testing and grouping based on the validity by the system functionality.

Automatic Classification of Equivalent Mutants in Mutation Testing of Android Applications

Abstract: Software and symmetric testing methodologies are primarily used in detecting software defects, but these testing methodologies need to be optimized to mitigate the wasting of resources. As mobile applications are becoming more prevalent in recent times, the need to have mobile applications that satisfy software quality through testing cannot be overemphasized. Testing suites and software quality assurance techniques have also become prevalent, which underscores the need to evaluate the efficacy of these tools in the testing of the applications. Mutation testing is one such technique, which is the process of injecting small changes into the software under test (SUT), thereby creating mutants. These mutants are then tested using mutation testing techniques alongside the SUT to determine the effectiveness of test suites through mutation scoring. Although mutation testing is effective, the cost of implementing it, due to the problem of equivalent mutants, is very high. Many research works gave varying solutions to this problem, but none used a standardized dataset. In this research work, we employed a standard mutant dataset tool called MutantBench to generate our data. Subsequently, an Abstract Syntax Tree (AST) was used in conjunction with a tree-based convolutional neural network (TBCNN) as our deep learning model to automate the classification of the equivalent mutants to reduce the cost of mutation testing in software testing of android applications. The result shows that the proposed model produces a good accuracy rate of 94%, as well as other performance metrics such as recall (96%), precision (89%), F1-score (92%), and Matthew’s correlation coefficients (88%) with fewer False Negatives and False Positives during testing, which is significant as it implies that there is a decrease in the risk of misclassification.

Black-box Test Case Selection by Relating Code Changes with Previously Fixed Defects

Abstract: Software continuously changes to address new requirements and to fix defects. Regression testing is performed to ensure that the applied changes do not adversely affect existing functionality. The increasing number of test cases makes it infeasible to execute the whole regression test suite. Test case selection is adopted to select a subset of the test suite, which is associated with the changed parts of the software. These parts are assumed to be error-prone. We present and evaluate a test case selection approach in the context of black-box regression testing of embedded systems. In this context, it is challenging to relate test cases with a set of distinct source code elements to be able to select those test cases associated with the modified parts of the source code. We analyze previously fixed defects for this purpose. We relate test cases that detect these defects with the source files that are previously modified for fixing them. Then, we select test cases related with source code files that are modified in the subsequent revision. The strength of this relation is determined as the number of changes associated with fixed defects previously detected by the same test cases. We conduct a case study on 3 real projects from the consumer electronics domain. Results show that it is possible to detect from 65% up to 85% of the defects detected by the whole test suite by selecting from 30% up to 70% of the test cases.

JavaScript Instrumentation for Search-Based Software Testing: A Study with RESTful APIs

Abstract: JavaScript is one of the most popular programming languages. However, its dynamic nature poses several challenges to automated testing techniques. In this paper, we propose an approach and open-source tool support to enable white-box testing of JavaScript applications using Search-Based Software Testing techniques. We provide an automated approach to collect search-based heuristics like the common Branch Distance. To empirically evaluate our results, we integrated our technique into the EvoMASTER test generation tool, and carried out analyses on the automated system testing of RESTful APIs. Experiments on 5 NodeJS APIs show that our technique leads to significantly better results than existing black-box and grey-box testing tools in terms of code coverage and fault detection.

Investigating The Effectiveness of Model-Based Testing on Testing Skill Acquisition

Abstract: Software does not only need to be developed but also needs to get tested. Testing of software reduces the development and maintenance costs and increases software quality. Unfortunately, few software development courses focus on good testing practices. Some prior work has nevertheless researched possible ways of teaching software testing techniques to students. Unfortunately, all these approaches are code-oriented approaches, implying that a strong technical background is needed to effectively use them. They are also mostly focused on improving students’ knowledge on basic testing techniques. In contrast, TesCaV, a software tool used for teaching testing skills to university students, focuses on teaching testing to novice users with limited technical skills by providing a model-based testing (MBT) approach. MBT is a black-box testing technique in which the tests are automatically generated from a software model. This automatic generation allows for easy maintenance of the test suite when the software changes. These tests can be automatically generated by using a.o. Finite State Machines (FSMs), Markov Chains and Logic Programming. TesCaV is mainly based on Finite State Machines. The effect of using TesCaV on testing coverage is quantitatively analysed in this research. The preliminary results of TesCaV show that it is a promising tool for the teaching of MBT.

Reducing Redundant Test Executions in Software Product Line Testing—A Case Study

Abstract: In the context of software product line (SPL) engineering, test cases can be reused for testing a family of products that share common parts of source code. An approach to test the products of a product family is to exhaustively execute each test case on all the products. However, such an approach would be very inefficient because the common parts of source code will be tested multiple times unnecessarily. To reduce unnecessary repetition of testing, we previously proposed a method to avoid equivalent test executions of a product line in the context of regression testing. However, it turns out that the same approach can be used in a broader context than just regression testing of product families. In this paper, we argue the generality of the method in the sense that it can be used for testing of the first version of a product family as well as regression testing of its subsequent versions. In addition, in this paper, in order to make the method practically usable for users, we propose a process for applying it to SPL testing. We demonstrate the generality of our method and the practical applicability of the proposed process for the method by conducting a case study.

Bridging Fuzz Testing and Metamorphic Testing for Classification of Machine Learning

Abstract: Artificial Intelligence (AI) built-in Consumer Electronics is popular, but it is hard to test and evaluate AI-based system with the existing performance metrics. Even though AI-based systems are implemented in software with flexibility, bias and non-determinism property etc., they can suffer the same defects as other software. That is why new software testing approaches are needed when testing AI-based systems. Therefore, this paper proposes a bridging approach between fuzz testing and metamorphic testing focus on the classification of machine learning. This approach can be used as a test oracle for classification of training data.

On Technical Debt in Software Testing - Observations from Industry

Abstract: Testing large complex systems in an agile way of working was a tough transition for systems having large active legacy and honouring backward compatibility. Transition from manual test to full test execution automation resulted in increased speed and manifested technical debt. The agile way of working in continuous build and test, creates a lot of repetition by execution of the same tests. Overlap between agile teams producing similar test cases, causes a constant growth of the test suites. Despite the obvious improvement of automating millions of test cases, the numbers provide a false sense of security for management on how well the system is tested. The causes of technical debt should be addressed, instead of managing the symptoms. Technical debt in software testing could be addressed by refactoring, supported by known techniques like cloning, similarity analysis, test suite reduction, optimization and reducing known test smells. Increasing the system quality can also be improved by utilizing metrics, e.g. code coverage and mutation score or use one of the many automated test design technologies. Why this is not addressed in the industry has many causes. In this paper we describe observations from several industries, with the focus on large complex systems. The contribution lies in reflecting on observations made in the last decade, and providing a vision which identifies improvements in the area of test automation and technical debt in software test, i.e. test code, test suites, test organisation, strategy and execution. Our conclusion is that many test technologies are now mature enough to be brought into regular use. The main hindrance is skills and incentive to do so for the developer, as well as a lack of well educated testers.

Api test automation of search functionality with artificial intelligence

Abstract: One of the steps in software development is to test the software product. With the development of technology, the testing process has improved to automated testing, which reduces the impact of the human factor on error and speeds up testing. The main software products for testing are considered to be web applications, web services, mobile applications and performance testing. According to the testing pyramid, when testing web services, you need to develop more test cases than when testing a web application. Because automation involves writing software code for testing, the use of ready-made tools will speed up the software development process. One of the most important test indicators is the coverage of search functionality. The search functionality of a web application or web service requires a large number of cases, as you need to provide many conditions for its operation through the free entry of any information on the web page. There is an approach to data-based testing, which involves working with a test data set through files such as CSV, XLS, JSON, XML and others. However, finding input for testing takes a lot of time when creating test cases and automated test scenarios. It is proposed to use artificial data set generators based on real values and popular queries on the website to form a test data set. In addition, it is possible to take into account the probable techniques of developing test cases. It is proposed to conditionally divide the software for testing into several layers: client, test, work with data, checks and reports. The Java programming language has a number of libraries for working at each of these levels. It is proposed to use Rest Assured as a Restful client, TestNG as a library for writing tests with checks, and Allure report for generating reports. It is noted that the proposed approach uses artificial intelligence for automated selection of test cases when creating a test to diversify test approaches and simulate human input and behavior to maximize the use of cases.

A Review on Software Testing Using Machine Learning Techniques

Abstract: Software testing is the method of executing the program with the intension of finding the errors. This process is very complex and time intensive and costly. Automation of software testing process has turn out to be very workable solution in software engineering. For this purpose, machine learning algorithm is being used. We chose 48 primary research studies, tried to explain each machine learning-based approach in software testing. It gives an overview of how the input that is available data given to the model for learning and then how it is helping in giving predictions. We discovered that machine learning methods were mostly employed for test case development, refining, and evaluation. Machine learning has also been used to evaluate the production of test oracles and predict the cost of testing procedures.

Cloud-Based Simulated Automated Testing Platform for Virtual Coupling System

Abstract: Research on virtual coupling has made further progress. It needs many tests before operation. The problem of connecting the new railway with the existing railway will arise. The increasingly large and complex hubs make a test of virtual coupling difficult. Test of the virtual coupling system, especially combined test, has a heavy workload. Restricted by time and the influence of the existing transportation, it is challenging to discover defects and hidden dangers of virtual coupling systems through dynamic on-site test procedures. The laboratory “full scene, full elements” engineering testing urgently needs to be executed before the opening of new lines. Limited to supply of real equipment, we proposed a semi-physical and semi-simulation testing scheme. The virtualization transplantation of real equipment is one of the main challenges to realizing simulated testing. A cloud-based platform is designed to carry the simulated testing environment. Thus, most of the work that depends on dynamic on-site testing can be replaced by simulated testing. It can reduce the dynamic testing cost of the signal system and its impact on transportation. The advantages of simulated testing technologies can also effectively improve the quality, depth, and breadth of virtual coupling system testing.

TORACLE 2021 Workshop Summary

Abstract: Testing is an important activity in software engineering, especially with the growing adoption of software systems in safety-critical domains. Testing research, however, is mostly focused on deter- mining which test inputs to use (e.g., proposing and evaluating test coverage criteria or automatic test generation tools). Re- gardless of the used coverage criterion, we need to know whether a given program executes correctly on a given input. Indeed, a test execution for which we cannot discriminate between success or failure serves little purpose. This corresponds to the so-called øracle problem", the problem of knowing whether a program be- haves correctly for a speci c input. Although the importance of the oracle problem is well understood, only a few alternatives exist to manually deriving test oracles. This makes automated test or- acle generation one of the main bottlenecks for achieving full test automation. Therefore, novel approaches and tools are needed to address this important problem.

An efficient automatic testing framework for NFV systems

Abstract: NFV systems are becoming more complicated with various of software modules, vendors and interfaces introduced. Testing such systems is very time consuming and requires lots of expertise. In this paper, we propose an efficient automatic testing framework to simplify and speedup the testing process. There are two critical components in this framework, i.e., a structured and reusable test case format and an abstraction layer to adapt different testing tools. Based on this framework, a tester can quickly compose the test cases by just setting the corresponding parameters of the test tools and target components. After this setting, the framework will automatically arrange the testing order and invoke the testing tools to fulfil the test. The experiment on real systems shows that our framework can greatly improve the testing efficiency.