Self-driving cars rely on software which needs to be thoroughly tested. Testing self-driving car software in real traffic is not only expensive but also dangerous, and has already caused fatalities. Virtual tests, in which self-driving car software is tested in computer simulations, offer a more efficient and safer alternative compared to naturalistic field operational tests. However, creating suitable test scenarios is laborious and difficult. In this paper we combine procedural content generation, a technique commonly employed in modern video games, and search-based testing, a testing technique proven to be effective in many domains, in order to automatically create challenging virtual scenarios for testing self-driving car soft- ware. Our AsFault prototype implements this approach to generate virtual roads for testing lane keeping, one of the defining features of autonomous driving. Evaluation on two different self-driving car software systems demonstrates that AsFault can generate effective virtual road networks that succeed in revealing software failures, which manifest as cars departing their lane. Compared to random testing AsFault was not only more efficient, but also caused up to twice as many lane departures.

Automatically testing self-driving cars with search-based procedural content generation

Context Embedded systems have overwhelming penetration around the world. Innovations are increasingly triggered by software embedded in automotive, transportation, medical-equipment, communication, energy, and many other types of systems. To test embedded software in an effective and efficient manner, a large number of test techniques, approaches, tools and frameworks have been proposed by both practitioners and researchers in the last several decades. Objective However, reviewing and getting an overview of the entire state-of-the-art and the –practice in this area is challenging for a practitioner or a (new) researcher. Also unfortunately, as a result, we often see that many companies reinvent the wheel (by designing a test approach new to them, but existing in the domain) due to not having an adequate overview of what already exists in this area. Method To address the above need, we conducted and report in this paper a systematic literature review (SLR) in the form of a systematic literature mapping (SLM) in this area. After compiling an initial pool of 588 papers, a systematic voting about inclusion/exclusion of the papers was conducted among the authors, and our final pool included 312 technical papers. Results Among the various aspects that we aim at covering, our review covers the types of testing topics studied, types of testing activity, types of test artifacts generated (e.g., test inputs or test code), and the types of industries in which studies have focused on, e.g., automotive and home appliances. Furthermore, we assess the benefits of this review by asking several active test engineers in the Turkish embedded software industry to review its findings and provide feedbacks as to how this review has benefitted them. Conclusion The results of this review paper have already benefitted several of our industry partners in choosing the right test techniques / approaches for their embedded software testing challenges. We believe that it will also be useful for the large world-wide community of software engineers and testers in the embedded software industry, by serving as an “index” to the vast body of knowledge in this important area. Our results will also benefit researchers in observing the latest trends in this area and for identifying the topics which need further investigations.

Testing embedded software: A survey of the literature

Automotive Original Equipment Manufacturers (OEMs) require varying levels of functionalities and model details at different phases of the electric vehicles (EV) development process, with a trade-off between accuracy and execution time. This article proposes a scalable modelling approach depending on the multi-objective targets between model functionalities, accuracy and execution time. In this article, four different fidelity levels of modelling approaches are described based on the model functionalities, accuracy and execution time. The highest error observed between the low fidelity (LoFi) map-based model and the high fidelity (HiFi) physics-based model is 5.04%; while, the simulation time of the LoFi model is ~10
 4
 times faster than corresponding one of the HiFi model. A detailed comparison of all characteristics between multi-fidelity models is demonstrated in this paper. Furthermore, a dSPACE SCALEXIO Hardware-in-the-Loop (HiL) testbench, equipped with a minimal latency of 18 μ sec, is used for real-time (RT) model implementation of the EV's HV DC/DC converter. The performance of the entire HiL setup is compared with the Model-in-the-Loop (MiL) setup and the highest RMSE is limited to 0.54 among the HiL and MiL results. Moreover, the accuracy (95.7%) of the passive component loss estimation is verified through the Finite Element Method (FEM) software model. Finally, the experimental results of a full-scale 30-kW SiC DC/DC converter prototype are presented to validate the accuracy and correlation between multi-fidelity models. It has been observed that the efficiency deviation between the hardware prototype and multi-fidelity models is less than 1.25% at full load. Furthermore, the SiC Interleaved Bidirectional Converter (IBC) prototype achieves a high efficiency of 98.4% at rated load condition.

/pdf/scalable-modeling-approach-and-robust-hardware-in-the-loop-1zlaref41w.pdf

Scalable Modeling Approach and Robust Hardware-in-the-Loop Testing of an Optimized Interleaved Bidirectional HV DC/DC Converter for Electric Vehicle Drivetrains

The main goal of the Functional Mock-up Interface (FMI) standard is to allow the sharing of simulation models across tools. To accomplish this, FMI relies on a combination of XML-files and compiled C-code packaged in a zip archive. This archive is called a Functional Mock-up Unit (FMU). In theory, an FMU can support multiple platforms, but not necessarily in practice. Furthermore, software libraries for interacting with FMUs may not be available in a particular language or platform. Another issue is related to the protection of intellectual property (IP). While an FMU is free to only provide the C-code in its binary form, other resources within the FMU may be unprotected. Distributing models in binary form also opens up the possibility that they may contain malicious code. In order to meet these challenges, this paper presents an open-source co-simulation framework based on FMI, which is language and platform independent thanks to the use of well-established remote procedure call (RPC) technologies. One or more FMUs are wrapped inside a server program supporting one or more language independent RPC systems over various network protocols. Together, they allow cross-platform invocation of FMUs from multiple, including previously unsupported, languages. The client-server architecture allows the effective protection of IP while also providing a means of protecting users from malicious code.

/pdf/a-language-and-platform-independent-co-simulation-framework-4p9frhc0bo.pdf

A Language and Platform Independent Co-Simulation Framework Based on the Functional Mock-Up Interface

As future power systems become increasingly complex and interconnected to other energy carriers, a single research infrastructure can rarely provide the required test-beds to study a complete energy system, especially if different types of real power hardware are expected to be in-the-loop. Therefore, virtual interconnection of laboratories for large-scale systems plays an important role for geographically distributed realtime simulation. This paper presents the improvements made in simulation fidelity as well as usability for establishing future simulator and laboratory connections. A general procedure is proposed and analyzed for geographically distributed real-time simulation, which allows users easily to adapt this procedure to specific test cases. A systematic and comprehensive analysis of a dynamic phasor based co-simulation interface algorithm and its improvements are provided to demonstrate the advantages as well as limitations of this approach.

/pdf/improvements-to-the-co-simulation-interface-for-1owuv1wzxl.pdf

Improvements to the Co-simulation Interface for Geographically Distributed Real-time Simulation

The development of automotive electronics and software systems is often associated with high costs due to their multi-domain nature (including control engineering, electronics, hydraulics, mechanics, etc). The involvement of these different disciplines makes it difficult for control engineers to test their controllers with them being integrated in the whole system, in early development phases. By introducing the "XiL approach", ETAS wants to leverage virtualization techniques in order to bring embedded systems to the desk of every developer. The XiL approach implies fast development cycles due to easy integration of software, vehicle and plant components on the PC. XiL strives for seamless transition between X-in-the-loop setups, with X representing any control model (M), software (S), or hardware (H) under test. This paper will outline this methodology with an engine management system example.

Testing automotive embedded systems under X-in-the-loop setups

Virtual system development gets more and more important in many industrial domains. It is considered to reduce development times, lower computing costs, and shorten time-to-market. Co-simulation is a particularly promising approach for modular and interoperable development. In practice the integration and coupling of heterogeneous systems still require enormous efforts. The configuration and operation of distributed hardware-in-the-loop systems and simulations contribute to efficiency of testing. Currently no standardized interface or protocol specification is available, which allows the interaction of real-time and non-real-time systems of different vendors. This paper for the first time presents the Distributed Co-simulation Protocol (DCP) which is subject to proposal as a standard for real-time and non-real-time system integration and simulation. The DCP consists of a data model, a finite state machine, and a communication protocol including a set of protocol data units. It is designed as a tool independent standard. It was developed in context of the ACOSAR project and is subject to standardization as a Modelica Association Project (MAP). It enables the definition, configuration and execution of a wide range of different simulations and test scenarios. It supports a master-slave architecture for simulation setup and control. The specification defines the design of a slave only, the design of a master is not in scope of the specification. To highlight the industrial applicability of the DCP, three examples from the automotive domain are shown.

Natarajan Nagarajan

Papers

Testing automotive embedded systems under X-in-the-loop setups

The Distributed Co-simulation Protocol for the integration of real-time systems and simulation environments

A Virtual Environment for an early development and validation of next generation Digital Vehicle Entry Functions