Mutation Testing is a fault-based software testing technique that has been widely studied for over three decades The literature on Mutation Testing has contributed a set of approaches, tools, developments, and empirical results This paper provides a comprehensive analysis and survey of Mutation Testing The paper also presents the results of several development trend analyses These analyses provide evidence that Mutation Testing techniques and tools are reaching a state of maturity and applicability, while the topic of Mutation Testing itself is the subject of increasing interest

An Analysis and Survey of the Development of Mutation Testing

The Object Management Group's (OMG) Model Driven Architecture (MDA) strategy envisages a world where models play a more direct role in software production, being amenable to manipulation and transformation by machine. Model Driven Engineering (MDE) is wider in scope than MDA. MDE combines process and analysis with architecture. This article sets out a framework for model driven engineering, which can be used as a point of reference for activity in this area. It proposes an organisation of the modelling 'space' and how to locate models in that space. It discusses different kinds of mappings between models. It explains why process and architecture are tightly connected. It discusses the importance and nature of tools. It identifies the need for defining families of languages and transformations, and for developing techniques for generating/configuring tools from such definitions. It concludes with a call to align metamodelling with formal language engineering techniques.

Model Driven Engineering

International Technology Roadmap for Semiconductors 2003の要求清浄度について － シリコンウエハ表面と雰囲気環境に要求される清浄度, 分析方法の現状について －

Summary) The abstract should follow the structure of the article (relevance, degree of exploration of the problem, the goal, the main results, conclusion) and characterize the theoretical and practical significance of the study results. The abstract should not contain wording echoing the title, cumbersome grammatical structures and abbreviations. The text should be written in scientific style. The volume of abstracts (summaries) depends on the content of the article, but should not be less than 250 words. All abbreviations must be disclosed in the summary (in spite of the fact that they will be disclosed in the main text of the article), references to the numbers of publications from reference list should not be made. The sentences of the abstract should constitute an integral text, which can be made by use of the words “consequently”, “for example”, “as a result”. Avoid the use of unnecessary introductory phrases (eg, “the author of the article considers...”, “The article presents...” and so on.)

/pdf/ministry-of-education-and-science-of-the-russian-federation-7m2jn0xnm1.pdf

Ministry of Education and Science of the Russian Federation

IEEE transactions on computer-aided design of integrated circuits and systems : a publication of the IEEE Circuits and Systems Society

Mutation analysis is a widely-adopted strategy in software testing with two main purposes: measuring the quality of test suites, and identifying redundant code in programs. Similar approaches are applied in hardware verification and testing too, especially at RTL or gate level, where mutants are generally referred as faults, and mutation analysis is performed by means of fault modeling and fault simulation. However, in modern embedded systems there is a close integration between HW and SW parts, and verification strategies should be applied early in the design flow. This requires the definition of new mutation analysis-based strategies that work at system level, where HW and SW functionalities are not partitioned yet. In this context, the paper proposes a mutation model for perturbing transaction level modeling (TLM) SystemC descriptions. In particular, the main constructs provided by the SystemC TLM 2.0 library have been analyzed, and a set of mutants is proposed to perturb the primitives related to the TLM communication interfaces.

/pdf/a-mutation-model-for-the-systemc-tlm-2-0-communication-16nq9zlyf1.pdf

A mutation model for the SystemC TLM 2.0 communication interfaces

More and more people are starting to use the SystemC description language to model and simulate new designs. This is due mainly to the simplicity and power of the language. The number of models written in SystemC currently available is still very limited and testing SystemC descriptions is still an open issue, since the language is new and researchers are looking for efficient error models and coverage metrics. This paper presents AMLETO, a multi-language environment developed to efficiently test embedded systems and IP-Cores. Using IIR, an HDL language independent representation, it supplies: fast translation from VHDL to SystemC of design descriptions and viceversa, generation and setup of customized TPGs for the design under test and generation of erroneous models capable of simulating the presence of design errors.

AMLETO: a multi-language environment for functional test generation

Transaction level modeling (TLM) is becoming a usual practice for simplifying system-level design and architecture exploration. It allows the designers to focus on the functionality of the design, while abstracting away implementation details that will be added at lower abstraction levels. However, moving from transaction level to RTL requires redefining TLM test benches and assertions. Such a wasteful and error prone conversion can be avoided by adopting transactor-based verification (TBV). Many recent works adopt this strategy to propose verification methodologies that allow: (1) mixing TLM and RTL components; and (2) reusing TLM assertions and test benches at RTL. Even if practical advantages of such an approach are evident, there are no papers in the literature that evaluate the effectiveness of the TBV compared to a more traditional RTL verification strategy. This paper is intended to fill in the gap. It theoretically compares the quality of the TBV towards the rewriting of assertions and test benches at RTL with respect to both fault coverage and assertion coverage

/pdf/on-the-evaluation-of-transactor-based-verification-for-16ah542oto.pdf

On the Evaluation of Transactor-based Verification for Reusing TLM Assertions and Testbenches at RTL

Transaction-level modeling (TLM) is the most promising technique to deal with the increasing complexity of modern embedded systems. However, modeling a complex system completely at transaction level could be inconvenient when IP cores are available on the market, since they are usually modeled at register transfer level (RTL). In this context, modeling and verification methodologies based on transactors allow designers to reuse RTL IPs into TLM-RTL mixed designs, thus guaranteeing a considerable saving of time. Practical advantages of such an approach are evident, but mixed TLM-RTL designs cannot completely provide the well-known effectiveness in terms of simulation speed provided by TLM. This paper presents a methodology to automatically abstract RTL IPs into equivalent TLM descriptions. To do that, the paper first proposes a formal definition of equivalence based on events, showing how such a definition can be applied to prove the correctness of a code manipulation methodology, such as code abstraction. Then, the paper proposes a technique to automatically abstract RTL IPs into TLM descriptions. Finally, the paper shows that the TLM descriptions obtained by applying the proposed technique are correct by construction, relying on the given definition of event-based equivalence. A set of experimental results is reported to confirm the effectiveness of the methodology.

Automatic Abstraction of RTL IPs into Equivalent TLM Descriptions

Functional testing of HDL specifications is one of the most promising approaches for the verification of the functionalities of a design before synthesis. The contribution of this work is the development of a test generation algorithm targeting a new coverage metric (called bit-coverage) that provides full statement coverage, branch coverage, condition coverage and partial path coverage for behaviorally sequential models. The behavioral test sequences can be also the only way to evaluate testability of VHDL model for which a gate-level representation is not available (e.g third-party cores), since the behavioral error model is characterized also by a high correlation with the RT and gate-level stuck-at fault model. Moreover the preciseness of the proposed coverage metric makes the identified test sequences more effective in identifying design errors, than other test patterns developed by following standard coverage metrics.

Franco Fummi

Papers

A mutation model for the SystemC TLM 2.0 communication interfaces

AMLETO: a multi-language environment for functional test generation

On the Evaluation of Transactor-based Verification for Reusing TLM Assertions and Testbenches at RTL

Automatic Abstraction of RTL IPs into Equivalent TLM Descriptions

Functional test generation for behaviorally sequential models