Dynamic verification is widely used to increase confidence in the correctness of RTL circuits during the pre-silicon design phase. Despite numerous attempts over the last decades to automate the stimuli generation based on coverage feedback, Coverage Directed Test Generation (CDG) has not found the widespread adoption that one would expect. Based on new ideas from the software testing community around coverage-guided mutational fuzz testing, we propose a new approach to the CDG problem which requires minimal setup and takes advantage of FPGA-accelerated simulation for rapid testing. We provide test input and coverage definitions that allow fuzz testing to be applied to RTL circuit verification. In addition we propose and implement a series of transformation passes that make it feasible to reset arbitrary RTL designs quickly, a requirement for deterministic test execution. Alongside this paper we provide rfuzz, a fully featured implementation of our testing methodology which we make available as open-source software to the research community. An empirical evaluation of rfuzz shows promising results on archiving coverage for a wide range of different RTL designs ranging from communication IPs to an industry scale 64-bit CPU.

https://dl.acm.org/doi/pdf/10.1145/3240765.3240842

RFUZZ: coverage-directed fuzz testing of RTL on FPGAs

This chapter gives an overview of the field of model-based testing (MBT), particularly the recent advances in the last decade. It gives a summary of the MBT process, the modeling languages that are currently used by the various communities who practice MBT, the technologies used to generate tests from models, and discusses best practices, such as traceability between models and tests. It also briefly describes several findings from a recent survey of MBT users in industry, outlines the increasingly popular use of MBT for security testing, and discusses future challenges for MBT.

Recent Advances in Model-Based Testing

In this paper, we present BEACON, a Branch-oriented Evolutionary Ant Colony OptimizatioN method which is a bio-inspired meta-heuristic for design validation and functional test generation. BEACON combines an evolutionary search technique with Ant Colony Optimization (ACO) for improved search capability. BEACON first cross-compiles the Verilog circuit source to a C++ base for fast simulation. Then, it profiles the code, keeping track of each branch and the number of times it has been visited in a database. Branch coverage provides a very useful metric for exploring the design, especially visiting the most critical states, including corner states, in the design. At 100% branch coverage, we can conclude that every control state described in the RTL has been visited. Thus, during execution, BEACON trims highly visited branches from the search and focuses the search on rarely occurring branches and paths. This approach gives a significant performance boost while maintaining a high level of coverage. Experimental results show that BEACON is able to achieve very high branch coverages with a fraction of computational cost. In addition, previous hard-to-reach corner states in the ITC99 benchmarks have now been reached by BEACON. New states can also be discovered from the RTL descriptions. For many circuits, one to two orders of magnitude speedups over existing methods have been achieved.

Design validation of RTL circuits using evolutionary swarm intelligence

The measurement of the Quality of Experience (QoE) in 3DTV recently became an important research topic as it relates to the development of the 3D industry. Pair comparison is a reliable method as it is easier for the observers to provide their preference on a pair rather than give an absolute scale value to a stimulus. The QoE measured by pair comparison is thus called “Preference of Experience (PoE)”. In this paper, we introduce some efficient designs for pair comparison which can reduce the number of comparisons. The constraints of the presentation order of the stimuli in pair comparison test are listed. Finally, some analysis methods for pair comparison data are provided accompanied with some examples from the studies of the measurement of PoE.

/pdf/subjective-assessment-methodology-for-preference-of-4qxtge15p9.pdf

Subjective assessment methodology for preference of experience in 3DTV

Test generation for hard-to-reach states has been one of the hardest tasks in functional verification. In this paper, we present PACOST, a PAth Constraint Solving based Test generation method which operates in an abstraction-guided simulation framework to cover hard-to-reach states. PACOST combines concrete simulation and symbolic simulation in a path constraint solver to generate a set of valid input vectors for exploring different simulation paths, followed by next state selection considering abstract distances. In addition, two backtracking strategies are proposed to alleviate the dead end problem and ensure fast converge to the target state. Experimental results show that PACOST is effective in covering hard-to-reach states.

Path Constraint Solving Based Test Generation for Hard-to-Reach States

Although stochastic search techniques have shown promise in test generation and design validation, they often fail when there is a specific, random-resistant sequence of vectors required to exercise a target In order to combat this, deterministic techniques are added, resulting in a hybrid solution to maintain high speed of execution while improving metric performance This paper presents a formal hybridization that combines a Register Transfer Level (RTL) stochastic swarm intelligence based test vector generation with the Verilator Verilog-to-C++ source-to-source compiler Verilator generates a fast cycle accurate C++ simulation unit for Verilog descriptions and provides instrumentation for branch and toggle coverage metrics This RTL model can also be used to generate a bounded model checking (BMC) instance During the stochastic search, the bounded model checker is launched to expand the unexplored search frontier and aid in the navigation of narrow paths Additionally, an inductive reach ability test is applied in order to eliminate unreachable branches from our search space These additions have significantly improved branch coverage, reaching 100% in several ITC99 benchmarks Additionally, compared to previous functional test generation methods, we achieve substantial speedup achieved with purely stochastic methods

https://www.computer.org/csdl/pds/api/csdl/proceedings/download-article/12OmNC3FGjP/pdf

Functional Test Generation at the RTL Using Swarm Intelligence and Bounded Model Checking

Automatic test pattern generation for non-scan sequential circuits is an extremely challenging task. If successful, it can offer many benefits to the EDA community, ranging from manufacturing and functional test to post-silicon validation. High-level test generators often miss the low-level details, thus missing the detection of some gate-level faults. On the other hand, gate-level test generators miss the high-level path traversal knowledge to more effectively traverse the state space. In this work, we present a fine-grain mixed-level test generator that utilizes co-simulation of register-transfer and gate levels to generate high quality vectors. The algorithm, based on an ant colony optimization, targets branch coverage at the RTL and simultaneously attempts to associate rare fault excitations with a sequence of branch activations. By weighting these sequences within the fitness function across the two levels, the algorithm is able to achieve high fault coverage in the presence of deep hard-to-reach states without scan. The result is that the test sequences obtained offer both high branch coverage as well high stuck-at coverage with low computational costs. In particular, for hard-to-test circuits such as the ITC'99 circuit b12, >98% branch coverage and >90% stuck-at coverage are achieved, vastly improving over other state of the art non-scan tools.

Dual-Purpose Mixed-Level Test Generation Using Swarm Intelligence

Register-transfer level (RTL) verification is a challenging problem for today's complex circuits. A sub-problem of verification is reachability of basic blocks or branches in the code. This paper proposes a novel analysis based on the domain of signal values in the RTL code to reason about the reachability of all branches without explicit circuit unrolling. This analysis takes into account all assignments, activating and preceding conditions in the code, to derive an assignment table that lists all the possible sequences of branches required to reach a target branch. In the process, it proves unreachable branches as well as provides guidance for reachable branches. This analysis resolves branches more efficiently compared to other methods for the ITC99 [1] benchmarks, especially for larger benchmarks containing previously unresolved branches.

Signal domain based reachability analysis in RTL circuits

In this paper, we propose a novel parallel state justification tool, GACO, utilizing Ant Colony Optimization (ACO) on Graphical Processing Units (GPU). With the high degree of parallelism supported by the GPU, GACO is capable of launching a large number of artificial ants to search for the target state. A novel parallel simulation technique, utilizing partitioned navigation tracks as guides during the search, is proposed to achieve extremely high computation efficiency for state justification. We present the results on a GPU platform from NVIDIA (a GeForce GTX 285 graphics card) that demonstrate a speedup of up to 228× compared to deterministic methods and a speedup of up to 40× over previous state-of-the-art heuristic based serial tools.

Kelson Gent

Papers

Design validation of RTL circuits using evolutionary swarm intelligence

Functional Test Generation at the RTL Using Swarm Intelligence and Bounded Model Checking

Dual-Purpose Mixed-Level Test Generation Using Swarm Intelligence

Signal domain based reachability analysis in RTL circuits

Utilizing GPGPUs for design validation with a modified Ant Colony Optimization