In recent years, the reliability of Deep Neural Networks (DNN) has become the focus of an increasing number of research activities. In particular, researchers have focused on understanding how a DNN behaves when the underlying hardware is affected by a fault. This is a challenging task: slight changes in a network architecture can significantly impact how the network reacts to faults. There are several approaches to simulate the behaviour of a faulty network: the most accurate one is to perform low-level fault simulations. Nonetheless, this task is very time-consuming and costly to be implemented. Even though the injection time can be reduced by injecting faults at the application level, for sufficiently large networks, this time is still very high, requiring weeks to complete a single simulation. This work aims at providing a fast and accurate solution for injecting software-level faults in a DNN that is independent of its architecture and does not require any modification to its structure. For this reason, this paper introduces SCI-FI, a Smart, aCcurate and unIntrusive Fault-Injector. SCI-FI smartly reduces the fault injection time required for a complete fault simulation of the network by taking advantage of two fundamental mechanisms: Fault Dropping and Delayed Start. Experimental results from various ResNet, DenseNet and EfficientNet architectures targeting the CIFAR-10 and ImageNet datasets show that combining these techniques drastically reduces the simulation time, which can last up to 70% less.

SCI-FI: a Smart, aCcurate and unIntrusive Fault-Injector for Deep Neural Networks

In recent decades, deep learning (DL)-based solutions have gained a great deal of interest in industry and academia due to their outstanding computational capabilities. The usage of electronic devices running applications based on Artificial Neural Networks (ANNs) is spreading in several areas, including safety-critical applications such as self-driving cars, robots, and space applications. ANNs are often regarded as inherently robust and fault-tolerant, being brain-inspired and redundant computing models. However, to use them safely in human contexts, there is a compelling need to assess their reliability. Indeed, when they are deployed on resource-constrained hardware devices, single physical faults might jeopardize the activity of multiple neurons, leading to undesirable results. Since reliability assessment is becoming a growing concern, many efforts have been made in recent decades to propose efficient approaches to assess ANN-based systems reliability. The intent of this article is to overview the main reliability assessment methodologies for ANN-based systems, focusing mainly on Fault Injection techniques used to evaluate the ANN resilience at different abstraction levels.

Reliability Assessment Methodologies for ANN-based Systems

—In safety-critical applications, microcontrollers have to be tested to satisfy strict quality and performances constraints. It has been demonstrated that on-chip ring oscillators can be used as speed monitors to reliably predict the performances. However, any machine-learning model is likely to be inaccurate if trained on an inadequate dataset, and labeling data for training is quite a costly process. In this paper, we present a methodology based on active learning to select the best samples to be included in the training set, signiﬁcantly reducing the time and cost required. Moreover, since different speed measurements are available, we designed a multi-label technique to take advantage of their correlations. Experimental results demonstrate that the approach halves the training-set size, with respect to a random- labelling, while it increases the predictive accuracy, with respect to standard single-label machine-learning models.

A Multi-Label Active Learning Framework for Microcontroller Performance Screening

Nowadays, Deep Neural Networks (DNNs) are one of the most computationally-intensive algorithms because of the (i) huge amount of data to be transferred from/to the memory, and (ii) the huge amount of matrix multiplications to compute. These issues motivate the design of custom DNN hardware accelerators. These accelerators are widely used for low-latency safety-critical applications such as object detection in autonomous cars. Safety-critical applications have to be resilient with respect to hardware faults and Deep Learning (DL) accelerators are subjected to hardware faults that can cause functional failures, potentially leading to catastrophic consequences. Although DNNs possess a certain level of intrinsic resilience, it varies depending on the hardware on which they are run. The intent of the paper is to assess the resilience of a systolic-array-based DNN accelerator in the presence of hardware faults, in order to identify the architectural parameters that may mainly impact the DNN resilience.

Resilience-Performance Tradeoff Analysis of a Deep Neural Network Accelerator

Deep Learning, and in particular, Deep Neural Network (DNN) is nowadays widely used in many scenarios, including safety-critical applications such as autonomous driving. In this context, besides energy efficiency and performance, reliability plays a crucial role since a system failure can jeopardize human life. As with any other device, the reliability of hardware architectures running DNNs has to be evaluated, usually through costly fault injection campaigns. This paper explores approximation and fault resiliency of DNN accelerators. We propose to use approximate (AxC) arithmetic circuits to agilely emulate errors in hardware without performing fault injection on the DNN. To allow fast evaluation of AxC DNN, we developed an efficient GPU-based simulation framework. Further, we propose a fine-grain analysis of fault resiliency by examining fault propagation and masking in networks.

Special Session: Approximation and Fault Resiliency of DNN Accelerators

This paper encompasses three contributions by industry professionals and university researchers. The contributions describe different trends in automotive products, including both manufacturing test and run-time reliability strategies. The subjects considered in this session deal with critical factors, from optimizing the final test before shipment to market to in-field reliability during operative life.

Test, Reliability and Functional Safety Trends for Automotive System-on-Chip

Assessing the reliability of modern devices running CNN algorithms is a very difficult task. Actually, the complexity of the state-of-the-art devices makes exhaustive Fault Injection (FI) campaigns impractical and typically out of the computational capabilities. A possible solution consists of resorting to statistical FI campaigns that allow a reduction in the number of needed experiments by injecting only a carefully selected small part of it. Under specific hypothesis, statistical FIs guarantee an accurate picture of the problem, albeit selecting a reduced sample size. The main problems today are related to the choice of the sample size, the location of the faults, and the correct understanding of the statistical assumptions. The intent of this paper is twofold: first, we describe how to correctly specify statistical FIs for Convolutional Neural Networks; second, we propose a data analysis on the CNN parameters that drastically reduces the number of FIs needed to achieve statistically significant results without compromising the validity of the proposed method. The methodology is experimentally validated on two CNNs, ResNet-20 and MobileNetV2, and the results show that a statistical FI campaign on about 1.21% and 0.55% of the possible faults, provides very precise information of the CNN reliability. The statistical results have been confirmed by the exhaustive FI campaigns on the same cases of study.

Assessing Convolutional Neural Networks Reliability through Statistical Fault Injections

The widespread use of artificial intelligence (AI)-based systems has raised several concerns about their deployment in safety-critical systems. Industry standards, such as ISO26262 for automotive, require detecting hardware faults during the mission of the device. Similarly, new standards are being released concerning the functional safety of AI systems (e.g., ISO/IEC CD TR 5469). Hardware solutions have been proposed for the infield testing of the hardware executing AI applications; however, when used in applications such as Convolutional Neural Networks (CNNs) in image processing tasks, their usage may increase the hardware cost and affect the application performances. In this paper, for the very first time, a methodology to develop high-quality test images, to be interleaved with the normal inference process of the CNN application is proposed. An Image Test Library (ITL) is developed targeting the on-line test of GPU functional units. The proposed approach does not require changing the actual CNN (thus incurring in costly memory loading operations) since it is able to exploit the actual CNN structure. Experimental results show that a 6-image ITL is able to achieve about 95% of stuck-at test coverage on the floating-point multipliers in a GPU. The obtained ITL requires a very low test application time, as well as a very low memory space for storing the test images and the golden test responses.

J. Aribido

Papers

Test, Reliability and Functional Safety Trends for Automotive System-on-Chip

Assessing Convolutional Neural Networks Reliability through Statistical Fault Injections

Image Test Libraries for the on-line self-test of functional units in GPUs running CNNs