With the emerging adoption of deep neural networks (DNNs) in the HPC domain, the reliability of DNNs is also growing in importance. As prior studies demonstrate the vulnerability of DNNs to hardware transient faults (i.e., soft errors), there is a compelling need for an efficient technique to protect DNNs from soft errors. While the inherent resilience of DNNs can tolerate some transient faults (which would not affect the system's output), prior work has found there are critical faults that cause safety violations (e.g., misclassification). In this work, we exploit the inherent resilience of DNNs to protect the DNNs from critical faults. In particular, we propose Ranger, an automated technique to selectively restrict the ranges of values in particular DNN layers, which can dampen the large deviations typically caused by critical faults to smaller ones. Such reduced deviations can usually be tolerated by the inherent resilience of DNNs. Ranger can be integrated into existing DNNs without retraining, and with minimal effort. Our evaluation on 8 DNNs (including two used in self-driving car applications) demonstrates that Ranger can achieve significant resilience boosting without degrading the accuracy of the model, and incurring negligible overheads.

Ranger: Boosting Error Resilience of Deep Neural Networks through Range Restriction.

The results of testing the recognition algorithm of the cecum achievement in colonoscopy video of the colon mucosa are presented. The image database was formed from the results of colonoscopy procedure together with the doctors of the Yaroslavl Regional Oncology Hospital. As the architecture of the convolutional neural network, the ResNet50 modification, previously trained on the standard ImageNet base, was chosen. As a result of applying the machine learning algorithm to the test set of endoscopic images, the metric values were AUC=0.95, Fscore=0.9, when a threshold is h=0.462. The results can be used to develop a quality control system for colonoscopy procedures. The introduction of such a system in medical practice will partially automate the analysis of video data, which will subsequently lead to a decrease in the number of subjective medical errors during colonoscopy.

Automatic Identification of Appendiceal Orifice on Colonoscopy Images Using Deep Neural Network

A method for creation of computer models in complex multiply connected technological objects based on the application of machine learning methods is described. For technological information processing hierarchical neural network structure integrated into cyber-physical systems of control is developed. It allows to monitor an object condition and forecast its development trends. A description for the algorithm and program, which performs the proposed method of model building, is given.

Deep Neural Networks Application in Models with Complex Technological Objects

Machine learning (ML) has increasingly been adopted in safety-critical systems such as autonomous vehicles (AVs) and industrial robotics. In these domains, reliability and safety are important considerations, and hence it is critical to ensure the resilience of ML systems to faults and errors. On the other hand, soft errors are becoming more frequent in commodity computer systems due to the effects of technology scaling and reduced supply voltages. Further, traditional solutions for masking hardware faults such as Triple-Modular Redundancy (TMR) are prohibitively expensive in terms of their energy and performance overheads. Therefore, there is a compelling need to ensure the resilience of ML applications to soft errors on commodity hardware platforms.We first experimentally assess the resilience of safety-critical ML applications to soft errors. We demonstrate through fault injection experiments that even a single bit flip due to a soft error can lead to misclassification in Deep Neural Network (DNN) applications deployed in AVs, leading to safety violations. However, not all the errors in an DNN will result in serve consequences such as safety violations, and hence it is sufficient to protect the DNN from the ones that do. Unfortunately, finding all possible errors that result in safety violations is a very compute intensive task. We propose BinFI, a fault injection approach that efficiently injects critical faults that are highly likely to result in safety violations, based on the unique properties of DNNs. Finally, we propose Ranger, an approach to protect DNNs from critical faults with minimal performance overheads and no accuracy loss. We will conclude by presenting some of our ongoing work, and the future challenges in this area.

Error Resilient Machine Learning for Safety-Critical Systems: Position Paper

An algorithm for segmentation of cell nuclei in histological images is developed and studied. It is based on the implementation of U-Net neural network. As a means of additional processing, the use of morphological filtration is proposed. The training and testing of the algorithm was carried out on NVIDIA DGX-1 supercomputer using a histological image dataset with automatically generated markup. In the course of studies, the values of such metrics as simple match coefficient, Tversky index, and Sorensen coefficient are obtained. According to the results of calculating these metrics, a quality of histological image segmentation developed by the morphological filtering algorithm exceeds the similar performance for the algorithm without using morphological filtering, which allows us to recommend the morphological filter as a means of additional image processing at the output of the neural network algorithm in real medical practice.

Histological Images Segmentation by Convolutional Neural Network with Morphological Post-filtration

Computer-aided diagnostics of cancer pathologies based on histological image segmentation is a promising area in the field of computer vision and machine learning. To date, the successes of neural networks in image segmentation in a number of tasks are comparable to human results and can even exceed them. The paper presents a fast algorithm of histological image segmentation based on the convolutional neural network U-Net. Using this approach allows to get better results in the tasks of medical image segmentation. The developed algorithm based on neural network AlexNet was used for the creation of the automatic markup of the histological image database. The neural network algorithms were trained and tested on the NVIDIA DGX-1 supercomputer using histological images. The results of the research show that the fast algorithm based on neural network U-Net can be successfully used for the histological image segmentation in real medical practice, which is confirmed by the high level of similarity of the obtained markup with the expert one.

Anastasiya Srednyakova

Papers

Using Convolutional Neural Networks in the Problem of Cell Nuclei Segmentation on Histological Images

Histological Images Segmentation by Convolutional Neural Network with Morphological Post-filtration