Proceedings ArticleDOI
On the Diversity of Machine Learning Models for System Reliability
Fumio Machida
- pp 276-285
Reads0
Chats0
TLDR
The reliability model for three-version machine learning architecture is constructed with a diversity measure defined as the intersection of error spaces in the sample space and a necessary condition under which three- version architecture achieves a higher system reliability than a single machine learning module is derived.Abstract:
The diversity of system components is one of the important contributing factors of reliable and secure software systems. In a software fault-tolerant system using diverse versions of software components, a component failure caused by defects or malicious attacks can be covered by other versions. Machine learning systems can also benefit from such a multi-version approach to improve the system reliability. Nevertheless, there are few studies addressing this issue. In this paper, we experimentally analyze how outputs of machine learning modules can be diversified by using different versions of machine learning algorithms, neural network architectures and perturbated input data. The experiments are conducted on image classification tasks of MNIST data set and Belgian Traffic Sign data set. Different neural network architectures, support vector machines and random forests are used for constructing diverse machine learning models. The diversity is characterized by the coverage of errors over the test samples. We observe that the different machine learning models have quite different error coverages that can be leveraged for system reliability design. Based on the experimental results, we construct the reliability model for three-version machine learning architecture with a diversity measure defined as the intersection of error spaces in the sample space. From the presented reliability model, we derive a necessary condition under which three-version architecture achieves a higher system reliability than a single machine learning module.read more
Citations
More filters
Journal ArticleDOI
Software Engineering for AI-Based Systems: A Survey
TL;DR: In this paper , the authors conducted a systematic mapping study on software engineering approaches for building, operating, and maintaining AI-based systems and identified multiple SE approaches for AIbased systems, which they classified according to the SWEBOK areas.
Posted Content
Software Engineering for AI-Based Systems: A Survey.
Silverio Martínez-Fernández,Justus Bogner,Xavier Franch,Marc Oriol,Julien Siebert,Adam Trendowicz,Anna Maria Vollmer,Stefan Wagner +7 more
TL;DR: In this paper, the authors conducted a systematic mapping study on software engineering approaches for building, operating, and maintaining AI-based systems and identified multiple SE approaches for AIbased systems, which they classified according to the SWEBOK areas.
Proceedings ArticleDOI
A Queueing Analysis of Multi-model Multi-input Machine Learning Systems
TL;DR: In this article, the authors proposed queuing models for analyzing a multi-model multi-input machine learning system (MLS) performance in two architectures, namely a parallel MLS and a shared MLS.
Proceedings ArticleDOI
Reliability Models and Analysis for Triple-model with Triple-input Machine Learning Systems
Qiang Wen,Fumio Machida +1 more
TL;DR: This study aims to improve the ML system reliability through a software architecture approach inspired by N-version programming to compare the reliability of a triple-model with triple-input (TMTI) architecture with other variants of three-version and two-version architectures.
Proceedings ArticleDOI
Reliability Models and Analysis for Triple-model with Triple-input Machine Learning Systems
TL;DR: Wang et al. as discussed by the authors proposed a software architecture approach inspired by N-version programming to improve the ML system reliability through using diversity metrics for ML models and input data sets to compare the reliability of a triple-model with triple-input (TMTI) architecture with other variants of three-version and two-version architectures.
References
More filters
Journal ArticleDOI
Random Forests
TL;DR: Internal estimates monitor error, strength, and correlation and these are used to show the response to increasing the number of features used in the forest, and are also applicable to regression.
Journal ArticleDOI
Deep learning
TL;DR: Deep learning is making major advances in solving problems that have resisted the best attempts of the artificial intelligence community for many years, and will have many more successes in the near future because it requires very little engineering by hand and can easily take advantage of increases in the amount of available computation and data.
Book
Deep Learning
TL;DR: Deep learning as mentioned in this paper is a form of machine learning that enables computers to learn from experience and understand the world in terms of a hierarchy of concepts, and it is used in many applications such as natural language processing, speech recognition, computer vision, online recommendation systems, bioinformatics, and videogames.
Journal ArticleDOI
Support-Vector Networks
Corinna Cortes,Vladimir Vapnik +1 more
TL;DR: High generalization ability of support-vector networks utilizing polynomial input transformations is demonstrated and the performance of the support- vector network is compared to various classical learning algorithms that all took part in a benchmark study of Optical Character Recognition.
Proceedings Article
Explaining and Harnessing Adversarial Examples
TL;DR: It is argued that the primary cause of neural networks' vulnerability to adversarial perturbation is their linear nature, supported by new quantitative results while giving the first explanation of the most intriguing fact about them: their generalization across architectures and training sets.