Technical foundations introduction software reliability and system reliability the operational profile software reliability modelling survey model evaluation and recalibration techniques practices and experiences best current practice of SRE software reliability measurement experience measurement-based analysis of software reliability software fault and failure classification techniques trend analysis in validation and maintenance software reliability and field data analysis software reliability process assessment emerging techniques software reliability prediction metrics software reliability and testing fault-tolerant SRE software reliability using fault trees software reliability process simulation neural networks and software reliability. Appendices: software reliability tools software failure data set repository.

/pdf/handbook-of-software-reliability-engineering-32dwvaljmm.pdf

Handbook of software reliability engineering

コンピュータ・サイエンス : ACM computing surveys

Cloud services and applications prove indispensable amid today’s modern utility-based computing. The cloud has displayed a disruptive and growing impact on everyday computing tasks. However, facilitating the orchestration of cloud resources to build such cloud services and applications is yet to unleash its entire magnitude of power. Accordingly, it is paramount to devise a unified and comprehensive analysis framework to accelerate fundamental understanding of cloud resource orchestration in terms of concepts, paradigms, languages, models, and tools. This framework is essential to empower effective research, comprehension, comparison, and selection of cloud resource orchestration models, languages, platforms, and tools. This article provides such a comprehensive framework while analyzing the relevant state of the art in cloud resource orchestration from a novel and holistic viewpoint.

/pdf/a-taxonomy-and-survey-of-cloud-resource-orchestration-2nhpex4ar7.pdf

A Taxonomy and Survey of Cloud Resource Orchestration Techniques

This paper presents an in-depth characterization of the resiliency of more than 5 million HPC application runs completed during the first 518 production days of Blue Waters, a 13.1 petaflop Cray hybrid supercomputer. Unlike past work, we measure the impact of system errors and failures on user applications, i.e., the compiled programs launched by user jobs that can execute across one or more XE (CPU) or XK (CPU+GPU) nodes. The characterization is performed by means of a joint analysis of several data sources, which include workload and error/failure logs. In order to relate system errors and failures to the executed applications, we developed LogDiver, a tool to automate the data pre-processing and metric computation. Some of the lessons learned in this study include: i) while about 1.53% of applications fail due to system problems, the failed applications contribute to about 9% of the production node hours executed in the measured period, i.e., the system consumes computing resources, and system-related issues represent a potentially significant energy cost for the work lost, ii) there is a dramatic increase in the application failure probability when executing full-scale applications: 20x (from 0.008 to 0.162) when scaling XE applications from 10,000 to 22,000 nodes, and 6x (from 0.02 to 0.129) when scaling GPU/hybrid applications from 2000 to 4224 nodes, and iii) the resiliency of hybrid applications is impaired by the lack of adequate error detection capabilities in hybrid nodes.

Measuring and Understanding Extreme-Scale Application Resilience: A Field Study of 5,000,000 HPC Application Runs

Technologies for virtual machine placement within a data center are described herein. An example method may include determining a shared threat potential for a virtual machine based, at least in part, on a degree of co-location the virtual machine has with a current virtual machine operating on a physical machine, determining a workload threat potential for the virtual machine based, at least in part, on a level of advantage associated with placing the virtual machine on the physical machine, determining a threat potential for the virtual machine based, at least in part, on a combination of the shared threat potential and the workload threat potential, and placing the virtual machine on the physical machine based on the threat potential.

Virtual machine placement

This paper characterizes operational failures of a production Custom Package Good Software-as-a-Service (SaaS) platform. Events log collected over 283 days of in-field operation are used to characterize platform failures. The characterization is performed by estimating (i) common failure types of the platform, (ii) key factors impacting platform failures, (iii) failure rate, and (iv) how user workload (files submitted for processing) impacts on the failure rate. The major findings are: (i) 34.1% of failures are caused by unexpected values in customers' data, (ii) nearly 33% of the failures are because of timeout, and (iii) the failure rate increases if the workload intensity (transactions/second) increases, while there is no statistical evidence of being influenced by the workload volume (size of users' data). Finally, the paper presents the lessons learned and how the findings and the implemented analysis tool allow platform developers to improve platform code, system settings and customer management.

Characterization of operational failures from a business data processing SaaS platform

In response to the revival of virtualized technology by Rosenblum and Garfinkel [2005], NIST defined cloud computing, a new paradigm in service computing infrastructures. In cloud environments, the basic security mechanism is ingrained in virtualization—that is, the execution of instructions at different privilege levels. Despite its obvious benefits, the caveat is that a crashed virtual machine (VM) is much harder to recover than a crashed workstation. When crashed, a VM is nothing but a giant corrupt binary file and quite unrecoverable by standard disk-based forensics. Therefore, VM crashes should be avoided at all costs. Security is one of the major contributors to such VM crashes. This includes compromising the hypervisor, cloud storage, images of VMs used infrequently, and remote cloud client used by the customer as well as threat from malicious insiders. Although using secure infrastructures such as private clouds alleviate several of these security problems, most cloud users end up using cheaper options such as third-party infrastructures (i.e., private clouds), thus a thorough discussion of all known security issues is pertinent. Hence, in this article, we discuss ongoing research in cloud security in order of the attack scenarios exploited most often in the cloud environment. We explore attack scenarios that call for securing the hypervisor, exploiting co-residency of VMs, VM image management, mitigating insider threats, securing storage in clouds, abusing lightweight software-as-a-service clients, and protecting data propagation in clouds. Wearing a practitioner's glasses, we explore the relevance of each attack scenario to a service company like Infosys. At the same time, we draw parallels between cloud security research and implementation of security solutions in the form of enterprise security suites for the cloud. We discuss the state of practice in the form of enterprise security suites that include cryptographic solutions, access control policies in the cloud, new techniques for attack detection, and security quality assurance in clouds.

Secure the Cloud: From the Perspective of a Service-Oriented Organization

This paper investigates SLA violations of a production SaaS platform by means of joint use of field failure data analysis (FFDA) and fault injection. The objective of this study is to diagnose the causes of SLA violations, pinpoint critical failure modes under realistic error assumptions and identify potential means to increase the user perceived availability of the platform and assurance of SLA requirements. We base our study on 283 days of logs obtained during the production time of the platform, while it was employed to process business data received by 42 customers in 22 countries. In this paper, we develop a set of tools that include i) a FFDA toolset used to analyze the data extracted from the platform and by the operating system event logs and ii) a. NET/C++ injector able to automate the injection of specific runtime errors in the production code and the collection of results. Major findings include i) 93% of all service level agreement (SLA) violations were due to system failures, ii) there were a few cases of bursts of SLA violations that could not be diagnosed from the logs and were revealed from the performed injections, and iii) the error injection revealed several error propagation paths leading to data corruptions that could not be detected from the analysis of failure data.

Analysis and Diagnosis of SLA Violations in a Production SaaS Cloud

A system and method for evaluating performance of a software application. The present invention includes analyzing by one or more computing devices a plurality of program code lines of the software application stored in one or more computer databases. Further, one or more equivalent program regions within the plurality of program code lines may be identified. One or more markers in the identified one or more equivalent program regions may be inserted and stored in the one or more computer databases. Further, the plurality of program code lines may be compiled and assembled respectively to generate an executable code. The executable code may include a plurality of instructions. Further, performance metrics of the software application may be measured by manipulating the plurality of instructions based on the one or more equivalent program regions identified by the inserted one or more markers and executing the executable code.

Methods of software performance evaluation by run-time assembly code execution and devices thereof

This article consists of a collection of slides from the author's PowerPoint conference presentation. A procedure and a statistical model to measure the risk of error when processing a workload X is proposed. It is concluded that architecting hazard-aware load balancer for Cloud SaaS can be promising.

Geetika Goel

Papers

Characterization of operational failures from a business data processing SaaS platform

Secure the Cloud: From the Perspective of a Service-Oriented Organization

Analysis and Diagnosis of SLA Violations in a Production SaaS Cloud

Methods of software performance evaluation by run-time assembly code execution and devices thereof

Measurements-Based Analysis of Workload-Error Relationship in a Production SaaS Cloud