Reliable performance evaluations require the use of representative workloads. This is no easy task since modern computer systems and their workloads are complex, with many interrelated attributes and complicated structures. Experts often use sophisticated mathematics to analyze and describe workload models, making these models difficult for practitioners to grasp. This book aims to close this gap by emphasizing the intuition and the reasoning behind the definitions and derivations related to the workload models. It provides numerous examples from real production systems, with hundreds of graphs. Using this book, readers will be able to analyze collected workload data and clean it if necessary, derive statistical models that include skewed marginal distributions and correlations, and consider the need for generative models and feedback from the system. The descriptive statistics techniques covered are also useful for other domains.

Workload Modeling for Computer Systems Performance Evaluation

Recent years have seen the massive migration of enterprise applications to the cloud. One of the challenges posed by cloud applications is Quality-of-Service (QoS) management, which is the problem of allocating resources to the application to guarantee a service level along dimensions such as performance, availability and reliability. This paper aims at supporting research in this area by providing a survey of the state of the art of QoS modeling approaches suitable for cloud systems. We also review and classify their early application to some decision-making problems arising in cloud QoS management.

/pdf/quality-of-service-in-cloud-computing-modeling-techniques-1ivbgaewet.pdf

Quality-of-service in cloud computing: modeling techniques and their applications

A method and apparatus for managing, modeling, predicting, allocating and utilizing resources and bottlenecks in a computer network managing, predicting and displaying of capacity, allocating and utilizing of resources, as well as actual and potential performance-degrading resource shortages in a computer network, is provided. Specifically, exemplary implementations of the present invention provide a method, system and apparatus for calculating, detecting, predicting, and presenting resource allocation, utilization, capacity bottlenecks and availability information, in a computer network, particularly in a virtualized computer environment.

Method, System and Apparatus for Managing, Modeling, Predicting, Allocating and Utilizing Resources and Bottlenecks in a Computer Network

SLA based resource allocation policies in autonomic environments

Workload flows in enterprise systems that use the multi-tier paradigm are often characterized as bursty, i.e., exhibit a form of temporal dependence. Burstiness often results in dramatic degradation of the perceived user performance, which is extremely difficult to capture with existing capacity planning models. The main reason behind this deficiency of traditional capacity planning models is that the user perceived performance is the result of the complex interaction of a very complex workload with a very complex system. In this paper, we propose a simple and effective methodology for detecting burstiness symptoms in multi-tier systems rather than identifying the low-level exact cause of burstiness as traditional models would require. We provide an effective way to incorporate this information into a surprisingly simple and effective modeling methodology. This new modeling methodology is based on the index of dispersion of the service process at a server, which is inferred by observing the number of completions within the concatenated busy periods of that server. The index of dispersion together with other measurements that reflect the "estimated" mean and the 95th percentile of service times are used to derive a Markov-modulated process that captures well burstiness and variability of the true service process, despite inevitable inaccuracies that result from inexact and limited measurements. Detailed experimentation on a TPC-W testbed where all measurements are obtained by HP (Mercury) Diagnostics, a commercially available tool, shows that the proposed technique offers a simple yet powerful solution to the difficult problem of inferring accurate descriptors of the service time process from coarse measurements of a given system. Experimental and model prediction results are in excellent agreement and argue strongly for the effectiveness of the proposed methodology under both bursty and non-bursty workloads.

/pdf/burstiness-in-multi-tier-applications-symptoms-causes-and-59wpz026f2.pdf

Burstiness in multi-tier applications: symptoms, causes, and new models

A system and method for performance modeling for an information technology (IT) system having a server(s) for performing a number of types of transactions includes receiving data for system topology and transaction flows and receiving performance measurement data for the IT system. The measurement data is clustered into multiple regimes based on similarities. Service demand and network delay parameters may be inferred based on clustered data.

Method and system for on-line performance modeling using inference for real production it systems

Parameter inference of queueing models for IT systems using end-to-end measurements

As Information Technology (IT) matures and expands in the scope of available applications, IT systems increase at a fascinating rate in both size and complexity. For example, today, a typical Web service hosting center may have hundreds of nodes and dozens of different applications simultaneously running on it. Each of the nodes in turn has often multiple processors and layered caches. These nodes make use of both local and shared storage systems. The size and complexity of such systems make performance modeling much more difficult, if at all tractable. Detailed modeling, fine tuning and accurate analysis can be carried out only on very small IT systems or very small components in a system. On the other hand, IT systems have become vital to business. Losses of millions of dollars per minute when an IT system goes down are well-documented, leading users to impose stringent QoS requirements. In IT outsourcing, servicelevel agreements (SLA) stipulate service quality guarantees, often with associated penalties in case of violations. Thus, predictive modeling is vital in the capacity planning and QoS management of IT systems. We propose a method based on queueing networks. While many parameters such as the queueing discipline, number of servers in a queueing station, buffer size, etc., of the queueing network can be easily set by the modeler, the service requirements (i.e. job processing times) of customers are more difficult to estimate. However, without an accurate estimation of the service requirements, performance predictions can vary wildly. In other words, a principal difficulty in building a valid queueing network of an IT system is the fine tuning of the service requirements. This step requires benchmarking of the real system and an adjustment of the

With the advances of computer hardware and software technologies, electronic businesses are moving towards the on-demand era, where services and applications can be deployed or accommodated in a dynamic and autonomic fashion. This leads to a more flexible and efficient way to manage various system resources. For on-demand services and applications, performance modeling and analysis play key roles in many aspects of such an autonomic system. In this paper, we present a comprehensive toolset developed for workload characterization, performance modeling and analysis, and on-line control. The development of the toolset is based on state-of-the art techniques in statistical analysis, queueing theory, scheduling techniques, and on-line control methodologies. Built on a flexible software architecture, this toolset provides significant value for key business processes. This includes capacity planning, performance prediction, performance engineering and on-line control of system resources.

Fan Zhang

Papers

Method and system for on-line performance modeling using inference for real production it systems

Parameter inference of queueing models for IT systems using end-to-end measurements

Parameter inference of queueing models for IT systems using end-to-end measurements

A Comprehensive Toolset for Workload Characterization, Performance Modeling, and Online Control