This issue of the IBM Systems Journal explores a broad set of ideas and approaches to autonomic computing--some first steps in what we see as a journey to create more self-managing computing systems Autonomic computing represents a collection and integration of technologies that enable the creation of an information technology computing infrastructure for IBM's agenda for the next era of computing--e-business on demand This paper presents an overview of IBM's autonomic computing initiative It examines the genesis of autonomic computing, the industry and marketplace drivers, the fundamental characteristics of autonomic systems, a framework for how systems will evolve to become more self-managing, and the key role for open industry standards needed to support autonomic behavior in heterogeneous system environments Technologies explored in each of the papers presented in this issue are introduced for the reader

https://cs.drexel.edu/~spiros/teaching/CS576/papers/autonomic.pdf

The dawning of the autonomic computing era

Scheduling jobs on the IBM SP2 system and many other distributed-memory MPPs is usually done by giving each job a partition of the machine for its exclusive use. Allocating such partitions in the order in which the jobs arrive (FCFS scheduling) is fair and predictable, but suffers from severe fragmentation, leading to low utilization. This situation led to the development of the EASY scheduler which uses aggressive backfilling: Small jobs are moved ahead to fill in holes in the schedule, provided they do not delay the first job in the queue. We compare this approach with a more conservative approach in which small jobs move ahead only if they do not delay any job in the queue and show that the relative performance of the two schemes depends on the workload. For workloads typical on SP2 systems, the aggressive approach is indeed better, but, for other workloads, both algorithms are similar. In addition, we study the sensitivity of backfilling to the accuracy of the runtime estimates provided by the users and find a very surprising result. Backfilling actually works better when users overestimate the runtime by a substantial factor.

/pdf/utilization-predictability-workloads-and-user-runtime-58ij8b99dn.pdf

Utilization, predictability, workloads, and user runtime estimates in scheduling the IBM SP2 with backfilling

System, method, and tangible computer-readable storage media are disclosed for providing a brokering service for compute resources The method includes, at a brokering service, polling a group of separately administered compute environments to identify resource capabilities and information, each compute resource environment including the group of managed nodes for processing workload, receiving a request for compute resources at the brokering service system, the request for compute resources being associated with a service level agreement (SLA) and based on the resource capabilities across the group of compute resource environments, selecting compute resources in one or more of the group of compute resource environments The brokering service system receives workload associated with the request and communicates the workload to the selected resources for processing The brokering services system can aggregate resources for multiple cloud service providers and act as an advocate for or a guarantor of the SLA associated with the workload

System and method of brokering cloud computing resources

The scheduling of jobs on parallel supercomputer is becoming the subject of much research. However, there is concern about the divergence of theory and practice. We review theoretical research in this area, and recommendations based on recent results. This is contrasted with a proposal for standard interfaces among the components of a scheduling system, that has grown from requirements in the field.

Theory and Practice in Parallel Job Scheduling

The workload on parallel supercomputers: modeling the characteristics of rigid jobs

In this paper we have characterized the inter-arrival time and service time distributions for jobs at a large MPP supercomputing center. Our findings show that the distributions are dispersive and complex enough that they require Hyper Erlang distributions to capture the first three moments of the observed workload. We also present the parameters from the characterization so that they can be easily used for both theoretical studies and the simulations of various scheduling algorithms.

Modeling of Workload in MPPs

A logical partition in an IBM pSeriesTM symmetric multiprocessor (SMP) system is a subset of the hardware of the SMP that can host an operating system (OS) instance. Dynamic reconfiguration (DR) on these logically partitioned servers enables the movement of hardware resources (such as processors, memory, and I/O slots) from one logical partition to another without requiring reboots. This capability also enables an autonomic agent to monitor usage of the partitions and automatically move hardware resources to a needy OS instance nondisruptively. Today, as SMPs and nonuniform memory access (NUMA) systems become larger and larger, the ability to run several instances of an operating system(s) on a given hardware system, so that each OS instance plus its subsystems scale or perform well, has the advantage of an optimal aggregate performance, which can translate into cost savings for customers. Though static partitioning provides a solution to this overall performance optimization problem, DR enables an improved solution by providing the capability to dynamically move hardware resources to a needy OS instance in a timely fashion to match workload demands. Hence, DR capabilities serve as key building blocks for workload managers to provide self-optimizing and self-configuring features. Besides dynamic resource balancing, DR also enables Dynamic Capacity Upgrade on Demand, and self-healing features such as Dynamic CPU Sparing, a winning solution for users in this age of rapid growth in Web servers on the Internet.

Dynamic reconfiguration: Basic building blocks for autonomic computing on IBM pSeries servers

System, method and computer program product for supporting a plurality of Access Control List types for a file system in an operating system in a data processing system. An Access Control List supporting system for managing access to a file system in a data processing system has at least one file system in an operating system of the data processing system, and an Access Control List management framework in the operating system and external to the at least one file system for managing access to the at least one file system. The Access Control List supporting system of the invention removes ACL management and access check-related functions from the at least one file system to an external Access Control List management framework, thus enabling an operating system to support a plurality of Access Control List types using the same Access Control List management framework and enabling new Access Control List types to be added to the operating system dynamically while the operating system is running.

System and method for supporting a plurality of access control list types for a file system in an operating system

A method and apparatus for sharing memory in a multiprocessor computing system. More specifically, this invention provides a number of system buses with each bus being connected to a respective memory controller which controls a corresponding partition of the memory. Any one of the processors can use any one of the system buses to send real addresses to the connected memory controller which then converts the real addresses into physical addresses corresponding to the partition of memory that is controlled by the receiving memory controller. The processors can be dynamically assigned to different partitions of the memory by via a switching mechanism.

Secure partitioning of shared memory based multiprocessor system

In this paper we analyze the behavior of a gang-scheduling system that we are developing for the ASCI Blue-Pacific machines. Starting with a real workload obtained from job logs of one of the ASCI machines, we generate a statistical model of this workload using Hyper Erlang distributions. We then vary the parameters of those distributions to generate various workloads, representative of different operating points of the machine. Through simulation we obtain performance characteristics for three different scheduling strategies: (i) first-come first-serve, (ii) gang-scheduling, and (iii) backfilling. Our results show that both backfilling and gang-scheduling with moderate multiprogramming levels are much more effective than simple first-come first-serve scheduling. In addition, we show that gang-scheduling can display better performance characteristics than backfilling, particularly for large production jobs.

/pdf/an-evaluation-of-parallel-job-scheduling-for-asci-blue-2wiqori5mj.pdf

Joefon Jann

Papers

Modeling of Workload in MPPs

Dynamic reconfiguration: Basic building blocks for autonomic computing on IBM pSeries servers

System and method for supporting a plurality of access control list types for a file system in an operating system

Secure partitioning of shared memory based multiprocessor system

An Evaluation of Parallel Job Scheduling for ASCI Blue-Pacific