There is, I think, something ethereal about i —the square root of minus one. I remember first hearing about it at school. It seemed an odd beast at that time—an intruder hovering on the edge of reality.

Usually familiarity dulls this sense of the bizarre, but in the case of i it was the reverse: over the years the sense of its surreal nature intensified. It seemed that it was impossible to write mathematics that described the real world in …

I and i

We will review some of the major results in random graphs and some of the more challenging open problems. We will cover algorithmic and structural questions. We will touch on newer models, including those related to the WWW.

Random graphs

"Grid" computing has emerged as an important new field, distinguished from conventional distributed computing by its focus on large-scale resource sharing, innovative applications, and, in some cases, high performance orientation. In this article, the authors define this new field. First, they review the "Grid problem," which is defined as flexible, secure, coordinated resource sharing among dynamic collections of individuals, institutions, and resources--what is referred to as virtual organizations. In such settings, unique authentication, authorization, resource access, resource discovery, and other challenges are encountered. It is this class of problem that is addressed by Grid technologies. Next, the authors present an extensible and open Grid architecture, in which protocols, services, application programming interfaces, and software development kits are categorized according to their roles in enabling resource sharing. The authors describe requirements that they believe any such mechanisms must satisfy and discuss the importance of defining a compact set of intergrid protocols to enable interoperability among different Grid systems. Finally, the authors discuss how Grid technologies relate to other contemporary technologies, including enterprise integration, application service provider, storage service provider, and peer-to-peer computing. They maintain that Grid concepts and technologies complement and have much to contribute to these other approaches.

The Anatomy of the Grid: Enabling Scalable Virtual Organizations

Cloud Computing, the long-held dream of computing as a utility, has the potential to transform a large part of the IT industry, making software even more attractive as a service and shaping the way IT hardware is designed and purchased. Developers with innovative ideas for new Internet services no longer require the large capital outlays in hardware to deploy their service or the human expense to operate it. They need not be concerned about overprovisioning for a service whose popularity does not meet their predictions, thus wasting costly resources, or underprovisioning for one that becomes wildly popular, thus missing potential customers and revenue. Moreover, companies with large batch-oriented tasks can get results as quickly as their programs can scale, since using 1000 servers for one hour costs no more than using one server for 1000 hours. This elasticity of resources, without paying a premium for large scale, is unprecedented in the history of IT. Cloud Computing refers to both the applications delivered as services over the Internet and the hardware and systems software in the datacenters that provide those services. The services themselves have long been referred to as Software as a Service (SaaS). The datacenter hardware and software is what we will call a Cloud. When a Cloud is made available in a pay-as-you-go manner to the general public, we call it a Public Cloud; the service being sold is Utility Computing. We use the term Private Cloud to refer to internal datacenters of a business or other organization, not made available to the general public. Thus, Cloud Computing is the sum of SaaS and Utility Computing, but does not include Private Clouds. People can be users or providers of SaaS, or users or providers of Utility Computing. We focus on SaaS Providers (Cloud Users) and Cloud Providers, which have received less attention than SaaS Users. From a hardware point of view, three aspects are new in Cloud Computing.

/pdf/above-the-clouds-a-berkeley-view-of-cloud-computing-tkj3etim13.pdf

Above the Clouds: A Berkeley View of Cloud Computing

A 2001 IBM manifesto observed that a looming software complexity crisis -caused by applications and environments that number into the tens of millions of lines of code - threatened to halt progress in computing. The manifesto noted the almost impossible difficulty of managing current and planned computing systems, which require integrating several heterogeneous environments into corporate-wide computing systems that extend into the Internet. Autonomic computing, perhaps the most attractive approach to solving this problem, creates systems that can manage themselves when given high-level objectives from administrators. Systems manage themselves according to an administrator's goals. New components integrate as effortlessly as a new cell establishes itself in the human body. These ideas are not science fiction, but elements of the grand challenge to create self-managing computing systems.

The vision of autonomic computing

Overlays have enabled several new and popular distributed applications such as Akamai, Kazaa, and Bittorrent However, the lack of an overlay-aware network stack has hindered the widespread use of general purpose overlay packet delivery services [16, 29, 26] In this paper, we describe the design and implementation of Oasis, a system and toolkit that enables legacy operating systems to access overlay-based packet delivery services Oasis combines a set of ideas - network address translation, name resolution, packet capture, dynamic code execution - to provide greater user choice We are in the process of making the Oasis toolkit available for public use, specifically, to ease the development of PlanetLab-based packet delivery services

/pdf/oasis-an-overlay-aware-network-stack-4m6dcx3ecm.pdf

Oasis: an overlay-aware network stack

Rapid advances in high performance computing are making possible more complete and accurate computer-based modeling of complex physical phenomena, such as weather front interactions, dynamics of chemical reactions, numerical aerodynamic analysis of airframes, and ocean-land-atmosphere interactions Many of these 'grand challenge' applications are as demanding of the underlying storage system, in terms of their capacity and bandwidth requirements, as they are on the computational power of the processor A global view of the Earth's ocean chlorophyll and land vegetation requires over 2 terabytes of raw satellite image data In this paper, we describe our planned research program in high capacity, high bandwidth storage systems The project has four overall goals First, we will examine new methods for high capacity storage systems, made possible by low cost, small form factor magnetic and optical tape systems Second, access to the storage system will be low latency and high bandwidth To achieve this, we must interleave data transfer at all levels of the storage system, including devices, controllers, servers, and communications links Latency will be reduced by extensive caching throughout the storage hierarchy Third, we will provide effective management of a storage hierarchy, extending the techniques already developed for the Log Structured File System Finally, we will construct a protototype high capacity file server, suitable for use on the National Research and Education Network (NREN) Such research must be a Cornerstone of any coherent program in high performance computing and communications

Robo-line storage: Low latency, high capacity storage systems over geographically distributed networks

We propose FlexNIC, a flexible network DMA interface that can be used by operating systems and applications alike to reduce packet processing overheads. The recent surge of network I/O performance has put enormous pressure on memory and software I/O processing subsystems. Yet even at high speeds, flexibility in packet handling is still important for security, performance isolation, and virtualization.

Thus, our proposal moves some of the packet processing traditionally done in software to the NIC DMA controller, where it can be done flexibly and at high speed. We show how FlexNIC can benefit widely used data center server applications, such as key-value stores.

/pdf/flexnic-rethinking-network-dma-4snmll771m.pdf

FlexNIC: rethinking network DMA

Rapid advances in high performance computing are making possible more complete and accurate computer-based modeling of complex physical phenomena, such as weather front interactions, dynamics of chemical reactions, numerical aerodynamic analysis of airframes, and ocean-land-atmosphere interactions. Many of these "grand challenge" applications are as demanding of the underlying storage system, in terms of their capacity and bandwidth requirements, as they are on the computational power of the processor. A global view of the Earth''s ocean chlorophyll and land vegetation requires over 2 terabytes of raw satellite image data [ISTP91]! In this paper, we describe our planned research program in high capacity, high bandwidth storage systems. The project has four overall goals. First, we will examine new methods for high capacity storage systems, made possible by low cost, small formfactor magnetic and optical tape systems. Second, access to the storage system will be low latency and high bandwidth. To achieve this, we must interleave data transfer at all levels of the storage system, including devices, controllers, servers, and communications links. Latency will be reduced by extensive caching throughout the storage hierarchy, extending the techniques already developed by Ousterhout for his Log Structured File System. Finally, we will construct a prototype high capacity file server, suitable for use on the National Research and Education Network (NREN). Such research must be a cornerstone of any coherent program in high performance computing and communications.

Robo-line Storage: Low Latency, High Capacity Storage Systems over

In the past 5 years, disk costs have been falling at a factor of 2 per year. Today, terabyte capacity disk storage systems are feasible. Given the rapidly increasing areal density and disk transfer rates, these systems will have significant cost/performance advantages over tape libraries of similar capacity. If commodity hardware is used, large disk systems can avoid the high cost of custom designed disk arrays, as well as their limitations on scalability. This paper presents Tertiary Disk, a 3TB disk storage system built from commodity hardware. Tertiary Disk uses PCs and switched networks to connect 370 8GB disks. We show that even though commodity hardware is used, the overall system can be more reliable than a single disk. A cost analysis of our prototype shows that the additional infrastructure needed to create a terabyte scale storage system is a fraction of the cost of the underlying disks. In comparison, the costs of large disk arrays are many times the cost of the underlying disks. We also present performance measurements from our prototype, and show that the PC architecture is a good match for hosting a large number of disks. Overall, we show that storage system designs like Tertiary Disk have cost/performance and reliability advantages over most choices available today for terabyte scale storage.

Thomas Anderson

Papers

Oasis: an overlay-aware network stack

Robo-line storage: Low latency, high capacity storage systems over geographically distributed networks

FlexNIC: rethinking network DMA

Robo-line Storage: Low Latency, High Capacity Storage Systems over

Tertiary Disk: Large Scale Distributed Storage