http://cloud.pubs.dbs.uni-leipzig.de/sites/cloud.pubs.dbs.uni-leipzig.de/files/Brandic2008CloudcomputingandemergingITplatformsVisionhypeand.pdf

Cloud computing and emerging IT platforms: Vision, hype, and reality for delivering computing as the 5th utility

Matrix Factorization Techniques for Recommender Systems

Large-scale Internet services require a computing infrastructure that can beappropriately described as a warehouse-sized computing system. The cost ofbuilding datacenter facilities capable of delivering a given power capacity tosuch a computer can rival the recurring energy consumption costs themselves.Therefore, there are strong economic incentives to operate facilities as closeas possible to maximum capacity, so that the non-recurring facility costs canbe best amortized. That is difficult to achieve in practice because ofuncertainties in equipment power ratings and because power consumption tends tovary significantly with the actual computing activity. Effective powerprovisioning strategies are needed to determine how much computing equipmentcan be safely and efficiently hosted within a given power budget.In this paper we present the aggregate power usage characteristics of largecollections of servers (up to 15 thousand) for different classes ofapplications over a period of approximately six months. Those observationsallow us to evaluate opportunities for maximizing the use of the deployed powercapacity of datacenters, and assess the risks of over-subscribing it. We findthat even in well-tuned applications there is a noticeable gap (7 - 16%)between achieved and theoretical aggregate peak power usage at the clusterlevel (thousands of servers). The gap grows to almost 40% in wholedatacenters. This headroom can be used to deploy additional compute equipmentwithin the same power budget with minimal risk of exceeding it. We use ourmodeling framework to estimate the potential of power management schemes toreduce peak power and energy usage. We find that the opportunities for powerand energy savings are significant, but greater at the cluster-level (thousandsof servers) than at the rack-level (tens). Finally we argue that systems needto be power efficient across the activity range, and not only at peakperformance levels.

/pdf/power-provisioning-for-a-warehouse-sized-computer-3iyo1da2iy.pdf

Power provisioning for a warehouse-sized computer

Usually, a proof of a theorem contains more knowledge than the mere fact that the theorem is true. For instance, to prove that a graph is Hamiltonian it suffices to exhibit a Hamiltonian tour in it; however, this seems to contain more knowledge than the single bit Hamiltonian/non-Hamiltonian.In this paper a computational complexity theory of the “knowledge” contained in a proof is developed. Zero-knowledge proofs are defined as those proofs that convey no additional knowledge other than the correctness of the proposition in question. Examples of zero-knowledge proof systems are given for the languages of quadratic residuosity and 'quadratic nonresiduosity. These are the first examples of zero-knowledge proofs for languages not known to be efficiently recognizable.

/pdf/the-knowledge-complexity-of-interactive-proof-systems-31apre0ecf.pdf

The knowledge complexity of interactive proof-systems

Cloud computing systems fundamentally provide access to large pools of data and computational resources through a variety of interfaces similar in spirit to existing grid and HPC resource management and programming systems.  These types of systems offer a new programming target for scalable application developers and have gained popularity over the past few years.  However, most cloud computing systems in operation today are proprietary, rely upon infrastructure that is invisible to the research community, or are not explicitly designed to be instrumented and modified by systems researchers. In this work, we present Eucalyptus -- an open-source software framework for cloud computing that implements what is commonly referred to as Infrastructure as a Service (IaaS); systems that give users the ability to run and control entire virtual machine instances deployed across a variety physical resources. We outline the basic principles of the Eucalyptus design, detail important operational aspects of the system, and discuss architectural trade-offs that we have made in order to allow Eucalyptus to be portable, modular and simple to use on infrastructure commonly found within academic settings.  Finally, we provide evidence that Eucalyptus enables users familiar with existing Grid and HPC systems to explore new cloud computing functionality while maintaining access to existing, familiar application development software and Grid middle-ware.

/pdf/the-eucalyptus-open-source-cloud-computing-system-2ahud2ixme.pdf

The Eucalyptus Open-Source Cloud-Computing System

This paper describes the experience of developing a system for creation of distributed linked configurations of heterogeneous resources (slices) in GENI. Our work leverages a number of unique architectural solutions (distributed architecture, declarative resource specifications, unique approach to slice instantiation) which is applicable to a wider set of problems related to autonomic co-scheduling and provisioning of heterogeneous networked resources. We discuss the architecture, the resource description mechanisms and some of the algorithms used to enable our system. We conclude with an analysis of a real experiment at allocating resources from multiple providers across a very wide geographic area (spanning Massachusetts, Illinois and North Carolina) to create a single private Layer 2 network connecting virtual machines on the campus of Duke University to a sensor testbed at University of Massachusetts, Amherst.

/pdf/networked-cloud-orchestration-a-geni-perspective-7sm3yhq2n8.pdf

Networked cloud orchestration: A GENI perspective

Cloud providers have begun to allow users to bid for surplus servers on a spot market. These servers are allocated if a user's bid price is higher than their market price and revoked otherwise. Thus, analyzing price data to derive optimal bidding strategies has become a popular research topic. In this paper, we argue that sophisticated bidding strategies, in practice, do not provide any advantages over simple strategies for multiple reasons. First, due to price characteristics, there are a wide range of bid prices that yield the optimal cost and availability. Second, given the large number of spot markets, there is always a market with available surplus resources. Thus, if resources become unavailable due to a price spike, users need not wait until the spike subsides, but can instead provision a new spot resource elsewhere and migrate to it. Third, current spot market rules enable users to place maximum bids for resources without any penalty. Given bidding's irrelevance, users can adopt trivial bidding strategies and focus instead on modifying applications to efficiently seek out and migrate to the lowest cost resources.

/pdf/how-not-to-bid-the-cloud-33s8hi85s8.pdf

How not to bid the cloud

Utility computing delivers compute and storage resources to applications as an 'on-demand utility', much like electricity, from a distributed collection of computing resources. There is great interest in running database applications on utility resources (e.g., Oracle's Grid initiative) due to reduced infrastructure and management costs, higher resource utilization, and the ability to handle sudden load surges. Virtual Machine (VM) technology offers powerful mechanisms to manage a utility resource infrastructure. However, provisioning VMs for applications to meet system performance goals, e.g., to meet service level agreements (SLAs), is an open problem. We are building two systems at Duke - Shirako and NIMO - that collectively address this problem. Shirako is a toolkit for leasing VMs to an application from a utility resource infrastructure. NIMO learns application performance models using novel techniques based on active learning, and uses these models to guide VM provisioning in Shirako. We will demonstrate: (a) how NIMO learns performance models in an online and automatic fashion using active learning; and (b) how NIMO uses these models to do automated and on-demand provisioning of VMs in Shirako for two classes of database applications - multi-tier web services and computational science workflows.

https://users.cs.duke.edu/~shivnath/papers/sigmod090d-shivam.pdf

Automated and on-demand provisioning of virtual machines for database applications

Monitoring and controlling electrical loads is crucial for demand-side energy management in smart grids. Home automation (HA) protocols, such as X10 and Insteon, have provided programmatic load control for many years, and are being widely deployed in early smart grid field trials. While HA protocols include basic monitoring functions, extreme bandwidth limitations (

/pdf/exploiting-home-automation-protocols-for-load-monitoring-in-26chtjg9sc.pdf

Exploiting home automation protocols for load monitoring in smart buildings

Buildings account for over 75% of the electricity consumption in the United States. To reduce electricity usage and peak demand, many utilities are introducing market-based time-of-use (TOU) pricing models. In parallel, government programs that increase the fraction of renewable energy are incentivizing residential consumers to adopt on-site renewables and energy storage. Connecting on-site renewables and energy storage between homes forms a sustainable microgrid capable of generating, storing, and sharing electricity to balance local generation and consumption in residential areas. In this paper, we investigate how to minimize the costs of electricity from a utility for a microgrid under market-based TOU pricing models. In particular, we (i) present a system architecture for an energy-sharing microgrid; and (ii) develop optimal energy-sharing algorithms for homes within the microgrid. We conduct an extensive evaluation under two typical TOU pricing models that use data from more than 40 homes. Our results indicate that our system reduces the costs of Alternating Current (AC) electricity by 20%, even for homes with similar energy usage patterns.

/pdf/minimizing-electricity-costs-by-sharing-energy-in-2wz3htc1zo.pdf

David Irwin

Papers

Networked cloud orchestration: A GENI perspective

How not to bid the cloud

Automated and on-demand provisioning of virtual machines for database applications

Exploiting home automation protocols for load monitoring in smart buildings

Minimizing electricity costs by sharing energy in sustainable microgrids