/pdf/a-break-in-the-clouds-towards-a-cloud-definition-svovsw1r5a.pdf

A Break in the Clouds: Towards a Cloud Definition

The Grid 2: Blueprint for a New Computing Infrastructure

Google's Borg system is a cluster manager that runs hundreds of thousands of jobs, from many thousands of different applications, across a number of clusters each with up to tens of thousands of machines. It achieves high utilization by combining admission control, efficient task-packing, over-commitment, and machine sharing with process-level performance isolation. It supports high-availability applications with runtime features that minimize fault-recovery time, and scheduling policies that reduce the probability of correlated failures. Borg simplifies life for its users by offering a declarative job specification language, name service integration, real-time job monitoring, and tools to analyze and simulate system behavior. We present a summary of the Borg system architecture and features, important design decisions, a quantitative analysis of some of its policy decisions, and a qualitative examination of lessons learned from a decade of operational experience with it.

/pdf/large-scale-cluster-management-at-google-with-borg-4i7d9qnw5x.pdf

Large-scale cluster management at Google with Borg

To better understand the challenges in developing effective cloud-based resource schedulers, we analyze the first publicly available trace data from a sizable multi-purpose cluster. The most notable workload characteristic is heterogeneity: in resource types (e.g., cores:RAM per machine) and their usage (e.g., duration and resources needed). Such heterogeneity reduces the effectiveness of traditional slot- and core-based scheduling. Furthermore, some tasks are constrained as to the kind of machine types they can use, increasing the complexity of resource assignment and complicating task migration. The workload is also highly dynamic, varying over time and most workload features, and is driven by many short jobs that demand quick scheduling decisions. While few simplifying assumptions apply, we find that many longer-running jobs have relatively stable resource utilizations, which can help adaptive resource schedulers.

http://www.istc-cc.cmu.edu/publications/papers/2012/googletrace-socc2012.pdf

Heterogeneity and dynamicity of clouds at scale: Google trace analysis

Network-based cloud computing is rapidly expanding as an alternative to conventional office-based computing. As cloud computing becomes more widespread, the energy consumption of the network and computing resources that underpin the cloud will grow. This is happening at a time when there is increasing attention being paid to the need to manage energy consumption across the entire information and communications technology (ICT) sector. While data center energy use has received much attention recently, there has been less attention paid to the energy consumption of the transmission and switching networks that are key to connecting users to the cloud. In this paper, we present an analysis of energy consumption in cloud computing. The analysis considers both public and private clouds, and includes energy consumption in switching and transmission as well as data processing and data storage. We show that energy consumption in transport and switching can be a significant percentage of total energy consumption in cloud computing. Cloud computing can enable more energy-efficient use of computing power, especially when the computing tasks are of low intensity or infrequent. However, under some circum- stances cloud computing can consume more energy than conventional computing where each user performs all com- puting on their own personal computer (PC).

/pdf/green-cloud-computing-balancing-energy-in-processing-storage-2z2x5rjq9b.pdf

Green Cloud Computing: Balancing Energy in Processing, Storage, and Transport For processing large amounts of data, management and switching of communications may contribute significantly to energy consumption and cloud computing seems to be an alternative to office-based computing.

This document provides the second deliverable of EDGI JRA2. It is produced by the INRIA team, the SZTAKI team, the LAL/IN2P3 team and the University of Coimbra team. This document aims at describing achievements and results of JRA2 tasks "Advanced QoS Scheduler and Oracle" and "Support In Science Gateway". Hybrid Distributed Computing Infrastructures (DCIs) allow users to combine Grids, Desktop Grids, Clouds, etc. to obtain for their users large computing capabilities. The EDGI infrastructure belongs to this kind of DCIs. The document presents the SpeQuloS framework to provide quality of service (QoS) for application executed on the EDGI infrastructure. It also introduces EDGI QoS portal, an user-friendly and integrated access to QoS features for users of EDGI infrastructure. In this document, we first introduce new results from JRA2.1 task, which collected and analyzed batch execution on Desktop Grid. Then, we present the advanced Cloud Scheduling and Oracle strategies designed inside the SpeQuloS framework (task JRA2.2). We demonstrate efficiency of these strategies using performance evaluation carried out with simulations. Next, we introduce Credit System architecture and QoS user portal as part of the JRA2 Support In Science Gateway (task JRA2.3). Finally, we conclude and provide references to JRA2 production.

https://hal.inria.fr/hal-00819907/document

Advanced QoS Prototype for the EDGI Infrastructure

Cloud Computing is certainly one of the hottest subjects these days for people in academia, industry and businesses. At stake are the fascinating potentials and capabilities cloud technologies can provide. This panel will assess the state of the art in Cloud Computing, the main issues and challenges facing the community, differences and similarities with other concepts and systems, and address the suitability of current cloud platforms for HPC applications. The panel will address gaps, recommendations, and future needs and directions.

Derrick Kondo

Papers

Advanced QoS Prototype for the EDGI Infrastructure

Cloud Computing: Opportunities and Challenges for HPC Applications

Editorial: Preface to the special issue on volunteer computing and desktop grids