Proceedings ArticleDOI
Windows Azure Storage: a highly available cloud storage service with strong consistency
Brad Calder,Ju Wang,Aaron W. Ogus,Niranjan Nilakantan,Arild E. Skjolsvold,Sam McKelvie,Yikang Xu,Shashwat Srivastav,Jiesheng Wu,Huseyin Simitci,Jaidev Haridas,Chakravarthy Uddaraju,Hemal Khatri,Andrew James Edwards,Vaman Bedekar,Mainali Shane Kumar,Rafay Abbasi,Arpit Agarwal,Mian Fahim ul Haq,Muhammad Ikram ul Haq,Deepali Bhardwaj,Sowmya Dayanand,Anitha Adusumilli,Marvin McNett,Sriram Sankaran,Kavitha Manivannan,Leonidas Rigas +26 more
- pp 143-157
Reads0
Chats0
TLDR
The WAS architecture, global namespace, and data model is described, as well as its resource provisioning, load balancing, and replication systems.Abstract:
Windows Azure Storage (WAS) is a cloud storage system that provides customers the ability to store seemingly limitless amounts of data for any duration of time. WAS customers have access to their data from anywhere at any time and only pay for what they use and store. In WAS, data is stored durably using both local and geographic replication to facilitate disaster recovery. Currently, WAS storage comes in the form of Blobs (files), Tables (structured storage), and Queues (message delivery). In this paper, we describe the WAS architecture, global namespace, and data model, as well as its resource provisioning, load balancing, and replication systems.read more
Citations
More filters
Proceedings Article
Erasure coding in windows azure storage
Cheng Huang,Huseyin Simitci,Yikang Xu,Aaron W. Ogus,Brad Calder,Parikshit Gopalan,Jin Li,Sergey Yekhanin +7 more
TL;DR: This paper describes how LRC is used in WAS to provide low overhead durable storage with consistently low read latencies, and introduces a new set of codes for erasure coding called Local Reconstruction Codes (LRC).
Proceedings ArticleDOI
Naiad: a timely dataflow system
TL;DR: It is shown that many powerful high-level programming models can be built on Naiad's low-level primitives, enabling such diverse tasks as streaming data analysis, iterative machine learning, and interactive graph mining.
Journal ArticleDOI
Big Data computing and clouds
Marcos Dias De Assuncao,Rodrigo N. Calheiros,Silvia Cristina Sardela Bianchi,Marco A. S. Netto,Rajkumar Buyya +4 more
TL;DR: This paper discusses approaches and environments for carrying out analytics on Clouds for Big Data applications, and identifies possible gaps in technology and provides recommendations for the research community on future directions on Cloud-supported Big Data computing and analytics solutions.
Journal ArticleDOI
XORing elephants: novel erasure codes for big data
Maheswaran Sathiamoorthy,Megasthenis Asteris,Dimitris S. Papailiopoulos,Alexandros G. Dimakis,Ramkumar Vadali,Scott Chen,Dhruba Borthakur +6 more
TL;DR: In this article, the authors present a family of erasure codes that are efficient repairable and offer higher reliability compared to Reed-Solomon codes, which is the standard design choice and their high repair cost is often considered an unavoidable price to pay for high storage efficiency and high reliability.
Proceedings ArticleDOI
Paragon: QoS-aware scheduling for heterogeneous datacenters
TL;DR: Paragon is an online and scalable DC scheduler that is heterogeneity and interference-aware, derived from robust analytical methods and uses collaborative filtering techniques to quickly and accurately classify an unknown, incoming workload, by identifying similarities to previously scheduled applications.
References
More filters
Journal ArticleDOI
The Google file system
TL;DR: This paper presents file system interface extensions designed to support distributed applications, discusses many aspects of the design, and reports measurements from both micro-benchmarks and real world use.
Proceedings Article
Bigtable: A Distributed Storage System for Structured Data (Awarded Best Paper!).
Fay W. Chang,Jeffrey Dean,Sanjay Ghemawat,Wilson C. Hsieh,Deborah A. Wallach,Michael Burrows,Tushar Deepak Chandra,Andrew Fikes,Robert Gruber +8 more
TL;DR: Bigtable as mentioned in this paper is a distributed storage system for managing structured data that is designed to scale to a very large size: petabytes of data across thousands of commodity servers, including web indexing, Google Earth and Google Finance.
Proceedings ArticleDOI
Dynamo: amazon's highly available key-value store
Giuseppe deCandia,Deniz Hastorun,Madan Mohan Rao Jampani,Gunavardhan Kakulapati,Avinash Lakshman,Alex Pilchin,Swaminathan Sivasubramanian,Peter Sven Vosshall,Werner Vogels +8 more
TL;DR: D Dynamo is presented, a highly available key-value storage system that some of Amazon's core services use to provide an "always-on" experience and makes extensive use of object versioning and application-assisted conflict resolution in a manner that provides a novel interface for developers to use.
Journal ArticleDOI
Bigtable: A Distributed Storage System for Structured Data
Fay W. Chang,Jeffrey Dean,Sanjay Ghemawat,Wilson C. Hsieh,Deborah A. Wallach,Michael Burrows,Tushar Deepak Chandra,Andrew Fikes,Robert E. Gruber +8 more
TL;DR: The simple data model provided by Bigtable is described, which gives clients dynamic control over data layout and format, and the design and implementation of Bigtable are described.
Journal ArticleDOI
The part-time parliament
TL;DR: The Paxon parliament's protocol provides a new way of implementing the state machine approach to the design of distributed systems.