Proceedings ArticleDOI
Dynamo: amazon's highly available key-value store
Giuseppe deCandia,Deniz Hastorun,Madan Mohan Rao Jampani,Gunavardhan Kakulapati,Avinash Lakshman,Alex Pilchin,Swaminathan Sivasubramanian,Peter Sven Vosshall,Werner Vogels +8 more
- Vol. 41, Iss: 6, pp 205-220
TLDR
D Dynamo is presented, a highly available key-value storage system that some of Amazon's core services use to provide an "always-on" experience and makes extensive use of object versioning and application-assisted conflict resolution in a manner that provides a novel interface for developers to use.Abstract:
Reliability at massive scale is one of the biggest challenges we face at Amazon.com, one of the largest e-commerce operations in the world; even the slightest outage has significant financial consequences and impacts customer trust. The Amazon.com platform, which provides services for many web sites worldwide, is implemented on top of an infrastructure of tens of thousands of servers and network components located in many datacenters around the world. At this scale, small and large components fail continuously and the way persistent state is managed in the face of these failures drives the reliability and scalability of the software systems.This paper presents the design and implementation of Dynamo, a highly available key-value storage system that some of Amazon's core services use to provide an "always-on" experience. To achieve this level of availability, Dynamo sacrifices consistency under certain failure scenarios. It makes extensive use of object versioning and application-assisted conflict resolution in a manner that provides a novel interface for developers to use.read more
Citations
More filters
Journal Article
Above the Clouds: A Berkeley View of Cloud Computing
Michael Armbrust,Armando Fox,Rean Griffith,Anthony D. Joseph,Randy H. Katz,Andy Konwinski,Gunho Lee,David A. Patterson,Ariel Rabkin,Ion Stoica,Matei Zaharia +10 more
TL;DR: This work focuses on SaaS Providers (Cloud Users) and Cloud Providers, which have received less attention than SAAS Users, and uses the term Private Cloud to refer to internal datacenters of a business or other organization, not made available to the general public.
Journal ArticleDOI
A scalable, commodity data center network architecture
TL;DR: This paper shows how to leverage largely commodity Ethernet switches to support the full aggregate bandwidth of clusters consisting of tens of thousands of elements and argues that appropriately architected and interconnected commodity switches may deliver more performance at less cost than available from today's higher-end solutions.
Proceedings ArticleDOI
Benchmarking cloud serving systems with YCSB
TL;DR: This work presents the "Yahoo! Cloud Serving Benchmark" (YCSB) framework, with the goal of facilitating performance comparisons of the new generation of cloud data serving systems, and defines a core set of benchmarks and reports results for four widely used systems.
Journal ArticleDOI
Cassandra: a decentralized structured storage system
Avinash Lakshman,Prashant Malik +1 more
TL;DR: Cassandra is a distributed storage system for managing very large amounts of structured data spread out across many commodity servers, while providing highly available service with no single point of failure.
Journal ArticleDOI
Big Data: A Survey
Min Chen,Shiwen Mao,Yunhao Liu +2 more
TL;DR: The background and state-of-the-art of big data are reviewed, including enterprise management, Internet of Things, online social networks, medial applications, collective intelligence, and smart grid, as well as related technologies.
References
More filters
Proceedings ArticleDOI
Storage management and caching in PAST, a large-scale, persistent peer-to-peer storage utility
Antony Rowstron,Peter Druschel +1 more
TL;DR: PAST as mentioned in this paper is a large-scale P2P persistent storage utility based on a self-organizing, Internet-based overlay network of storage nodes that cooperatively route file queries, store multiple replicas of files, and cache additional copies of popular files.
Proceedings ArticleDOI
The dangers of replication and a solution
TL;DR: In this article, a two-tier replication algorithm is proposed that allows mobile (disconnected) applications to propose tentative update transactions that are later applied to a master copy to avoid the instability of other replication schemes.
Journal ArticleDOI
A Majority consensus approach to concurrency control for multiple copy databases
TL;DR: A “majority consensus” algorithm which represents a new solution to the update synchronization problem for multiple copy databases is presented and can function effectively in the presence of communication and database site outages.
Proceedings ArticleDOI
Managing update conflicts in Bayou, a weakly connected replicated storage system
TL;DR: Bayou as discussed by the authors is a replicated, weakly consistent storage system designed for a mobile computing environment that includes portable machines with less than ideal network connectivity, and it includes novel methods for conflict detection, called dependency checks, and per-write conflict resolution based on client-provid ed merge procedures.
Journal ArticleDOI
Farsite: federated, available, and reliable storage for an incompletely trusted environment
Atul Adya,William J. Bolosky,Miguel Castro,Cermak Gerald F,Ronnie Chaiken,John R. Douceur,Jon Howell,Jacob R. Lorch,Marvin M. Theimer,Roger Wattenhofer +9 more
TL;DR: The design of Farsite is reported on and the lessons learned by implementing much of that design are reported, including how to locally caching file data, lazily propagating file updates, and varying the duration and granularity of content leases.