Open AccessProceedings Article
TAO: Facebook's distributed data store for the social graph
Nathan Bronson,Zach Amsden,George Cabrera,Prasad Chakka,Peter Dimov,Hui Ding,Jack Ferris,Anthony Giardullo,Sachin Kulkarni,Harry Li,Mark Marchukov,Dmitri Petrov,Lovro Puzar,Yee Jiun Song,Venkat Venkataramani +14 more
- pp 49-60
Reads0
Chats0
TLDR
TAO is a geographically distributed data store that provides efficient and timely access to the social graph for Facebook's demanding workload using a fixed set of queries.Abstract:
We introduce a simple data model and API tailored for serving the social graph, and TAO, an implementation of this model. TAO is a geographically distributed data store that provides efficient and timely access to the social graph for Facebook's demanding workload using a fixed set of queries. It is deployed at Facebook, replacing memcache for many data types that fit its model. The system runs on thousands of machines, is widely distributed, and provides access to many petabytes of data. TAO can process a billion reads and millions of writes each second.read more
Citations
More filters
Proceedings ArticleDOI
Inside the Social Network's (Datacenter) Network
TL;DR: The contrasting locality, stability, and predictability of network traffic in Facebook's datacenters are reported on, and their implications for network architecture, traffic engineering, and switch design are commented on.
Proceedings ArticleDOI
FaRM: fast remote memory
TL;DR: The design and implementation of FaRM is described, a new main memory distributed computing platform that exploits RDMA to improve both latency and throughput by an order of magnitude relative to state of the art main memory systems that use TCP/IP.
Proceedings ArticleDOI
f4: Facebook's warm BLOB storage system
Subramanian Muralidhar,Wyatt Lloyd,Sabyasachi Roy,Cory Hill,Ernest Lin,Weiwen Liu,Satadru Pan,Shiva Shankar,Viswanath Sivakumar,Linpeng Tang,Sanjeev Kumar +10 more
TL;DR: Facebook's corpus of photos, videos, and other Binary Large OBjects (BLOBs) that need to be reliably stored and quickly accessible is massive and continues to grow, as the footprint of BLOBs increases, storing them in the traditional storage system, Haystack, is becoming increasingly inefficient.
Proceedings ArticleDOI
LinkBench: a database benchmark based on the Facebook social graph
TL;DR: LinkBench provides a realistic and challenging test for persistent storage of social and web service data, filling a gap in the available tools for researchers, developers and administrators.
Journal ArticleDOI
PowerLyra: Differentiated Graph Computation and Partitioning on Skewed Graphs
TL;DR: It is argued that skewed distributions in natural graphs also necessitate differentiated processing on high-degree and low-degree vertices, and PowerLyra, a new distributed graph processing system that embraces the best of both worlds of existing graph-parallel systems is introduced.
References
More filters
Proceedings ArticleDOI
Chord: A scalable peer-to-peer lookup service for internet applications
TL;DR: Results from theoretical analysis, simulations, and experiments show that Chord is scalable, with communication cost and the state maintained by each node scaling logarithmically with the number of Chord nodes.
Book ChapterDOI
Pastry: Scalable, Decentralized Object Location, and Routing for Large-Scale Peer-to-Peer Systems
Antony Rowstron,Peter Druschel +1 more
TL;DR: Pastry as mentioned in this paper is a scalable, distributed object location and routing substrate for wide-area peer-to-peer ap- plications, which performs application-level routing and object location in a po- tentially very large overlay network of nodes connected via the Internet.
Proceedings ArticleDOI
A scalable content-addressable network
TL;DR: The concept of a Content-Addressable Network (CAN) as a distributed infrastructure that provides hash table-like functionality on Internet-like scales is introduced and its scalability, robustness and low-latency properties are demonstrated through simulation.
Proceedings Article
Bigtable: A Distributed Storage System for Structured Data (Awarded Best Paper!).
Fay W. Chang,Jeffrey Dean,Sanjay Ghemawat,Wilson C. Hsieh,Deborah A. Wallach,Michael Burrows,Tushar Deepak Chandra,Andrew Fikes,Robert Gruber +8 more
TL;DR: Bigtable as mentioned in this paper is a distributed storage system for managing structured data that is designed to scale to a very large size: petabytes of data across thousands of commodity servers, including web indexing, Google Earth and Google Finance.
Proceedings ArticleDOI
Dynamo: amazon's highly available key-value store
Giuseppe deCandia,Deniz Hastorun,Madan Mohan Rao Jampani,Gunavardhan Kakulapati,Avinash Lakshman,Alex Pilchin,Swaminathan Sivasubramanian,Peter Sven Vosshall,Werner Vogels +8 more
TL;DR: D Dynamo is presented, a highly available key-value storage system that some of Amazon's core services use to provide an "always-on" experience and makes extensive use of object versioning and application-assisted conflict resolution in a manner that provides a novel interface for developers to use.