Proceedings ArticleDOI
PErasure: A parallel Cauchy Reed-Solomon coding library for GPUs
Xiaowen Chu,Chengjian Liu,Kai Ouyang,Ling Sing Yung,Hai Liu,Yiu-Wing Leung +5 more
- pp 436-441
Reads0
Chats0
TLDR
PErasure, a parallel Cauchy Reed-Solomon coding library, is designed and implemented and can achieve up to 10GB/s of overall encoding speed using just a single GPU for a large storage system that can withstand up to 8 disk failures.Abstract:
In recent years, erasure coding has been adopted by large-scale cloud storage systems to replace data replication. With the increase of disk I/O throughput and network bandwidth, the speed of erasure coding becomes one of the key system bottlenecks. In this paper, we propose to offload the task of erasure coding to Graphics Processing Units (GPUs). Specifically, we have designed and implemented PErasure, a parallel Cauchy Reed-Solomon (CRS) coding library. We compare the performance of PErasure with that of two state-of-the-art libraries: Jerasure (for CPUs) and Gibraltar (for GPUs). Our experiments show that the raw coding speed of PErasure on a $500 Nvidia GTX780 card is about 10 times faster than that of multithreaded Jerasure on a quad-core modern CPU, and 2-4 times faster than Gibraltar on the same GPU. PErasure can achieve up to 10GB/s of overall encoding speed using just a single GPU for a large storage system that can withstand up to 8 disk failures.read more
Citations
More filters
Proceedings ArticleDOI
Partial-parallel-repair (PPR): a distributed technique for repairing erasure coded storage
TL;DR: This paper proposes a novel distributed reconstruction technique, called Partial Parallel Repair (PPR), which divides the reconstruction operation to small partial operations and schedules them on multiple nodes already involved in the data reconstruction, and reduces repair time and degraded read time significantly.
Posted Content
A Survey and Measurement Study of GPU DVFS on Energy Conservation
TL;DR: The state-of-the-art GPU DVFS characterizations are surveyed, the recent research works on GPU power and performance models are summarized, and the experimental results suggest thatGPU DVFS has significant potential for energy saving.
Journal ArticleDOI
A survey and measurement study of GPU DVFS on energy conservation
TL;DR: In this paper, the impact of GPU DVFS on application performance and power consumption, and furthermore, on energy conservation is explored, and the effect of scaling core voltage and memory voltage depends on not only the GPU architectures, but also the characteristic of GPU applications.
Journal ArticleDOI
A Black-Box Fork-Join Latency Prediction Model for Data-Intensive Applications
TL;DR: A black-box Fork-Join model is proposed that covers a wide range of Fork- join structures for the prediction of tail and mean latency, called ForkTail and ForkMean, respectively, and can be used as a powerful tool to aid the design of tail-and-mean-latency guaranteed job scheduling and resource provisioning, especially at high load, for datacenter applications.
Journal ArticleDOI
G-CRS: GPU Accelerated Cauchy Reed-Solomon Coding
TL;DR: To maximize the coding performance of G-CRS, a set of optimization strategies, such as a compact structure to store the bitmatrix in GPU constant memory, efficient data access through shared memory, and decoding parallelism, are designed and implemented to fully utilize the GPU resources.
References
More filters
Journal ArticleDOI
The Google file system
TL;DR: This paper presents file system interface extensions designed to support distributed applications, discusses many aspects of the design, and reports measurements from both micro-benchmarks and real world use.
Proceedings Article
Erasure coding in windows azure storage
Cheng Huang,Huseyin Simitci,Yikang Xu,Aaron W. Ogus,Brad Calder,Parikshit Gopalan,Jin Li,Sergey Yekhanin +7 more
TL;DR: This paper describes how LRC is used in WAS to provide low overhead durable storage with consistently low read latencies, and introduces a new set of codes for erasure coding called Local Reconstruction Codes (LRC).
Proceedings ArticleDOI
Windows Azure Storage: a highly available cloud storage service with strong consistency
Brad Calder,Ju Wang,Aaron W. Ogus,Niranjan Nilakantan,Arild E. Skjolsvold,Sam McKelvie,Yikang Xu,Shashwat Srivastav,Jiesheng Wu,Huseyin Simitci,Jaidev Haridas,Chakravarthy Uddaraju,Hemal Khatri,Andrew James Edwards,Vaman Bedekar,Mainali Shane Kumar,Rafay Abbasi,Arpit Agarwal,Mian Fahim ul Haq,Muhammad Ikram ul Haq,Deepali Bhardwaj,Sowmya Dayanand,Anitha Adusumilli,Marvin McNett,Sriram Sankaran,Kavitha Manivannan,Leonidas Rigas +26 more
TL;DR: The WAS architecture, global namespace, and data model is described, as well as its resource provisioning, load balancing, and replication systems.
Journal ArticleDOI
XORing elephants: novel erasure codes for big data
Maheswaran Sathiamoorthy,Megasthenis Asteris,Dimitris S. Papailiopoulos,Alexandros G. Dimakis,Ramkumar Vadali,Scott Chen,Dhruba Borthakur +6 more
TL;DR: In this article, the authors present a family of erasure codes that are efficient repairable and offer higher reliability compared to Reed-Solomon codes, which is the standard design choice and their high repair cost is often considered an unavoidable price to pay for high storage efficiency and high reliability.
An XOR-based erasure-resilient coding scheme
TL;DR: A erasure-resilient coding scheme that is based on a version of Reed-Solomon codes and which has the property that r = m is described, customized to give the rst real-time implementations of Priority Encoding Transmission (PET) for medium quality video transmission on Sun SPARCstation 20 workstations.
Related Papers (5)
Optimizing Cauchy Reed-Solomon Codes for Fault-Tolerant Network Storage Applications
James S. Plank,Lihao Xu +1 more