Topic
Data Corruption
About: Data Corruption is a research topic. Over the lifetime, 435 publications have been published within this topic receiving 6784 citations.
Papers published on a yearly basis
Papers
More filters
Patent•
23 Apr 2014
TL;DR: In this article, a method and a device used for data storage is described, where the processing device determines one or a plurality of corresponding candidate storage scheme information according to data storage requests, and determines corresponding optimized storage scheme according to performance index information corresponding to the candidate storage schemas.
Abstract: The invention aims to provide a method and a device used for data storage The processing device determines one or a plurality of corresponding candidate storage scheme information according to data storage requests, and determines corresponding optimized storage scheme information according to performance index information corresponding to the candidate storage scheme information, so as to process data storage requests Compared with the prior art, according to different hardware medium service qualities, data is distributed to proper storage mediums according to the degree of importance, so that the data corruption and loss probabilities can be reduced; meanwhile, the service quality of the medium is changed in real time according to a practical situation, and potential faults can be earlier found by using the device, so that the damage caused by imminent hardware trouble can be prevented; therefore, a data grading topology which senses the storage medium service qualities can be realized, the influence on storage performance and reliability of the storage system caused by the different storage medium service qualities can be eliminated, and a strategy for controlling the storage cost can be realized
3 citations
Posted Content•
TL;DR: Simulation-based evaluation with seven data-intensive applications shows Tvarak's performance and energy efficiency, including Redis set-only performance by only 3%, compared to 50% reduction for a state-of-the-art software-only approach.
Abstract: Tvarak efficiently implements system-level redundancy for direct-access (DAX) NVM storage. Production storage systems complement device-level ECC (which covers media errors) with system-checksums and cross-device parity. This system-level redundancy enables detection of and recovery from data corruption due to device firmware bugs (e.g., reading data from the wrong physical location). Direct access to NVM penalizes software-only implementations of system-level redundancy, forcing a choice between lack of data protection or significant performance penalties. Offloading the update and verification of system-level redundancy to Tvarak, a hardware controller co-located with the last-level cache, enables efficient protection of data from such bugs in memory controller and NVM DIMM firmware. Simulation-based evaluation with seven data-intensive applications shows Tvarak's performance and energy efficiency. For example, Tvarak reduces Redis set-only performance by only 3%, compared to 50% reduction for a state-of-the-art software-only approach.
3 citations
Patent•
IBM1
TL;DR: In this article, a trusted register can be included into a memory system to allow for immediate access to one piece of trusted data, so that the system can overlap the parity checking and delivery of a location of data with the reading of the next location from the memory array.
Abstract: A method, an apparatus, and a computer program are provided for sequentially determining parity of stored data. Because of the inherent instabilities that exist in most memory arrays, data corruption can be a substantial problem. Parity checking and other techniques are typically employed to counteract the problem. However, with parity checking and other techniques, there are tradeoffs. Time required to perform the parity check, for example, can cause system latencies. Therefore, to reduce latencies, a trusted register can be included into a memory system to allow for immediate access to one piece of trusted data. By being able to read one piece of trusted data, the system can overlap the parity checking and delivery of a location of data with the reading of the next location of data from the memory array. Hence, a full cycle of latency can be eliminated without the reduction of the clock frequency.
3 citations
01 Apr 2013
TL;DR: This paper considers two reliability threats: memory errors, where bits in DRAM are flipped due to cosmic rays, and software bugs, where programming errors may ultimately result in data corruption and crashes, and argues that by making use of checksums, it can significantly reduce the probability that either threat results in any application-visible effects.
Abstract: In this paper, we aim to improve the reliability of a central part of the operating system storage stack: the page cache. We consider two reliability threats: memory errors, where bits in DRAM are flipped due to cosmic rays, and software bugs, where programming errors may ultimately result in data corruption and crashes. We argue that by making use of checksums, we can significantly reduce the probability that either threat results in any application-visible effects. In particular, we can use checksums to detect memory corruption as well as validate the integrity of the cache's internal state for recovery after a crash. We show that in many cases, we can avoid the overhead of computing checksums especially for these purposes. We implement our ideas in the Loris storage stack. Our analysis and evaluation show that our approach improves the overall reliability of the cache at relatively little added cost.
3 citations
Patent•
08 Mar 2015
TL;DR: In this paper, the authors present a distributed memory system where data sets are erasure-coded and the resulting fragments are stored in random access memory modules distributed throughout the system, where read operations include reconstructing data sets from fetched data fragments, and write operations allow conversion of data sets into fragments which are then streamed and distributively stored.
Abstract: Various systems to achieve data resiliency in a shared memory pool are presented. Multiple memory modules are associated with multiple data interfaces, one or multiple erasure-coding interfaces are communicatively connected with the multiple data interfaces, and multiple compute elements are communicatively connected with one or multiple erasure-coding interfaces. Data sets are erasure-coded, and the resulting fragments are stored in random access memory modules distributed throughout the system. Storage in RAM allows real-time fetching of fragments using random-access read cycles and streaming of fragments using random-access write cycles, in which read operations include reconstruction of data sets from fetched data fragments, and write operations allow conversion of data sets into fragments which are then streamed and distributively stored. Distributed memory creates data resiliency to reconstruct original data sets in cases such as data corruption, failure of a memory module, failure of a data interface, or failure of a compute element.
3 citations