Data Corruption

Abstract: Energy has become a first-class design constraint in computer systems. Memory is a significant contributor to total system power. This paper introduces Flikker, an application-level technique to reduce refresh power in DRAM memories. Flikker enables developers to specify critical and non-critical data in programs and the runtime system allocates this data in separate parts of memory. The portion of memory containing critical data is refreshed at the regular refresh-rate, while the portion containing non-critical data is refreshed at substantially lower rates. This partitioning saves energy at the cost of a modest increase in data corruption in the non-critical data. Flikker thus exposes and leverages an interesting trade-off between energy consumption and hardware correctness. We show that many applications are naturally tolerant to errors in the non-critical data, and in the vast majority of cases, the errors have little or no impact on the application's final outcome. We also find that Flikker can save between 20-25% of the power consumed by the memory sub-system in a mobile device, with negligible impact on application performance. Flikker is implemented almost entirely in software, and requires only modest changes to the hardware.

Abstract: An important threat to reliable storage of data is silent data corruption. In order to develop suitable protection mechanisms against data corruption, it is essential to understand its characteristics. In this article, we present the first large-scale study of data corruption. We analyze corruption instances recorded in production storage systems containing a total of 1.53 million disk drives, over a period of 41 months. We study three classes of corruption: checksum mismatches, identity discrepancies, and parity inconsistencies. We focus on checksum mismatches since they occur the most.We find more than 400,000 instances of checksum mismatches over the 41-month period. We find many interesting trends among these instances, including: (i) nearline disks (and their adapters) develop checksum mismatches an order of magnitude more often than enterprise-class disk drives, (ii) checksum mismatches within the same disk are not independent events and they show high spatial and temporal locality, and (iii) checksum mismatches across different disks in the same storage system are not independent. We use our observations to derive lessons for corruption-proof system design.

Abstract: The quality of data stored in a memory system is assessed by different methods, and the memory system is operated according to the assessed quality. The data quality can be assessed during read operations. Subsequent use of an Error Correction Code can utilize the quality indications to detect and reconstruct the data with improved effectiveness. Alternatively, a statistics of data quality can be constructed and digital data values can be associated in a modified manner to prevent data corruption. In both cases the corrective actions can be implemented specifically on the poor quality data, according to suitably chosen schedules, and with improved effectiveness because of the knowledge provided by the quality indications. These methods can be especially useful in high-density memory systems constructed of multi-level storage memory cells.

Abstract: The present disclosure describes a unique way for each of multiple processes to operate in parallel using (e.g., reading, modifying, and writing to) the same shared data without causing corruption to the shared data. For example, each of multiple processes utilizes current and past data values associated with a global counter or clock for purposes of determining whether any shared variables used to produce a respective transaction outcome were modified (by another process) when executing a respective transaction. If a respective process detects that shared data used by respective process was modified during a transaction, the process can abort and retry the transaction rather than cause data corruption by storing locally maintained results associated with the transaction to a globally shared data space.

Abstract: Most commercial database systems provide a recovery mechanism that is used to restore data integrity in the event of a hardware or software failure. Many of these systems restore the database from the most recent backup file by rolling forward all transactions from the most recent undamaged transaction log. The present invention discloses a novel system and method for recovering data from user or application errors. Embodiments of the present invention selectively target and undo only those transactions that caused data corruption. In this way, the present invention is able to recover fine-grained database objects such as a table or a row within a table. The present invention has the further advantage of minimizing the number of transactions that are lost after recovery.