scispace - formally typeset
Search or ask a question
Topic

Data Corruption

About: Data Corruption is a research topic. Over the lifetime, 435 publications have been published within this topic receiving 6784 citations.


Papers
More filters
Patent
14 Feb 2008
TL;DR: In this article, a data management device extracts ID numbers of all pages to generate memory management information, manages the storage position of data identified by each ID number, and determines a writable area for writing new data by specifying continuous pages with ID numbers showing non-use.
Abstract: PROBLEM TO BE SOLVED: To efficiently store data and backup data using a memory while improving the memory corruption resistance of a NAND flash memory. SOLUTION: When data is stored in the NAND flash memory 11, the data is stored by using each page in order. An identification number (ID number) is assigned to each data to be stored, and the identification number is inserted to the header of each data. The data management device extracts ID numbers of all pages to generate memory management information, manages the storage position of data identified by each ID number, and determines a writable area for writing new data by specifying continuous pages with ID numbers showing non-use. When data corruption is recognized by checksum of the headers, the pages are retroactively checked to find data of the same ID number stored previously, and data is restored by use of this data. COPYRIGHT: (C)2008,JPO&INPIT

10 citations

Patent
David Allen1
30 Mar 2007
TL;DR: In this paper, the point-in-time application data is accessible by an application prior to completion of the restoration, and particular data may be restored to the original logical volume or to the new logical volume prior to an operation to access the particular data.
Abstract: Providing protection of point-in-time application data includes obtaining a plurality of original snapshot copies of an original logical volume containing the application data, causing the original snapshot copies to be tested for data corruption, and restoring data corresponding to one of the original snapshot copies. Application data is accessible by an application prior to completion of the restoration. One of the original snapshot copies may be restored to the original logical volume and/or to a new logical volume different from the original logical volume. Prior to completion of the restoration, particular data may be restored to the original logical volume or to the new logical volume prior to an operation to access the particular data. The original snapshot copies may be provided as virtual devices.

10 citations

Proceedings ArticleDOI
17 Feb 2014
TL;DR: This work shows that a runtime checker must enforce the atomicity and durability properties of the file system on every write, in addition to checking transactions at commit time, to provide the strong guarantee that every block write will maintain file system consistency.
Abstract: Data corruption is the most common consequence of file-system bugs, as shown by a recent study. When such corruption occurs, the file system's offline check and recovery tools need to be used, but they are error prone and cause significant downtime. Previous work has shown that a runtime checker for the Ext3 journaling file system can verify that metadata updates within a transaction are mutually consistent, helping detect corruption in metadata blocks at commit time. However, corruption can still be caused when a bug in the file system's transactional mechanism loses, misdirects, or corrupts writes. We show that a runtime checker needs to enforce the atomicity and durability properties of the file system on every write, in addition to checking transactions at commit time, to provide the strong guarantee that every block write will maintain file system consistency.In this paper, we identify the invariants that need to be enforced on journaling and shadow paging file systems to preserve the integrity of committed transactions. We also describe the key properties that make it feasible to check these invariants for a file system. Based on this characterization, we have implemented runtime checkers for a modified version of the Ext3 file system and for the Btrfs file system. Our evaluation shows that both checkers detect data corruption effectively, and they can be used during normal operation with low overhead.

10 citations

Patent
Russell J. Henry1
18 Aug 2003
TL;DR: In this paper, a virtual end-to-end address is established, which is associated with the data integrity field, wherein the virtual end to end address transfers encoded information to a controller of the computer through one or more addresses of a read and/or write request thereof.
Abstract: Methods and systems for end-to-end data protection in a computer are disclosed. A data integrity field is generally associated with data transferred along a data path in a computer. A virtual end-to-end address can be established, which is associated with the data integrity field, wherein the virtual end-to-end address transfers encoded information to a controller of the computer through one or more addresses of a read and/or write request thereof. The encoded information can be utilized to identify an offending entity within the data path. An end-to-end access list can also be associated with the virtual end-to-end address, such that the end-to-end access list contains at least one entry for every data transfer request provided to an interface device. Data corruption is therefore preventable in the entire I/O path in a computer from a host bus adapter through to the actual physical drive.

10 citations

Proceedings ArticleDOI
28 Jul 2019
TL;DR: In this article, the authors explore an approach of assuring data integrity - considering either malicious or accidental corruption - for workflow executions orchestrated by the Pegasus Workflow Management System, and introduce Chaos Jungle, a toolkit providing an environment for validating integrity verification mechanisms by allowing researchers to introduce a variety of integrity errors during data transfers and storage.
Abstract: With the continued rise of scientific computing and the enormous increases in the size of data being processed, scientists must consider whether the processes for transmitting and storing data sufficiently assure the integrity of the scientific data. When integrity is not preserved, computations can fail and result in increased computational cost due to reruns, or worse, results can be corrupted in a manner not apparent to the scientist and produce invalid science results. Technologies such as TCP checksums, encrypted transfers, checksum validation, RAID and erasure coding provide integrity assurances at different levels, but they may not scale to large data sizes and may not cover a workflow from end-to-end, leaving gaps in which data corruption can occur undetected. In this paper we explore an approach of assuring data integrity - considering either malicious or accidental corruption - for workflow executions orchestrated by the Pegasus Workflow Management System. To validate our approach, we introduce Chaos Jungle - a toolkit providing an environment for validating integrity verification mechanisms by allowing researchers to introduce a variety of integrity errors during data transfers and storage. In addition to controlled experiments with Chaos Jungle, we provide analysis of integrity errors that we encountered when running production workflows.

10 citations


Network Information
Related Topics (5)
Network packet
159.7K papers, 2.2M citations
82% related
Software
130.5K papers, 2M citations
81% related
Wireless sensor network
142K papers, 2.4M citations
78% related
Wireless network
122.5K papers, 2.1M citations
77% related
Cluster analysis
146.5K papers, 2.9M citations
76% related
Performance
Metrics
No. of papers in the topic in previous years
YearPapers
20221
202121
202025
201927
201827
201727