Data Corruption

About: Data Corruption is a research topic. Over the lifetime, 435 publications have been published within this topic receiving 6784 citations.

Protecting database records against a failed write operation

Abstract: Described are techniques for processing data requests in connection with an I/O operation. A write data request is sent from a host to a target data storage system. The host performs a data validation, such as a checksum calculation, using the data of the data request. The data request is sent to the target data storage system. The target data storage system may be enabled to perform data validation processing on a per device basis by setting one or more device flag bits for a device. The target data storage system performs data validation processing in accordance with the flag bit settings of a device associated with a data request. A target checksum value using the data received on the target data storage system is determined and compared to the host checksum value in order to determined data validity. Data recovery processing is performed if data corruption is determined. A partial write of the data requested may be performed for those data portions occurring prior to the corrupted data that have been successfully validated.

An FPGA-Based High-Speed Error Resilient Data Aggregation and Control for High Energy Physics Experiment

Abstract: Due to the dramatic increase of data volume in modern high energy physics (HEP) experiments, a robust high-speed data acquisition (DAQ) system is very much needed to gather the data generated during different nuclear interactions. As the DAQ works under harsh radiation environment, there is a fair chance of data corruption due to various energetic particles like alpha, beta, or neutron. Hence, a major challenge in the development of DAQ in the HEP experiment is to establish an error resilient communication system between front-end sensors or detectors and back-end data processing computing nodes. Here, we have implemented the DAQ using field-programmable gate array (FPGA) due to some of its inherent advantages over the application-specific integrated circuit. A novel orthogonal concatenated code and cyclic redundancy check (CRC) have been used to mitigate the effects of data corruption in the user data. Scrubbing with a 32-b CRC has been used against error in the configuration memory of FPGA. Data from front-end sensors will reach to the back-end processing nodes through multiple stages that may add an uncertain amount of delay to the different data packets. We have also proposed a novel memory management algorithm that helps to process the data at the back-end computing nodes removing the added path delays. To the best of our knowledge, the proposed FPGA-based DAQ utilizing optical link with channel coding and efficient memory management modules can be considered as first of its kind. Performance estimation of the implemented DAQ system is done based on resource utilization, bit error rate, efficiency, and robustness to radiation.

Method and apparatus for selective and power-aware memory error protection and memory management

Abstract: A method for providing selective memory error protection responsive to a predictable failure notification associated with at least one portion of a memory in a computing system includes: obtaining an active error correcting code (ECC) configuration corresponding to the portion of the memory; determining whether the active ECC configuration is sufficient to correct at least one error in the portion of the memory affected by the predictable failure notification; when the active ECC configuration is insufficient to correct the error, determining whether data corruption can be tolerated by an application running on the computing system; when data corruption cannot be tolerated by the application, determining whether a stronger ECC level is available and, if a stronger ECC level is available, increasing a strength of the active ECC configuration; and when data corruption can be tolerated, performing page reassignment and aggregation of non-critical data.

Programming non-volatile storage system with multiple memory die

Abstract: A system and methods for programming a set of data onto non-volatile memory elements, maintaining copies of the data pages to be programmed, as well as surrounding data pages, internally or externally to the memory circuit, verifying programming correctness after programming, and upon discovering programming error, recovering the safe copies of the corrupted data to be reprogrammed in alternative non-volatile memory elements. Additionally, a system and methods for programming one or more sets of data across multiple die of a non-volatile memory system, combining data pages across the multiple die by means such as the XOR operation prior to programming the one or more sets of data, employing various methods to determine the correctness of programming, and upon identifying data corruption, recovering safe copies of data pages by means such as XOR operation to reprogram the pages in an alternate location on the non-volatile memory system.

Smart Home Sensor Anomaly Detection Using Convolutional Autoencoder Neural Network

Abstract: We propose an autoencoder based approach to anomaly detection in smart grid systems. Data collecting sensors within smart home systems are susceptible to many data corruption issues, such as malicious attacks or physical malfunctions. By applying machine learning to a smart home or grid, sensor anomalies can be detected automatically for secure data collection and sensor-based system functionality. In addition, we tested the effectiveness of this approach on real smart home sensor data collected for multiple years. An early detection of such data corruption issues is essential to the security and functionality of the various sensors and devices within a smart home.