Data Corruption

About: Data Corruption is a research topic. Over the lifetime, 435 publications have been published within this topic receiving 6784 citations.

Scalable multithreaded network testing tool

Abstract: Embodiments of the present invention generally provide a system and method for testing integrity of data transmitted across a network connection. In one embodiment, the method generally includes creating one or more test threads on one or more test clients attached to a server through the network connection. The method further includes, for each test thread, generating a data load on the data connection by repetitively writing test data patterns to a data file on the server, reading data patterns from the data file, and comparing the data patterns read from the data file to the test data patterns written to the data file to detect data corruptions. The method may further include measuring data throughput between the one or more test clients and the server and generating debug information if a data corruption is detected by one of the test threads.

Data Repair for Distributed, Event-based IoT Applications

Abstract: Motivated by the growth of Internet of Things (IoT) technologies and the volumes and velocity of data that they can and will produce, we investigate automated data repair for event-driven, IoT applications. IoT devices are heterogeneous in their hardware architectures, software, size, cost, capacity, network capabilities, power requirements, etc. They must execute in a wide range of operating environments where failures and degradations of service due to hardware malfunction, software bugs, network partitions, etc. cannot be immediately remediated. Further, many of these failure modes cause corruption in the data that these devices produce and in the computations "downstream" that depend on this data.To "repair" corrupted data from its origin through its computational dependencies in a distributed IoT setting, we explore SANS-SOUCI--a system for automatically tracking causal data dependencies and re-initiating dependent computations in event-driven IoT deployment frameworks. SANS-SOUCI presupposes an event-driven programming model based on cloud functions, which we extend for portable execution across IoT tiers (device, edge, cloud). We add fast, persistent, append-only storage and versioning for efficient data robustness and durability. SANS-SOUCI records events and their causal dependencies using a distributed event log and repairs applications dynamically, across tiers via replay. We evaluate SANS-SOUCI using a portable, open source, distributed IoT platform, example applications, and microbenchmarks. We find that SANS-SOUCI is able to perform repair for both software (function) and sensor produced data corruption with very low overhead.

Corruption checking during backup

Abstract: In one embodiment, a data corruption checker is integrated into the data backup operation. In one embodiment, the data corruption checker can skip over minor corruptions and back up the non-corrupt files, or can abort the backup and locate a major corruption. In one embodiment, the present invention includes receiving a backup command at a storage server, the backup command requesting that the storage server perform a backup operation on a storage component, scanning a set of inodes associated with the storage component to determine a subset of inodes corresponding to files to be backed up, and, as part of the scanning, identifying one or more corrupted files.

Method and system for handling reallocated blocks in a file system

Abstract: A system for managing a file system. In response to a user selecting an image of the file system to perform a rollback operation, each data block in the file system that is owned by the image is searched for. Then, it is determined whether a data block owned by the image is marked in a first map and a second map. In response to determining that the data block owned by the image is marked in the first map and the second map, the data block is marked in a third map. In response to determining that the data block owned by the image is marked in the third map, a new data block is allocated in the file system. Subsequently, data from the data block owned by the image is copied to the new data block in the file system prior to performing the rollback operation to avoid a collision that causes data corruption.

Duplicate address discovery and action

Abstract: A duplicate address discovery process detects duplicate MAC addresses or duplicate unique port identifiers within the network, alerts attached devices of the duplicates, and takes action to avoid data corruption that might be caused by such duplicate addresses.