Showing papers on "Data Corruption published in 2010"

A Robust and Efficient Method for Fusing Heterogeneous Data from Traffic Sensors on Freeways

Abstract: : Fusing freeway traffic data such as spot speeds and travel times from a variety of traffic sensors (loops, cameras, automated vehicle identification systems) into a coherent, consistent, and reliable picture of the prevailing traffic conditions (eg, speeds, flows) is a critical task in any off- or online traffic management or data archival system This task is challenging as such data differ in terms of spatial and temporal resolution, accuracy, reliability, and most importantly in terms of spatiotemporal semantics In this article, we propose a data fusion algorithm (the extended generalized Treiber-Helbing filter [the EGTF]) which, although heuristic in nature, uses basic notions from traffic flow theory and is generic in the sense that it does not impose any restrictions on the way the data are structured in a temporal or spatial way This implies that the data can stem from any data source, given they provide a means to distinguish between free flowing and congested traffic On the basis of (ground truth and sensor) data from a micro-simulation tool, we demonstrate that the EGTF method results in accurate reconstructed traffic conditions and is robust to increasing degrees of data corruption Further research should focus on validating the approach on real data The method can be straightforwardly implemented in any traffic data archiving system or application which requires consistent and coherent traffic data from traffic sensors as inputs

Reducing Corruption of Communication in a Wireless Power Transmission System

Abstract: An embodiment of the invention provides a method for reducing data corruption in a wireless power transmission system. Power is transmitted from a primary coil to a secondary coil by induction. The voltage induced on the secondary coil by induction is rectified. The change in current supplied to a load configured to be coupled to the wireless power transmission system is limited.

Detecting inadvertent or malicious data corruption in storage subsystems and recovering data

Abstract: Embodiments of the invention detect inadvertent or malicious data corruption and for recovering data including receiving a query specifying corrupted application data; analyzing transaction logs to find update operations related to the data; determining where the data are stored, by mapping the table data to locations within the file system and mapping the file system locations to volume logical blocks; and analyzing snapshot volume bitmaps to determine if the bitmaps show changes to the table data stored in the volume logical blocks. Changes which are reflected in the bitmaps for the data, but which do not have corresponding entries in the transaction logs are flagged as unauthorized changes. Snapshots of the data, from a time prior to the time at which a flagged snapshot was taken, are identified for use in restoring data to its status prior to the unauthorized change.

Data recovery for web applications

Abstract: Web-based applications store their data at the server side. This design has several benefits, but it can also cause a serious problem because a misconfiguration, bug or vulnerability leading to data loss or corruption can affect many users. While data backup solutions can help resolve some of these issues, they do not help diagnose the events that led to the corruption or the precise set of changes caused by these events. In this paper, we describe the design of a recovery system that helps administrators recover from data corruption caused by bugs in web applications. Our system tracks application requests, helping identify requests that cause data corruption, and reuses undo logs already kept by databases to selectively recover from the effects of these requests. The main challenge is to correlate requests across the multiple tiers of the application to determine the correct recovery actions. We explore using dependencies both within and across requests at three layers (database, application, and client) to help identify data corruption accurately. We evaluate our system using known bugs in popular web applications, including Wordpress, Drupal and Gallery2. Our results show that our system enables recovery from data corruption without loss of critical data and incurs small runtime overhead.

Tools for modelling and simulating migrationbased preservation

Abstract: This report describes two tools for modelling and simulating the costs and risks of using IT storage systems for the long-term archiving of file-based AV assets. The tools include a model of storage costs, the ingest and access of files, the possibility of data corruption and loss from a range of mechanisms, and the impact of having limited resources with which to fulfill access requests and preservation actions. Applications include archive planning, development of a technology strategy, cost estimation for business planning, operational decision support, staff training and generally promoting awareness of the issues and challenges archives face in digital preservation.

Autonomous storage management for low-end computing environments

Abstract: To make storage management transparent to users, enterprises rely on expensive storage infrastructure, such as high end storage appliances, tape robots, and offsite storage facilities, maintained by full-time professional system administrators. From the user's perspective access to data is seamless regardless of location, backup requires no periodic, manual action by the user, and help is available to recover from storage problems. The equipment and administrators protect users from the loss of data due to failures, such as device crashes, user errors, or virii, as well as being inconvenienced by the unavailability of critical files. Home users and small businesses must manage increasing amounts of important data distributed among an increasing number of storage devices. At the same time, expert system administration and specialized backup hardware are rarely available in these environments, due to their high cost. Users must make do with error-prone, manual, and time-consuming ad hoc solutions, such as periodically copying data to an external hard drive. Non-technical users are likely to make mistakes, which could result in the loss of a critical piece of data, such as a tax return, customer database, or an irreplaceable digital photograph. In this thesis, we show how to provide transparent storage management for home and small business users We introduce two new systems: The first, PodBase, transparently ensures availability and durability for mobile, personal devices that are mostly disconnected. The second, SLStore, provides enterprise-level data safety (e.g. protection from user error, software faults, or virus infection) without requiring expert administration or expensive hardware. Experimental results show that both systems are feasible, perform well, require minimal user attention, and do not depend on expert administration during disaster-free operation. PodBase relieves home users of many of the burdens of managing data on their personal devices. In the home environment, users typically have a large number of personal devices, many of them mobile devices, each of which contain storage, and which connect to each other intermittently. Each of these devices contain data that must be made durable, and available on other storage devices. Ensuring durability and availability is difficult and tiresome for non-expert users, as they must keep track of what data is stored on which devices. PodBase transparently ensures the durability of data despite the loss or failure of a subset of devices; at the same time, PodBase aims to make data available on all the devices appropriate for a given data type. PodBase takes advantage of storage resources and network bandwidth between devices that typically goes unused. The system uses an adaptive replication algorithm, which makes replication transparent to the user, even when complex replication strategies are necessary. Results from a prototype deployment in a small community of users show that PodBase can ensure the durability and availability of data stored on personal devices under a wide range of conditions with minimal user attention. Our second system, SLStore, brings enterprise-level data protection to home office and small business computing. It ensures that data can be recovered despite incidents like accidental data deletion, data corruption resulting from software errors or security breaches, or even catastrophic storage failure. However, unlike enterprise solutions, SLStore does riot require professional system administrators, expensive backup hard- ware, or routine, manual actions on the part of the user. The system relies on storage leases, which ensure that data cannot be overwritten for a pre-determined period, and an adaptive storage management layer which automatically adapts the level of backup to the storage available. We show that this system is both practical, reliable and easy to manage, even in the presence of hardware and software faults.

Synthesis of BCH codes for enhancing data integrity in flash memories

Abstract: Flash memories have found extensive application for use in portable storage devices. They have been used for code storage as well as data storage. The storage density associated with these devices has increased tremendously in the past few years. This has necessitated very dense packing of data bits on the device. This gives rise to increased Raw Bit Error Rate (RBER) as a result of Inter Symbol Interference (ISI) between bits stored in adjacent cells. This necessitates the use of powerful error control codes to guarantee information integrity. With the increase in density of data storage, the raw bit error rate (RBER) associated with the storage device increases. Error Control Coding (ECC) can be used to reduce the RBER to acceptable values so that these devices can be employed to store information in applications where data corruption is unacceptable. In this paper, we describe the synthesis of BCH codes based on memory models proposed by the semiconductor industry. These codes have better error correcting capability than the codes used in current practice.

The effect of channel and time-slot assignment algorithms on data loss events in OFDM/TDMA multi-hop wireless networks

Abstract: In this study, we examine the effects of channel and time-slot allocation algorithms on data loss events in multi-hop wireless networks. We investigate networks where all nodes have a single-radio antenna, the MAC layer protocol is synchronized OFDM, and each channel uses TDMA. In such networks, data collisions can be avoided by reserving data time-slots. Data loss can also occur when a receiving node changes its assigned channel, hence it does not completely receive the data which is on the link. We call this type of loss data corruption. Our simulations show that if there are several available time slots to reserve, a proper channel and time-slot selection algorithm is required to prevent data corruption. Consequently, it is better that a node changes channels as infrequently as possible and any necessary change should take place at an appropriate time. The same is true about switching between receiving and sending modes, i.e. switching modes also leads to data loss, so it should be avoided whenever possible. In this study we propose a new algorithm to allocate channels and time-slots so as to decrease data loss.

Array storage device and method of controlling the same

Abstract: PROBLEM TO BE SOLVED: To avoid data corruption even if an unauthorized system error occurs during reconstruction. SOLUTION: An array storage device includes: a reconstruction module 204 which, if at least one of a plurality of storage devices fails and the failed storage device is replaced by a new storage device, performs a reconstruction process on the storage devices that have not failed; a reconstruction process management module 204 which stores reconstruction finishing point information 212 in a nonvolatile storage device each time the reconstruction process for a certain number of stripes is performed, the information indicating the position information of the stripe for which the reconstruction process is finished; and a writing module 201 which, if a write access is made to a stripe between the stripe corresponding to the reconstruction finishing point information 212 stored in the nonvolatile storage device and the stripe for which the reconstruction process is currently performed by a reconstruction means, writes data in the replaced storage device after writing data in the storage devices not replaced. COPYRIGHT: (C)2010,JPO&INPIT

Design of fault tolerant system based on runtime behavior tracing

Abstract: The current researches to improve the reliability of operating systems have been focusing on the evolution of kernel architecture or protecting against device driver errors. In particularly, the device driver errors are critical to the most of the complementary operating systems that have a kernel level device driver. Especially on special purpose embedded system, because of its limited resources and variety of devices, more serious problems are induced. Preventing data corruption or blocking the arrogation of operational level is not enough to cover the entire problems. For examples, when using device drivers, the violation of function's call sequence can cause a malfunction. Also a violation of behavior rules on system level involves the same problem. This type of errors is difficult to be detected by the previous methods. Accordingly, we designed a system that traces system behavior at runtime and recovers optimally when errors are detected. We experimented in Linux 2.6.24 kernel operating on GP2X-WIZ mobile game player.

Support for Stronger Error Detection Codes in TCP for Jumbo Frames

Abstract: There is a class of data serving protocols and applications that cannot tolerate undetected data corruption on the wire. Data corruption could occur at the source in software, in the network interface card, out on the link, on intermediate routers or at the destination network interface card or node. The Ethernet CRC and the 16-bit checksum in the TCP/UDP headers are used to detect data errors. Most applications rely on these checksums to detect data corruptions and do not use any checksums or CRC checks at their level. Research has shown that the TCP/UDP checksums are catching a significant number of errors, however, the research suggests that one packet in 10 billion will have an error that goes undetected for Ethernet MTU frames (MTU of 1500). Under certain situations, "bad" hosts can introduce undetected errors at a much higher frequency and order. With the use of Jumbo frames on the rise, and therefore more data bits on the wire that could be corrupted, the current 16-bit TCP/UDP checksum, or the Ethernet 32-bit CRC are simply not sufficient for detecting errors. This document specifies a proposal to use stronger checksum algorithms for TCP Jumbo Frames for IPv4 and IPv6 networks. The Castagnoli CRC 32C algorithm used in iSCSI and SCTP is proposed as the error detection code of choice.

Performance is dead, long live performance!

Abstract: In a world of social networking, security attacks, and hot mobile phones, the importance of application performance appears to have dimin-ished. My own research agenda has shifted from looking at the performance of memory allocation to building runtime systems that are more resilient to data corruption and security attacks. In my talk, I will outline a number of areas where code-generation and runtime tech-niques can be successfully applied to areas for purposes other than performance, such as fault tolerance, reliability, and security. Along the way, I will consider such questions as "Does it really matter if this corruption was caused by a software or hardware error?" and "Is it okay to let a malicious person allocate arbitrary data on my heap?".Despite these other opportunities, the importance of performance in modern applications remains undiminished, and current hardware trends place an increasing burden on software to provide needed performance boosts. In concluding, I will suggest several important trends that I believe will help define the next 10 years of code generation and optimization research.

Reflections on a SER-aware design flow

Abstract: Single Event Effects (SEEs) may cause system downtime, data corruption and maintenance incidents. Thus, the SEE are a threat to the overall system reliability, causing designers to be increasingly concerned about the analysis and the mitigation of radiation-induced failures, even for commercial systems performing in a natural working environment. Experts and reliability engineers are called in to support chip designers in the management of Single Event Effects. To this goal, we present a design-flow-oriented Soft Error Rate analysis methodology geared to allow practical and concrete decisions concerning implementation, design and functional choices in order to minimize SEEs impact on circuit and system behavior.