Flash memory is a type of electrically-erasable programmable read-only memory (EEPROM). Because flash memories are nonvolatile and relatively dense, they are now used to store files and other persistent objects in handheld computers, mobile phones, digital cameras, portable music players, and many other computer systems in which magnetic disks are inappropriate. Flash, like earlier EEPROM devices, suffers from two limitations. First, bits can only be cleared by erasing a large block of memory. Second, each block can only sustain a limited number of erasures, after which it can no longer reliably store data. Due to these limitations, sophisticated data structures and algorithms are required to effectively use flash memories. These algorithms and data structures support efficient not-in-place updates of data, reduce the number of erasures, and level the wear of the blocks in the device. This survey presents these algorithms and data structures, many of which have only been described in patents until now.

/pdf/algorithms-and-data-structures-for-flash-memories-5go86p74mj.pdf

Algorithms and data structures for flash memories

Write amplification is a critical factor limiting the random write performance and write endurance in storage devices based on NAND-flash memories such as solid-state drives (SSD). The impact of garbage collection on write amplification is influenced by the level of over-provisioning and the choice of reclaiming policy. In this paper, we present a novel probabilistic model of write amplification for log-structured flash-based SSDs. Specifically, we quantify the impact of over-provisioning on write amplification analytically and by simulation assuming workloads of uniformly-distributed random short writes. Moreover, we propose modified versions of the greedy garbage-collection reclaiming policy and compare their performance. Finally, we analytically evaluate the benefits of separating static and dynamic data in reducing write amplification, and how to address endurance with proper wear leveling.

/pdf/write-amplification-analysis-in-flash-based-solid-state-29qqajkwed.pdf

Write amplification analysis in flash-based solid state drives

A memory system and method of operating the same is described, where the memory system is used to store data in a RAIDed manner. The stored data may be retrieved, including the parity data so that the stored data is recovered when the first of either the stored data without the parity data, or the stored data from all but one memory module and the parity data, has been received. The writing of data, for low write data loads, is managed such that only one of the memory modules of a RAID stripe is being written to, or erased, during a time interval.

Memory management system and method

NAND flash memory is ubiquitous in everyday life today because its capacity has continuously increased and cost has continuously decreased over decades. This positive growth is a result of two key trends: 1) effective process technology scaling; and 2) multi-level (e.g., MLC, TLC) cell data coding. Unfortunately, the reliability of raw data stored in flash memory has also continued to become more difficult to ensure, because these two trends lead to 1) fewer electrons in the flash memory cell floating gate to represent the data; and 2) larger cell-to-cell interference and disturbance effects. Without mitigation, worsening reliability can reduce the lifetime of NAND flash memory. As a result, flash memory controllers in solid-state drives (SSDs) have become much more sophisticated: they incorporate many effective techniques to ensure the correct interpretation of noisy data stored in flash memory cells. In this article, we review recent advances in SSD error characterization, mitigation, and data recovery techniques for reliability and lifetime improvement. We provide rigorous experimental data from state-of-the-art MLC and TLC NAND flash devices on various types of flash memory errors, to motivate the need for such techniques. Based on the understanding developed by the experimental characterization, we describe several mitigation and recovery techniques, including 1) cell-to-cell interference mitigation; 2) optimal multi-level cell sensing; 3) error correction using state-of-the-art algorithms and methods; and 4) data recovery when error correction fails. We quantify the reliability improvement provided by each of these techniques. Looking forward, we briefly discuss how flash memory and these techniques could evolve into the future.

Error Characterization, Mitigation, and Recovery in Flash-Memory-Based Solid-State Drives

Hot data identification for flash memory storage systems not only imposes great impacts on flash memory garbage collection but also strongly affects the performance of flash memory access and its lifetime (due to wear-levelling). This research proposes a highly efficient method for on-line hot data identification with limited space requirements. Different from past work, multiple independent hash functions are adopted to reduce the chance of false identification of hot data and to provide predictable and excellent performance for hot data identification. This research not only offers an efficient implementation for the proposed framework, but also presents an analytic study on the chance of false hot data identification. A series of experiments was conducted to verify the performance of the proposed method, and very encouraging results are presented.

Efficient identification of hot data for flash memory storage systems

Flash-memory technology is becoming critical in building embedded systems applications because of its shock-resistant, power economic, and nonvolatile nature. With the recent technology breakthroughs in both capacity and reliability, flash-memory storage systems are now very popular in many types of embedded systems. However, because flash memory is a write-once and bulk-erase medium, we need a translation layer and a garbage-collection mechanism to provide applications a transparent storage service. In the past work, various techniques were introduced to improve the garbage-collection mechanism. These techniques aimed at both performance and endurance issues, but they all failed in providing applications a guaranteed performance. In this paper, we propose a real-time garbage-collection mechanism, which provides a guaranteed performance, for hard real-time systems. On the other hand, the proposed mechanism supports non-real-time tasks so that the potential bandwidth of the storage system can be fully utilized. A wear-leveling method, which is executed as a non-real-time service, is presented to resolve the endurance problem of flash memory. The capability of the proposed mechanism is demonstrated by a series of experiments over our system prototype.

https://www.csie.ntu.edu.tw/~d6526009/papers/ACM_TES_final.pdf

Real-time garbage collection for flash-memory storage systems of real-time embedded systems

This paper proposes a remote thin client system for real time multimedia streaming over VNC A remote frame can be split as two parts, ie high motion part and low motion part, and transmitted through the Internet from servers to clients according to the proposed hybrid RTP protocol A Dynamic Image Detection Scheme (DIDS) is proposed to automatically detecting the high motion part of a frame with only 1% of extra CPU loading In addition, an Error Detection Scheme (EDS) and a Dynamic Bit-rate Control Scheme (DBCS) are also proposed to ensure good video streaming quality under bandwidth limited applications This paper also proposes a linear time BU-level rate control algorithm to ensure the proposed DBCS can be finished in real time The proposed algorithm reduces the computational complexity from O(n2)to be O(n) By using the proposed thin client system, we can achieve about 22 fps of real-time SIF video streaming with good video quality under 32 KByte/s of bandwidth limitation, which speeds up about 172 times in remote frame display when compared to pure VNC

A remote thin client system for real time multimedia streaming over VNC

A dynamic-voltage-adjustment method is proposed to reduce the power consumption of a flash memory storage system, depending on the system workload. The usefulness of the proposed method is demonstrated by a series of experiments with very encouraging results.

/pdf/a-dynamic-voltage-adjustment-mechanism-in-reducing-the-power-nqvyr5oeio.pdf

A dynamic-voltage-adjustment mechanism in reducing the power consumption of flash memory for portable devices

Unlike a magnetic disk, a flash drive needs no seek time when performing random access. In addition, the read speed of a flash drive is faster than the write speed, and the write speed depends on the I/O request size: the bigger the request size, the faster the write speed. System hibernation stores the machine status completely in non-volatile memory. When the system reboots, the data will be reloaded to the machine, and the system status will be recovered completely. The user can continue immediately where they stopped the last time.We use the characteristic that the flash drives requires no seek time when performing random access to write the data in the memory to swap space or a hibernation file, or simply discard it without any I/O. The write process will try to combine random small writes into continuous large writes, if possible, to optimize the write speed. When the system resumes, it only reloads the data that the user needs, and the resume technique is based on paging on demand. Paging on demand is actually a kind of random access; however, since the flash drive requires no seek time, it does not drag down the system response time due to reducing the number of I/O.

Shi-Wu Lo

Papers

Real-time garbage collection for flash-memory storage systems of real-time embedded systems

A remote thin client system for real time multimedia streaming over VNC

A dynamic-voltage-adjustment mechanism in reducing the power consumption of flash memory for portable devices

Swap-before-hibernate: a time efficient method to suspend an OS to a flash drive

Location management in cellular mobile computing systems with dynamic hierarchical location databases