We present WiscKey, a persistent LSM-tree-based key-value store with a performance-oriented data layout that separates keys from values to minimize I/O amplification. The design of WiscKey is highly SSD optimized, leveraging both the sequential and random performance characteristics of the device. We demonstrate the advantages of WiscKey with both microbenchmarks and YCSB workloads. Microbenchmark results show that WiscKey is 2.5×-111× faster than LevelDB for loading a database and 1.6×-14× faster for random lookups. WiscKey is faster than both LevelDB and RocksDB in all six YCSB workloads.

WiscKey: separating keys from values in SSD-conscious storage

Key-value stores such as LevelDB and RocksDB offer excellent write throughput, but suffer high write amplification. The write amplification problem is due to the Log-Structured Merge Trees data structure that underlies these key-value stores. To remedy this problem, this paper presents a novel data structure that is inspired by Skip Lists, termed Fragmented Log-Structured Merge Trees (FLSM). FLSM introduces the notion of guards to organize logs, and avoids rewriting data in the same level. We build PebblesDB, a high-performance key-value store, by modifying HyperLevelDB to use the FLSM data structure. We evaluate PebblesDB using micro-benchmarks and show that for write-intensive workloads, PebblesDB reduces write amplification by 2.4-3x compared to RocksDB, while increasing write throughput by 6.7x. We modify two widely-used NoSQL stores, MongoDB and HyperDex, to use PebblesDB as their underlying storage engine. Evaluating these applications using the YCSB benchmark shows that throughput is increased by 18-105% when using PebblesDB (compared to their default storage engines) while write IO is decreased by 35-55%.

PebblesDB: Building Key-Value Stores using Fragmented Log-Structured Merge Trees

Recent demands for storing and querying big data have revealed various shortcomings of traditional relational database systems. This, in turn, has led to the emergence of a new kind of complementary nonrelational data store, named as NoSQL. This survey mainly aims at elucidating the design decisions of NoSQL stores with regard to the four nonorthogonal design principles of distributed database systems: data model, consistency model, data partitioning, and the CAP theorem. For each principle, its available strategies and corresponding features, strengths, and drawbacks are explained. Furthermore, various implementations of each strategy are exemplified and crystallized through a collection of representative academic and industrial NoSQL technologies. Finally, we disclose some existing challenges in developing effective NoSQL stores, which need attention of the research community, application designers, and architects.

A Survey on NoSQL Stores

We present WiscKey, a persistent LSM-tree-based key-value store with a performance-oriented data layout that separates keys from values to minimize I/O amplification. The design of WiscKey is highly SSD optimized, leveraging both the sequential and random performance characteristics of the device. We demonstrate the advantages of WiscKey with both microbenchmarks and YCSB workloads. Microbenchmark results show that WiscKey is 2.5 × to 111 × faster than LevelDB for loading a database (with significantly better tail latencies) and 1.6 × to 14 × faster for random lookups. WiscKey is faster than both LevelDB and RocksDB in all six YCSB workloads.

/pdf/wisckey-separating-keys-from-values-in-ssd-conscious-storage-4kzn5wjde8.pdf

WiscKey: Separating Keys from Values in SSD-Conscious Storage

Flash is a type of Electronically Erasable Programmable Read-Only Memory (EEPROM). Different from traditional magnetic disks, flash memories have no moving parts and are purely electronic devices, giving them unique advantages, such as lower access latency, lower power consumption, higher density, shock resistance, and lack of noise. However, existing applications cannot run directly on flash memories due to their special characteristics. Flash Translation Layer (FTL) is a software layer built on raw flash memories that emulates a normal block device like magnetic disks. Primary functionalities of the FTL include address translation, garbage collection, and wear leveling. This survey focuses on address translation technologies and provides a broad overview of existing schemes described in patents, journals, and conference proceedings.

/pdf/a-survey-of-address-translation-technologies-for-flash-38dujiiaq5.pdf

A survey of address translation technologies for flash memories

Indexing key-value data on persistent storage is an important factor for NoSQL databases. Most key-value storage engines use tree-like structures for data indexing, but their performance and space overhead rapidly get worse as the key length becomes longer. This also affects the merge or compaction cost which is critical to the overall throughput. In this paper, we present ForestDB, a key-value storage engine for a single node of large-scale NoSQL databases. ForestDB uses a new hybrid indexing scheme called HB $^+$ -trie, which is a disk-based trie-like structure combined with B $^+$ -trees. It allows for efficient indexing and retrieval of arbitrary length string keys with relatively low disk accesses over tree-like structures, even though the keys are very long and randomly distributed in the key space. Our evaluation results show that ForestDB significantly outperforms the current key-value storage engine of Couchbase Server [1] , LevelDB [2] , and RocksDB [3] , in terms of both the number of operations per second and the amount of disk writes per update operation.

ForestDB: A Fast Key-Value Storage System for Variable-Length String Keys

As NAND flash memory is gaining popularity as a storage medium for mobile embedded devices, many flash-aware file systems, flash-aware DBMSes, and flash translation layers (FTLs) require an flash-efficient index structure. This paper proposes a novel index structure called μ*-Tree which natively works on NAND flash memory, aiming at improving performance over B+-Tree. μ*-Tree stores all the nodes along the path from the root to the leaf into a single flash memory page in order to minimize the number of flash write operation when a node is updated. Furthermore, μ*-Tree has an adaptive page layout scheme which dynamically adjusts the page layout according to the workload characteristics on-the-fly. μ*-Tree also allows flash pages with different page layouts to coexist in the same tree. Our evaluation results with real workload traces show that μ*-Tree outperforms B+-Tree by up to 55 percent in terms of the time needed for flash operations. With a small in-memory cache of 32 KB, μ*-Tree improves the overall performance by up to five times compared to B+-Tree with the same cache size.

&#x03BC;*-Tree: An Ordered Index Structure for NAND Flash Memory with Adaptive Page Layout Scheme

An indexing method is based on a tree structure of a flash memory, which includes a plurality of pages. The indexing method stores an entry in the leaf node and an entry in an index node designating the leaf node, in the same page, and changes the maximum number of entries that are stored in the leaf node of the page and the maximum number of entries that are stored in the index node of the page on the basis of the number of entries in the leaf node and the number of entries in the index node, respectively.

Jung-Sang Ahn

Papers

ForestDB: A Fast Key-Value Storage System for Variable-Length String Keys

μ*-Tree: An Ordered Index Structure for NAND Flash Memory with Adaptive Page Layout Scheme

Indexing Method for Flash Memory

Jung-Sang Ahn

Papers

ForestDB: A Fast Key-Value Storage System for Variable-Length String Keys

&#x03BC;*-Tree: An Ordered Index Structure for NAND Flash Memory with Adaptive Page Layout Scheme

Indexing Method for Flash Memory

μ*-Tree: An Ordered Index Structure for NAND Flash Memory with Adaptive Page Layout Scheme