Data type and amount in human society is growing in amazing speed which caused by emerging new service such as cloud computing, internet of things and social network, the era of Big Data has come. Data has been fundamental resource from simple dealing object, and how to manage and utilize big data better has attracted much attention. Evolution or revolution on database research for big data is a problem. This paper discusses the concept of big data, and surveys its state of the art. The framework of big data is described and key techniques are studied. Finally some new challenges in the future are summarized.

https://crad.ict.ac.cn/EN/article/downloadArticleFile.do?attachType=PDF&id=715

Big Data Management: Concepts，Techniques and Challenges

Recent research has shown that applications often incorrectly implement crash consistency. We present the Crash-Consistent File System (ccfs), a file system that improves the correctness of application-level crash consistency protocols while maintaining high performance. A key idea in ccfs is the abstraction of a stream. Within a stream, updates are committed in program order, improving correctness; across streams, there are no ordering restrictions, enabling scheduling flexibility and high performance. We empirically demonstrate that applications running atop ccfs achieve high levels of crash consistency. Further, we show that ccfs performance under standard file-system benchmarks is excellent, in the worst case on par with the highest performing modes of Linux ext4, and in some cases notably better. Overall, we demonstrate that both application correctness and high performance can be realized in a modern file system.

Application Crash Consistency and Performance with CCFS.

Recently, the development of phase change memory (PCM) motivates new hybrid memory architectures that consist of PCM and DRAM. An important issue in such hybrid memory architectures is how to manage the pages resisting in heterogeneous memories. For example, when a requested page is missing in the hybrid memory and the memory has no free spaces, what pages in which type of memory (PCM or DRAM) should be replaced? This problem is much different from traditional buffer replacement management, where they do not consider the special properties of different types of memories. In particular, differing from DRAM, PCM is non-volatile but it has lower access speeds than DRAM. Further, PCM has a limited write endurance which implies that it cannot be written endlessly. Therefore, we have to design a new page replacement algorithm that can not only maintain a high hit ratio as traditional algorithms do but also can avoid frequent writes to PCM. In this paper, aiming to provide a new solution to the page replacement problem in PCM/DRAM-based hybrid memories, we propose a new algorithm called MHR-LRU (Maintain-hit-ratio LRU). The objective of our algorithm is to reduce PCM writes while maintaining a high hit ratio. Specially, it keeps recently updated pages in DRAM and performs page migrations between PCM and DRAM. The migrations take into account both page access patterns and the influences of page faults. We conduct trace-driven experiments and compared our proposal with some existing algorithms including LRU, LRU-WPAM, and CLOCK-DWF. The results show that our proposal is able to efficiently reduce PCM writes without degrading the hit ratio. Thus, our study offers a better solution for the page replacement issue in PCM/DRAM-based hybrid memory systems than previous approaches.

/pdf/a-novel-page-replacement-algorithm-for-the-hybrid-memory-2te0p9tnl7.pdf

A Novel Page Replacement Algorithm for the Hybrid Memory Architecture Involving PCM and DRAM

Over the past decade, storage has been the performance bottleneck in I/O-intensive programs such as online transaction processing applications To alleviate this bottleneck with minimal cost penalty, cost-effective design of a high-performance disk subsystem is of decisive importance in enterprise applications Data tiering is an efficient way to optimize cost, performance, and reliability in storage servers With the promising advantages of solid-state drives (SSDs) over hard disk drives (HDDs) such as lower power consumption and higher performance, traditional data tiering techniques should be revisited to use SSDs in a more efficient way Previously proposed tiering solutions have attempted to enhance performance based on different parameters such as request size or randomness These solutions, however, are mostly optimized towards one type of I/O workloads and are not applicable to workloads with different characteristics This paper presents an online data tiering technique at the operating system level with a linear weighted formulation to enhance I/O performance with minimal cost overhead The proposed technique characterizes the workload access pattern with respect to metadata versus user data, frequency of accesses, random versus sequential accesses, and read versus write accesses To evaluate the proposed technique, it is implemented on a Linux 314 equipped with ext2 filesystem The experimental results over I/O-intensive workloads show that the proposed technique improves performance up to 30 % as compared to the previous techniques while imposing negligible memory overhead to the system

http://dsn.ce.sharif.edu/wp-content/uploads/2012/10/Tiering-OS-JSC2015.pdf

Operating system level data tiering using online workload characterization

Phase change memory (PCM) has become one of the most promising storage media particularly for memory systems, due to its byte addressability, high access speed, and low energy consumption. In addition, hybrid memory systems involving both PCM and DRAM can utilize the merits of both media and overcome some typical drawbacks of PCM such as high write latency and limited lifecycle. In this paper, we present a novel page replacement algorithm called APP-LRU (Access-Pattern-prediction-based LRU) for PCM/DRAM-based hybrid memory systems. APP-LRU aims to reduce writes to PCM while maintaining stable time performance. Particularly, we detect read/write intensity for each page in the memory, and put read-intensive pages into PCM while placing write-intensive pages in DRAM. We conduct trace-driven experiments on six synthetic traces and one real OLTP trace. The results show that our proposal is able to reduce up to 5 times of migrations more than its competitors.

/pdf/app-lru-a-new-page-replacement-method-for-pcm-dram-based-gozeiwsgkt.pdf

APP-LRU: A New Page Replacement Method for PCM/DRAM-Based Hybrid Memory Systems

Flash-memory-based solid state drives (SSD) have been widely used in computer systems. Due to the high price and some specific features of SSD such as asymmetric read/write speeds and limited erasure endurance, it has been a very common solution, e.g., in modern data centers, to use hybrid storage systems involving SSD and traditional hard disks (HDD). However, the SSD/HDD-based hybrid storage systems introduce some new problems in the indexing schemes for data management. In this paper, we propose a new B+-tree-based index for such hybrid storage systems, which is called HybridB tree. The HybridBtree aims to reduce the random writes to SSD while keeping high time performance and low buffer costs. Particularly, we introduce a new design called huge leaf to avoid the splits and merges on B+-tree. A huge leaf node contains two or more leaf nodes in different states. We place the leaf nodes on HDD or SSD according to their current states, and dynamically adapt the states of leaf nodes when they are read or updated. After a detailed explanation on the structure and operations of the HybridB tree, we give a theoretical analysis on the costs of the HybridB tree. Then, we conduct experiments on two TPC-C traces, using a real hybrid storage system including one HDD and two SSDs, and compare the performance of our proposal with two implementations of B+-tree, namely the B+-tree on HDD and the B+-tree on SSD/HDD. The results show that our proposal has the best time performance and the fewest buffer costs. Moreover, our proposal is able to effectively reduce the random writes to SSD.

Optimizing B+-tree for hybrid storage systems

Flash memory has been widely used for storage devices in various embedded systems and enterprise computing environment, due to its shock-resistance, low power consumption, non-volatile, and high I/O speed. However, its physical characteristics impose several limitations in the design of flash-based solid state disks (SSDs). For example, its write operation costs much more time than read operation, and data in flash memory can not be overwritten before being erased. In particular, random write operations in flash memory have a very poor performance. To overcome these limitations, we propose a page-clustered LRU write buffer management scheme for flash-based SSDs, which is named BPCLC (Block Padding Cold and Large Cluster first). BPCLC adopts a new block padding technique to improve the write performance of flash-based SSDs. We conduct a trace-driven experiment and use two types of workloads to compare the performance of BPCLC with three competitors including FAB, BPLRU, and CLC. The results show that in both types of workloads, BPCLC outperforms its competitors with respect to write count, erase count, merge count, and overall I/O overhead.

BPCLC: An Efficient Write Buffer Management Scheme for Flash-Based Solid State Disks

Flash-memory-based solid-state drives (SSDs) are used widely for secondary storage. To be effective for SSDs, traditional indices have to be redesigned to cope with the special properties of flash memory, such as asymmetric read/write latencies (fast reads and slow writes) and out-of-place updates. Previous flash-optimized indices focus mainly on reducing random writes to SSDs, which is typically accomplished at the expense of a substantial number of extra reads. However, modern SSDs show a narrowing gap between read and write speeds, and read operations on SSDs increasingly affect the overall performance of indices on SSDs. As a consequence, how to optimize SSD-aware indices by reducing both write and read costs is a pertinent and open challenge. We propose a new tree index for SSDs that is able to reduce both writes and extra reads. In particular, we use an update buffer and overflow pages to reduce random writes, and we further exploit Bloom filters to reduce the extra reads to the overflow nodes in the tree. With this mechanism, we construct a read/write-optimized index that is capable of offering better overall performance than previous flash-aware indices. In addition, we present an analysis of the proposed index and show that the read and write costs of the operations on the index can be balanced by only tuning the false-positive rate of the Bloom filters. Our experimental results suggest that our proposal is efficient and represents an improvement over existing methods.

Read/write-optimized tree indexing for solid-state drives

In recent years, flash memory based storage device SSDs (solid state drives) have been regarded as the storage devices of next generation to replace HDDs (hard disk drivers). However, the high price of SSDs, especially those with high performance, results in the situation that SSDs and HDDS are both popularly used in real applications. In order to integrate the merits of SSDs and HDDS, it has become a hot research topic that using HDDs for SSDs to construct a hybrid storage system. The goal of this paper is to use the cheap low-end SSD and HDD to build a hybrid storage system with high efficiency, which is called HB-Storage. HB-Storage considers the characters of SSDs and HDDs, and builds a HDD write buffer to optimize the SSD write request. The write buffer is designed based on the data access load statistics. As a consequence, HB-Storage can utilize the higher read performance of SSDs, and can also improve the random write latency of SSDs. The experimental results show that HB-Storage can maintain a high read performance and significantly reduce the write requests on the SSD, and thus has higher overall performance.

HB-Storage: Optimizing SSDs with a HDD Write Buffer

Hybrid Storage servers combining high-speed SSDs and high-capacity HDDs are designed for high cost-effectiveness and provide μs-level responsiveness for applications. Observations from the production hybrid cloud storage system Pangu suggest that HDDs are often severely underutilized while SSDs are overused, especially for writes that dominate the hybrid storage. This lopsided utilization between HDDs and SSDs leads to not only fast wear-out in the latter but also very high tail latency due to frequent garbage collections induced by intensive writes to the latter. On the other hand, our extensive experimental study reveals that a series of sequential and continuous writes to HDDs exhibit a periodic, staircase shaped pattern of write latency, i.e., low (e.g., 35μs), middle (e.g., 55μs), and high latency (e.g., 12ms), resulting from buffered writes in HDD’s controller. This suggests that HDDs can potentially provide μs-level write IO delay (for appropriately scheduled writes), which is close to SSDs’ write performance. These observations inspire us to effectively exploit this performance potential of HDDs to absorb as many writes as possible to avoid SSD overuse without performance degradation. To achieve this goal, we first characterize performance behaviors of hybrid storage in general and its HDDs in particular. Based on the findings on sequential and continuous writes, we propose a prediction model to accurately determine next write latency state (i.e., fast, middle and slow). With this model, a Buffer-Controlled Write approach, BCW, is proposed to proactively and effectively control buffered writes so that lowand mid-latency periods in HDDs are scheduled with application write data and high-latency periods are filled with padded data. Based on BCW, we design a mixed IO scheduler (MIOS) to adaptively steer incoming data to SSDs and HDDs according to write patterns, runtime queue lengths, and disk status. We perform extensive evaluations under production workloads and benchmarks. The results show that MIOS removes up to 93% amount of data written to SSDs, reduces ∗Corresponding author. Email: caoqiang@hust.edu.cn Figure 1: Sequential writing in a 10TB Western Digital HDD. average and 99th-percentile latencies of the hybrid server by 65% and 85% respectively.

/pdf/bcw-buffer-controlled-writes-to-hdds-for-ssd-hdd-hybrid-34x9l73282.pdf

Puyuan Yang

Papers

Optimizing B+-tree for hybrid storage systems

BPCLC: An Efficient Write Buffer Management Scheme for Flash-Based Solid State Disks

Read/write-optimized tree indexing for solid-state drives

HB-Storage: Optimizing SSDs with a HDD Write Buffer

{BCW}: Buffer-Controlled Writes to HDDs for SSD-HDD Hybrid Storage Server