A storage system is provided with a memory region, a cache memory region, and a processor. The memory region stores the time relation information that indicates a time relationship of a data element that has been stored into the cache memory region and that is to be written to the logical region and a snapshot acquisition point of time to the primary volume. The processor judges whether or not the data element that has been stored into the cache memory region is a snapshot configuration element based on the time relation information for the data element that is to be written to a logical region of a write destination that conforms to the write request that specifies the primary volume and that has been stored into the cache memory region. In the case in which the result of the judgment is positive, the processor saves the data element to the secondary volume for holding a snapshot image in which the snapshot configuration element is a configuration element, and a data element of a write target is then stored into the cache memory region.

Storage system and storage control method

In many important applications-such as search engines and relational database systems-data are stored in the form of arrays of integers. Encoding and, most importantly, decoding of these arrays consumes considerable CPUtime. Therefore, substantial effort has been made to reduce costs associated with compression and decompression. In particular, researchers have exploited the superscalar nature of modern processors and single-instruction, multiple-data SIMD instructions. Nevertheless, we introduce a novel vectorized scheme called SIMD-BP128i¾? that improves over previously proposed vectorized approaches. It is nearly twice as fast as the previously fastest schemes on desktop processors varint-G8IU and PFOR. At the same time, SIMD-BP128i¾? saves up to 2bits/int. For even better compression, we propose another new vectorized scheme SIMD-FastPFOR that has a compression ratio within 10% of a state-of-the-art scheme Simple-8b while being two times faster during decoding. Copyright © 2013 John Wiley & Sons, Ltd.

Decoding billions of integers per second through vectorization

With the advent of approximation algorithms for NP-hard combinatorial optimization problems, several techniques from exact optimization such as the primal-dual method have proven their staying power and versatility. This book describes a simple and powerful method that is iterative in essence, and similarly useful in a variety of settings for exact and approximate optimization. The authors highlight the commonality and uses of this method to prove a variety of classical polyhedral results on matchings, trees, matroids, and flows. The presentation style is elementary enough to be accessible to anyone with exposure to basic linear algebra and graph theory, making the book suitable for introductory courses in combinatorial optimization at the upper undergraduate and beginning graduate levels. Discussions of advanced applications illustrate their potential for future application in research in approximation algorithms.

/pdf/iterative-methods-in-combinatorial-optimization-1jdohfilne.pdf

Iterative methods in combinatorial optimization

Database community has made significant research efforts to optimize query processing on GPUs in the past few years. However, we can hardly find that GPUs have been truly adopted in major warehousing production systems. Preparing to merge GPUs to the warehousing systems, we have identified and addressed several critical issues in a three-dimensional study of warehousing queries on GPUs by varying query characteristics, software techniques, and GPU hardware configurations. We also propose an analytical model to understand and predict the query performance on GPUs. Based on our study, we present our performance insights for warehousing query execution on GPUs. The objective of our work is to provide a comprehensive guidance for GPU architects, software system designers, and database practitioners to narrow the speed gap between the GPU kernel execution (the fast mode) and data transfer to prepare GPU execution (the slow mode) for high performance in processing data warehousing queries. The GPU query engine developed in this work is open source to the public.

/pdf/the-yin-and-yang-of-processing-data-warehousing-queries-on-1jiakvme50.pdf

The Yin and Yang of processing data warehousing queries on GPU devices

We present the first learned index that supports predecessor, range queries and updates within provably efficient time and space bounds in the worst case. In the (static) context of just predecessor and range queries these bounds turn out to be optimal. We call this learned index the Piecewise Geometric Model index (PGM-index). Its flexible design allows us to introduce three variants which are novel in the context of learned data structures. The first variant of the PGM-index is able to adapt itself to the distribution of the query operations, thus resulting in the first known distribution-aware learned index to date. The second variant exploits the repetitiveness possibly present at the level of the learned models that compose the PGM-index to further compress its succinct space footprint. The third one is a multicriteria variant of the PGM-index that efficiently auto-tunes itself in a few seconds over hundreds of millions of keys to satisfy space-time constraints which evolve over time across users, devices and applications.These theoretical achievements are supported by a large set of experimental results on known datasets which show that the fully-dynamic PGM-index improves the space occupancy of existing traditional and learned indexes by up to three orders of magnitude, while still achieving their same or even better query and update time efficiency. As an example, in the static setting of predecessor and range queries, the PGM-index matches the query performance of a cache-optimised static B+-tree within two orders of magnitude (83×) less space; whereas in the fully-dynamic setting, where insertions and deletions are allowed, the PGM-index improves the query and update time performance of a B+-tree by up to 71% within three orders of magnitude (1140×) less space.

/pdf/the-pgm-index-a-fully-dynamic-compressed-learned-index-with-36ni0el4i3.pdf

The PGM-index: a fully-dynamic compressed learned index with provable worst-case bounds

Major web search engines answer thousands of queries per second requesting information about billions of web pages. The data sizes and query loads are growing at an exponential rate. To manage the heavy workload, we consider techniques for utilizing a Graphics Processing Unit (GPU). We investigate new approaches to improve two important operations of search engines -- lists intersection and index compression.For lists intersection, we develop techniques for efficient implementation of the binary search algorithm for parallel computation. We inspect some representative real-world datasets and find that a sufficiently long inverted list has an overall linear rate of increase. Based on this observation, we propose Linear Regression and Hash Segmentation techniques for contracting the search range. For index compression, the traditional d-gap based compression schemata are not well-suited for parallel computation, so we propose a Linear Regression Compression schema which has an inherent parallel structure. We further discuss how to efficiently intersect the compressed lists on a GPU. Our experimental results show significant improvements in the query processing throughput on several datasets.

/pdf/efficient-parallel-lists-intersection-and-index-compression-7evuuk23tg.pdf

Efficient parallel lists intersection and index compression algorithms using graphics processing units

Lists intersection is an important operation in modern web search engines. Many prior studies have focused on the single-core or multi-core CPU platform or many-core GPU. In this paper, we propose a CPU-GPU cooperative model that can integrate the computing power of CPU and GPU to perform lists intersection more efficiently. In the so-called synchronous mode, queries are grouped into batches and processed by GPU for high throughput. We design a query-parallel GPU algorithm based on an element-thread mapping strategy for load balancing. In the traditional asynchronous model, queries are processed one-by-one by CPU or GPU to gain perfect response time. We design an online scheduling algorithm to determine whether CPU or GPU processes the query faster. Regression analysis on a huge number of experimental results concludes a regression formula as the scheduling metric. We perform exhaustive experiments on our new approaches. Experimental results on the TREC Gov and Baidu datasets show that our approaches can improve the performance of the lists intersection significantly.

Efficient lists intersection by CPU-GPU cooperative computing

Traditionally, full text retrieval over structure peer- to-peer network has been implemented by inverted index by keywords. However, search based on this index scheme only support literally word match, not taking into account the meaning of words. In this paper, we present a new index scheme, inverted index by concepts, for document retrieval over structure P2P network. We introduce ontology to capture the semantic relations between words, which provide a solution for ambiguity and richness of natural language. The index scheme proposed in this paper transforms the keyword search to the match process of concepts, and implementing keyword search based on semantic. Our index scheme also avoids the intersection of inverted lists between nodes when executing multi-keyword search. Simulation experiment shows that search based on our index scheme has higher retrieval performance than that based on keyword index, and generates lower network traffic.

Concept Index for Document Retrieval with Peer-to-Peer Network

In this paper, we study longest lowest-density MDS codes, a simple kind of multi-erasure array code with optimal redundancy and minimum update penalty. We prove some basic structure properties for longest lowest-density MDS codes. We define a "perfect" property for near-resolvable block designs (NRBs) and establish a bijection between 3-erasure longest lowest-density MDS codes (T-Codes) and perfect NRB(3? + 1, 3, 2)s. We present a class of NRB(3?+1, 3, 2)s, and prove that it produces a family of T-Codes. This family is infinite assuming Artin?s Conjecture. We also test some other NRBs and find some T-Code instances outside of this family.

T-Code: 3-Erasure Longest Lowest-Density MDS Codes

Snapshot technology is becoming prevalent to perform data protection and other tasks such as data mining and data cloning. To improve the performance and reliability of the traditional Linux LVM snapshot, we propose a novel cached dependent snapshot system, ESnap. ESnap decreases the total amount of data copy effectively using the data dependency among snapshot volumes. A new snapshot metadata organization scheme is designed to support massive snapshot volumes and the corresponding read/write algorithms are put forward. Also the automatic extending of snapshot volumes is implemented in ESnap to avoid the failure of the whole dependent snapshot chain due to space overflow of one snapshot. experimental results show that ESnap has higher performance, reliability and the resource utilization rate than traditional Linux LVM snapshot system.

Jing Liu

Papers

Efficient parallel lists intersection and index compression algorithms using graphics processing units

Efficient lists intersection by CPU-GPU cooperative computing

Concept Index for Document Retrieval with Peer-to-Peer Network

T-Code: 3-Erasure Longest Lowest-Density MDS Codes

ESnap - A Cached Dependent Snapshot System