In his seminal book, Shewhart (1931) makes no demand on the distribution of the characteristic to be plotted on a control chart. How then can we explain the idea that normality is, if not required, at least highly desirable? I believe that it has come about through the many statistical studies of control-chart behavior. If one is to study how a control chart behaves, it is necessary to relate it to some distribution. The obvious choice is the normal distribution because of its ubiquity as a satisfactory model. This is bolstered by the existence of the Central Limit Theorem.

Sampling: Design and Analysis

The Art of Multiprocessor Programming

Performance of in-memory key-value store (KVS) continues to be of great importance as modern KVS goes beyond the traditional object-caching workload and becomes a key infrastructure to support distributed main-memory computation in data centers. Recent years have witnessed a rapid increase of network bandwidth in data centers, shifting the bottleneck of most KVS from the network to the CPU. RDMA-capable NIC partly alleviates the problem, but the primitives provided by RDMA abstraction are rather limited. Meanwhile, programmable NICs become available in data centers, enabling in-network processing. In this paper, we present KV-Direct, a high performance KVS that leverages programmable NIC to extend RDMA primitives and enable remote direct key-value access to the main host memory. We develop several novel techniques to maximize the throughput and hide the latency of the PCIe connection between the NIC and the host memory, which becomes the new bottleneck. Combined, these mechanisms allow a single NIC KV-Direct to achieve up to 180 M key-value operations per second, equivalent to the throughput of tens of CPU cores. Compared with CPU based KVS implementation, KV-Direct improves power efficiency by 3x, while keeping tail latency below 10 μs. Moreover, KV-Direct can achieve near linear scalability with multiple NICs. With 10 programmable NIC cards in a commodity server, we achieve 1.22 billion KV operations per second, which is almost an order-of-magnitude improvement over existing systems, setting a new milestone for a general-purpose in-memory key-value store.

https://www.microsoft.com/en-us/research/wp-content/uploads/2017/12/kv-direct.pdf

KV-Direct: High-Performance In-Memory Key-Value Store with Programmable NIC

Distributed in-memory key-value stores (KVSs), such as memcached, have become a critical data serving layer in modern Internet-oriented datacenter infrastructure. Their performance and efficiency directly affect the QoS of web services and the efficiency of datacenters. Traditionally, these systems have had significant overheads from inefficient network processing, OS kernel involvement, and concurrency control. Two recent research thrusts have focused upon improving key-value performance. Hardware-centric research has started to explore specialized platforms including FPGAs for KVSs; results demonstrated an order of magnitude increase in throughput and energy efficiency over stock memcached. Software-centric research revisited the KVS application to address fundamental software bottlenecks and to exploit the full potential of modern commodity hardware; these efforts too showed orders of magnitude improvement over stock memcached. We aim at architecting high performance and efficient KVS platforms, and start with a rigorous architectural characterization across system stacks over a collection of representative KVS implementations. Our detailed full-system characterization not only identifies the critical hardware/software ingredients for high-performance KVS systems, but also leads to guided optimizations atop a recent design to achieve a record-setting throughput of 120 million requests per second (MRPS) on a single commodity server. Our implementation delivers 9.2X the performance (RPS) and 2.8X the system energy efficiency (RPS/watt) of the best-published FPGA-based claims. We craft a set of design principles for future platform architectures, and via detailed simulations demonstrate the capability of achieving a billion RPS with a single server constructed following our principles.

/pdf/architecting-to-achieve-a-billion-requests-per-second-2yyesnf1m7.pdf

Architecting to achieve a billion requests per second throughput on a single key-value store server platform

Non-volatile memory (NVM) as persistent memory is expected to substitute or complement DRAM in memory hierarchy, due to the strengths of non-volatility, high density, and near-zero standby power. However, due to the requirement of data consistency and hardware limitations of NVM, traditional indexing techniques originally designed for DRAM become inefficient in persistent memory. To efficiently index the data in persistent memory, this paper proposes a write-optimized and high-performance hashing index scheme, called level hashing, with low-overhead consistency guarantee and cost-efficient resizing. Level hashing provides a sharing-based two-level hash table, which achieves a constant-scale search/insertion/deletion/update time complexity in the worst case and rarely incurs extra NVM writes. To guarantee the consistency with low overhead, level hashing leverages log-free consistency schemes for insertion, deletion, and resizing operations, and an opportunistic log-free scheme for update operation. To cost-efficiently resize this hash table, level hashing leverages an inplace resizing scheme that only needs to rehash 1/3 of buckets instead of the entire table, thus significantly reducing the number of rehashed buckets and improving the resizing performance. Experimental results demonstrate that level hashing achieves 1:4×-3:0× speedup for insertions, 1:2×-2:1× speedup for updates, and over 4:3× speedup for resizing, while maintaining high search and deletion performance, compared with state-of-the-art hashing schemes.

/pdf/write-optimized-and-high-performance-hashing-index-scheme-2br6250ghh.pdf

Write-optimized and high-performance hashing index scheme for persistent memory

In-memory key-value stores play a critical role in data processing to provide high throughput and low latency data accesses. In-memory key-value stores have several unique properties that include (1) data intensive operations demanding high memory bandwidth for fast data accesses, (2) high data parallelism and simple computing operations demanding many slim parallel computing units, and (3) a large working set. As data volume continues to increase, our experiments show that conventional and general-purpose multicore systems are increasingly mismatched to the special properties of key-value stores because they do not provide massive data parallelism and high memory bandwidth; the powerful but the limited number of computing cores do not satisfy the demand of the unique data processing task; and the cache hierarchy may not well benefit to the large working set.In this paper, we make a strong case for GPUs to serve as special-purpose devices to greatly accelerate the operations of in-memory key-value stores. Specifically, we present the design and implementation of Mega-KV, a GPU-based in-memory key-value store system that achieves high performance and high throughput. Effectively utilizing the high memory bandwidth and latency hiding capability of GPUs, Mega-KV provides fast data accesses and significantly boosts overall performance. Running on a commodity PC installed with two CPUs and two GPUs, Mega-KV can process up to 160+ million key-value operations per second, which is 1.4-2.8 times as fast as the state-of-the-art key-value store system on a conventional CPU-based platform.

/pdf/mega-kv-a-case-for-gpus-to-maximize-the-throughput-of-in-1ntwrfh8ai.pdf

Mega-KV: a case for GPUs to maximize the throughput of in-memory key-value stores

In current databases, GPUs are used as dedicated accelerators to process each individual query. Sharing GPUs among concurrent queries is not supported, causing serious resource underutilization. Based on the profiling of an open-source GPU query engine running commonly used single-query data warehousing workloads, we observe that the utilization of main GPU resources is only up to 25%. The underutilization leads to low system throughput.To address the problem, this paper proposes concurrent query execution as an effective solution. To efficiently share GPUs among concurrent queries for high throughput, the major challenge is to provide software support to control and resolve resource contention incurred by the sharing. Our solution relies on GPU query scheduling and device memory swapping policies to address this challenge. We have implemented a prototype system and evaluated it intensively. The experiment results confirm the effectiveness and performance advantage of our approach. By executing multiple GPU queries concurrently, system throughput can be improved by up to 55% compared with dedicated processing.

/pdf/concurrent-analytical-query-processing-with-gpus-58ucbqfsvs.pdf

Concurrent analytical query processing with GPUs

Network Function Virtualization (NFV) virtualizes software network functions to offer flexibility in their design, management and deployment. Although GPUs have demonstrated their power in significantly accelerating network functions, they have not been effectively integrated into NFV systems for the following reasons. First, GPUs are severely underutilized in NFV systems with existing GPU virtualization approaches. Second, data isolation in the GPU memory is not guaranteed. Third, building an efficient network function on CPUGPU architectures demands huge development efforts. In this paper, we propose G-NET, an NFV system with a GPU virtualization scheme that supports spatial GPU sharing, a service chain based GPU scheduler, and a scheme to guarantee data isolation in the GPU. We also develop an abstraction for building efficient network functions on G-NET, which significantly reduces development efforts. With our proposed design, G-NET enhances overall throughput by up to 70.8% and reduces the latency by up to 44.3%, in comparison with existing GPU virtualization solutions.

/pdf/g-net-effective-gpu-sharing-in-nfv-systems-2nqyvu2fvc.pdf

G-NET: Effective GPU Sharing in NFV Systems

Core-to-core communication is critical to the effective use of multi-core processors. A number of software based concurrent lock-free queues have been proposed to address this problem. Existing solutions, however, suffer from performance degradation in real testbeds, or rely on auxiliary hardware or software timers to handle the deadlock problem when batching is used, making those solutions good in theory but difficult to use in practice. This paper describes the pros and cons of existing concurrent lock-free queues in both dummy and real testbeds and proposes B-Queue, an efficient and practical single-producer-single-consumer concurrent lock-free queue that solves the deadlock problem gracefully by introducing a self-adaptive backtracking mechanism. Experiments show that in real massively-parallel applications, B-Queue is faster than FastForward and MCRingBuffer, the two state-of-the-art concurrent lock-free queues, by up to 10x and 5x, respectively. Moreover, B-Queue outperforms FastForward and MCRingBuffer in terms of stability and scalability, making it a good candidate for fast core-to-core communication on multi-core architectures.

/pdf/b-queue-efficient-and-practical-queuing-for-fast-core-to-1attayjkks.pdf

B-Queue: Efficient and Practical Queuing for Fast Core-to-Core Communication

Long passwords are gaining popularity in password policy recommendations; however, data-driven guessing studies are woefully inadequate in adapting to long passwords, lacking in both guessing efficiency and their composition guidelines. For state-of-the-art data-driven password guessing methods such as PCFGs (Probabilistic Context-free Grammars), their guessing efficiency is limited by the presence of a large scale training data, or the lack thereof. Given that long passwords leaked in the real world are typically scarce, coupled with the fact that the data-driven methods’ performance depends on training data, obtaining good performance on long passwords has become a key challenge. To overcome the dataset limitation, we propose a framework TransPCFG , that transfers the knowledge, (i.e., grammars in PCFGs), from short passwords to facilitate long password guessing. We further perform an empirical evaluation based on three real-world datasets and the results demonstrate superior performance over the state-of-the-art data-driven guessing methods under ${10}^{14}$ offline guesses. For passwords with 16 characters, TransPCFG can compromise an average of 23.30% of the passwords, outperforming PCFG_v4.1 by 56.10%. Additionally,for better password-composition guidelines, we find that long password-composition policies requiring more segments are more resistant to guessing attacks. For the segment, the password 12zxcvbnword1997 has four segments since it follows the template ${Digit}_{2}{Keyboard}_{6}{Letter}_{4}{Year}_{4}$ . We thus recommend users to create long passwords with four or more segments instead of the widely recommended more character classes for security.

Kai Zhang

Papers

Mega-KV: a case for GPUs to maximize the throughput of in-memory key-value stores

Concurrent analytical query processing with GPUs

G-NET: Effective GPU Sharing in NFV Systems

B-Queue: Efficient and Practical Queuing for Fast Core-to-Core Communication

TransPCFG : Transferring the Grammars From Short Passwords to Guess Long Passwords Effectively