Emerging scale-out workloads require extensive amounts of computational resources. However, data centers using modern server hardware face physical constraints in space and power, limiting further expansion and calling for improvements in the computational density per server and in the per-operation energy. Continuing to improve the computational resources of the cloud while staying within physical constraints mandates optimizing server efficiency to ensure that server hardware closely matches the needs of scale-out workloads.In this work, we introduce CloudSuite, a benchmark suite of emerging scale-out workloads. We use performance counters on modern servers to study scale-out workloads, finding that today's predominant processor micro-architecture is inefficient for running these workloads. We find that inefficiency comes from the mismatch between the workload needs and modern processors, particularly in the organization of instruction and data memory systems and the processor core micro-architecture. Moreover, while today's predominant micro-architecture is inefficient when executing scale-out workloads, we find that continuing the current trends will further exacerbate the inefficiency in the future. In this work, we identify the key micro-architectural needs of scale-out workloads, calling for a change in the trajectory of server processors that would lead to improved computational density and power efficiency in data centers.

/pdf/clearing-the-clouds-a-study-of-emerging-scale-out-workloads-4tg9zvrf1a.pdf

Clearing the clouds: a study of emerging scale-out workloads on modern hardware

As architecture, systems, and data management communities pay greater attention to innovative big data systems and architecture, the pressure of benchmarking and evaluating these systems rises. However, the complexity, diversity, frequently changed workloads, and rapid evolution of big data systems raise great challenges in big data benchmarking. Considering the broad use of big data systems, for the sake of fairness, big data benchmarks must include diversity of data and workloads, which is the prerequisite for evaluating big data systems and architecture. Most of the state-of-the-art big data benchmarking efforts target evaluating specific types of applications or system software stacks, and hence they are not qualified for serving the purposes mentioned above.

http://prof.ict.ac.cn/BigDataBench/old/2.0/wp-content/uploads/2013/10/Wang_BigDataBench.pdf

BigDataBench: A big data benchmark suite from internet services

The use of virtualization technologies in high performance computing (HPC) environments has traditionally been avoided due to their inherent performance overhead. However, with the rise of container-based virtualization implementations, such as Linux VServer, OpenVZ and Linux Containers (LXC), it is possible to obtain a very low overhead leading to near-native performance. In this work, we conducted a number of experiments in order to perform an in-depth performance evaluation of container-based virtualization for HPC. We also evaluated the trade-off between performance and isolation in container-based virtualization systems and compared them with Xen, which is a representative of the traditional hypervisor-based virtualization systems used today.

/pdf/performance-evaluation-of-container-based-virtualization-for-4o4kfjpd6a.pdf

Performance Evaluation of Container-Based Virtualization for High Performance Computing Environments

MapReduce systems face enormous challenges due to increasing growth, diversity, and consolidation of the data and computation involved. Provisioning, configuring, and managing large-scale MapReduce clusters require realistic, workload-specific performance insights that existing MapReduce benchmarks are ill-equipped to supply. In this paper, we build the case for going beyond benchmarks for MapReduce performance evaluations. We analyze and compare two production MapReduce traces to develop a vocabulary for describing MapReduce workloads. We show that existing benchmarks fail to capture rich workload characteristics observed in traces, and propose a framework to synthesize and execute representative workloads. We demonstrate that performance evaluations using realistic workloads gives cluster operator new ways to identify workload-specific resource bottlenecks, and workload-specific choice of MapReduce task schedulers. We expect that once available, workload suites would allow cluster operators to accomplish previously challenging tasks beyond what we can now imagine, thus serving as a useful tool to help design and manage MapReduce systems.

http://yanpeichen.com/professional/MapReduceWorkloadMASCOTSCameraReady.pdf

The Case for Evaluating MapReduce Performance Using Workload Suites

A big data approach for logistics trajectory discovery from RFID-enabled production data

The MapReduce model is becoming prominent for the large-scale data analysis in the cloud. In this paper, we present the benchmarking, evaluation and characterization of Hadoop, an open-source implementation of MapReduce. We first introduce HiBench, a new benchmark suite for Hadoop. It consists of a set of Hadoop programs, including both synthetic micro-benchmarks and real-world Hadoop applications. We then evaluate and characterize the Hadoop framework using HiBench, in terms of speed (i.e., job running time), throughput (i.e., the number of tasks completed per minute), HDFS bandwidth, system resource (e.g., CPU, memory and I/O) utilizations, and data access patterns.

The HiBench benchmark suite: Characterization of the MapReduce-based data analysis

Although Big Data Cloud (e.g., MapReduce, Hadoop and Dryad) makes it easy to develop and run highly scalable applications, efficient provisioning and fine-tuning of these massively distributed systems remain a major challenge. In this paper, we describe a general approach to help address this challenge, based on distributed instrumentations and dataflow-driven performance analysis. Based on this approach, we have implemented HiTune, a scalable, lightweight and extensible performance analyzer for Hadoop. We report our experience on how HiTune helps users to efficiently conduct Hadoop performance analysis and tuning, demonstrating the benefits of dataflow-based analysis and the limitations of existing approaches (e.g., system statistics, Hadoop logs and metrics, and traditional profiling).

/pdf/hitune-dataflow-based-performance-analysis-for-big-data-q6fci04ddz.pdf

HiTune: dataflow-based performance analysis for big data cloud

High-quality I/O virtualization (that is, complete device semantics, full-feature set, close-to-native performance and real-time response) is critical to both server and client virtualizations. Existing solutions for I/O virtualization (e.g., full device emulation, paravirtualization and direct I/O) cannot meet the requirements of high-quality I/O virtualization due to high overheads, lack of complete semantic or full-feature set support.We have developed new techniques for high-quality I/O virtualization (including device semantic preservation, essential principles for avoiding device virtualization holes, and real-time VMM scheduler extensions), using direct I/O with hardware IOMMU. It not only meets the requirements of high quality I/O virtualization, but also is the basis for PCI-SIG I/O Virtualization (IOV). Experimental results show that our implementation can achieve up-to 98% of the native performance and up to 3.6X of the paravirtualization performance. In addition, it can improve the real-time-ness of the latency-sensitive application by up to 4.8X with the scheduler extensions.

Towards high-quality I/O virtualization

Modern network processors employs parallel processing engines (PEs) to keep up with explosive internet packet processing demands. Most network processors further allow processing engines to be organized in a pipelined fashion to enable higher processing throughput and flexibility. In this paper, we present a novel program transformation technique to exploit parallel and pipelined computing power of modern network processors. Our proposed method automatically partitions a sequential packet processing application into coordinated pipelined parallel subtasks which can be naturally mapped to contemporary high-performance network processors. Our transformation technique ensures that packet processing tasks are balanced among pipeline stages and that data transmission between pipeline stages is minimized. We have implemented the proposed transformation method in an auto-partitioning C compiler product for Intel Network Processors. Experimental results show that our method provides impressive speed up for the commonly used NPF IPv4 forwarding and IP forwarding benchmarks. For a 9-stage pipeline, our auto-partitioning C compiler obtained more than 4X speedup for the IPv4 forwarding PPS and the IP forwarding PPS (for both the IPv4 traffic and IPv6 traffic).

Automatically partitioning packet processing applications for pipelined architectures

Automatic software controlled caching generations in network applications are described herein. In one embodiment, a candidate representing a plurality of instructions of a plurality of threads that perform one or more external memory accesses is identified, where the external memory accesses have a substantially identical base address. One or more directives and/or instructions are inserted into an instruction stream corresponding to the identified candidate to maintain contents of at least one of a content addressable memory (CAM) and local memory (LM) of a processor, and to modify at least one of the external memory access to access at least one of the CAM and LM of the processor without having to perform the respective external memory access. Other methods and apparatuses are also described.

Jinquan Dai

Papers

The HiBench benchmark suite: Characterization of the MapReduce-based data analysis

HiTune: dataflow-based performance analysis for big data cloud

Towards high-quality I/O virtualization

Automatically partitioning packet processing applications for pipelined architectures

Automatic caching generation in network applications