As both CPUs and GPUs become employed in a wide range of applications, it has been acknowledged that both of these Processing Units (PUs) have their unique features and strengths and hence, CPU-GPU collaboration is inevitable to achieve high-performance computing. This has motivated a significant amount of research on heterogeneous computing techniques, along with the design of CPU-GPU fused chips and petascale heterogeneous supercomputers. In this article, we survey Heterogeneous Computing Techniques (HCTs) such as workload partitioning that enable utilizing both CPUs and GPUs to improve performance and/or energy efficiency. We review heterogeneous computing approaches at runtime, algorithm, programming, compiler, and application levels. Further, we review both discrete and fused CPU-GPU systems and discuss benchmark suites designed for evaluating Heterogeneous Computing Systems (HCSs). We believe that this article will provide insights into the workings and scope of applications of HCTs to researchers and motivate them to further harness the computational powers of CPUs and GPUs to achieve the goal of exascale performance.

https://dl.acm.org/doi/pdf/10.1145/2788396

A Survey of CPU-GPU Heterogeneous Computing Techniques

As preservers of history and the objects that others leave behind, it is easy to get distracted by a desire for those objects to be “perfect.” We worry about every little tear, stain or blemish, and we want nothing more than to return objects to their "original state." However, so many times those rough spots are part of the item’s history. It is important to understand the difference between the types of condition issues on objects that cause structural issues versus those that are purely cosmetic. Knowing and identifying these differences can help when determining whether or not an item requires conservation or deciding simple factors like the best way to store or exhibit an item.

Facebook“卖流量”新招

https://www.ime.usp.br/~kon/papers/Polato_JNCA_2014.pdf

A comprehensive view of Hadoop research—A systematic literature review

The work presented here is driven by two observations. First, heterogeneous architectures that integrate a CPU and a GPU on the same chip are emerging, and hold much promise for supporting power-efficient and scalable high performance computing. Second, MapReduce has emerged as a suitable framework for simplified parallel application development for many classes of applications, including data mining and machine learning applications that benefit from accelerators. This paper focuses on the challenge of scaling a MapReduce application using the CPU and GPU together in an integrated architecture. We develop different methods for dividing the work, which are the map-dividing scheme, where map tasks are divided between both devices, and the pipelining scheme, which pipelines the map and the reduce stages on different devices. We develop dynamic work distribution schemes for both the approaches. To achieve high load balance while keeping scheduling costs low, we use a runtime tuning method to adjust task block sizes for the map-dividing scheme. Our implementation of MapReduce is based on a continuous reduction method, which avoids the memory overheads of storing key-value pairs. We have evaluated the different design decisions using 5 popular MapReduce applications. For 4 of the applications, our system achieves 1.21 to 2.1 speedup over the better of the CPU-only and GPU-only versions. The speedups over a single CPU core execution range from 3.25 to 28.68. The runtime tuning method we have developed achieves very low load imbalance, while keeping scheduling overheads low. Though our current work is specific to MapReduce, many underlying ideas are also applicable towards intra-node acceleration of other applications on integrated CPU-GPU nodes.

Accelerating MapReduce on a coupled CPU-GPU architecture

Large graph processing is now a critical component of many data analytics. Graph processing is used from social networking web sites that provide context-aware services from user connectivity data to medical informatics that diagnose a disease from a given set of symptoms. Graph processing has several inherently parallel computation steps interspersed with synchronization needs. Graphics processing units (GPUs) are being proposed as a power-efficient choice for exploiting the inherent parallelism. There have been several efforts to efficiently map graph applications to GPUs. However, there have not been many characterization studies that provide an in-depth understanding of the interaction between the GPGPU hardware components and graph applications that are mapped to execute on GPUs. In this study, we compiled 12 graph applications and collected the performance and utilization statistics of the core components of GPU while running the applications on both a cycle accurate simulator and a real GPU card. We present detailed application execution characteristics on GPUs. Then, we discuss and suggest several approaches to optimize GPU hardware for enhancing the graph application performance.

/pdf/graph-processing-on-gpus-where-are-the-bottlenecks-nx5jio7kkv.pdf

Graph processing on GPUs: Where are the bottlenecks?

MapReduce is a programming model that enables efficient massive data processing in large-scale computing environments such as supercomputers and clouds. Such large-scale computers employ GPUs to enjoy its good peak performance and high memory bandwidth. Since the performace of each job is depending on running application characteristics and underlying computing environments, scheduling MapReduce tasks onto CPU cores and GPU devices for efficient execution is difficult. To address this problem, we have proposed a hybrid scheduling technique for GPU-based computer clusters, which minimizes the execution time of a submitted job using dynamic profiles of Map tasks running on CPU cores and GPU devices. We have implemented a prototype of our proposed scheduling technique by extending MapReduce framework, Hadoop. We have conducted some experiments for this prototype by using a K-means application as a benchmark on a supercomputer. The results show that the proposed technique achieves 1.93 times faster than the Hadoop original scheduling algorithm at 64 nodes (1024 CPU cores and 128 GPU devices). The results also indicate that the performance of map tasks, including both CPU and GPU tasks, is significantly affected by the overhead of map task invocation in the Hadoop framework.

/pdf/hybrid-map-task-scheduling-for-gpu-based-heterogeneous-4n76adnhxk.pdf

Hybrid Map Task Scheduling for GPU-Based Heterogeneous Clusters

GPUs can accelerate edge scan performance of graph processing applications; however, the capacity of device memory on GPUs limits the size of graph to process. Efficient techniques to handle GPU memory overflows, including overflow detection and performance analysis in large-scale systems, are not well investigated. To address the problem, we propose a MapReduce-based out-of-core GPU memory management technique for processing large-scale graph applications on heterogeneous GPU-based supercomputers. Our proposed technique automatically handles memory overflows from GPUs by dynamically dividing graph data into multiple chunks and overlaps CPU-GPU data transfer and computation on GPUs as much as possible. Our experimental results on TSUBAME2.5 using 1024 nodes (12288 CPU cores, 3072 GPUs) exhibit that our GPU-based implementation performs 2.10x faster than running on CPU when graph data size does not fit on GPUs. We also study the performance characteristics of our proposed out-of-core GPU memory management technique, including application’s performance and power efficiency of scale-up and scale-out approaches in terms of the number of GPUs. Keywords—Large-scale Graph Processing; GPGPU; MapReduce; Out-of-core Algorithms; Big Data Applications

/pdf/out-of-core-gpu-memory-management-for-mapreduce-based-large-51pmay6q9d.pdf

Out-of-core GPU memory management for MapReduce-based large-scale graph processing

Training deep neural networks requires a large amount of memory, making very deep neural networks difficult to fit on accelerator memories. In order to overcome this limitation, we present a method to reduce the amount of memory for training a deep neural network. The method enables to suppress memory increase during the backward pass, by reusing the memory regions allocated for the forward pass. Experimental results exhibit our method reduced the occupied memory size in training by 44.7% on VGGNet with no accuracy affection. Our method also enabled training speedup by increasing the mini batch size up to double.

Memory reduction method for deep neural network training

Fast processing for extremely large-scale graph is becoming increasingly important in various domains such as health care, social networks, intelligence, system biology, and electric power grids. The GIM-V algorithm based on MapReduce programing model is designed as a general graph processing method for supporting petabyte-scale graph data. On the other hand, recent large-scale data-intensive computing systems tend to employ GPU accelerators to gain good peak performance and high memory bandwidth, however, the validity of acceleration, including optimization techniques, of the GIM-V algorithm using GPUs is an open problem. To address the problem, we implemented a multi-GPU-based GIM-V application with load balance optimization between GPU devices. Our implementation extends the existing MapReduce library for supporting multi-GPU-environments using the MPI library and optimizes load balance between GPU devices by employing task scheduling-based graph partitioning. We conducted our implementation on the TSUBAME2.0 supercomputer using 256 nodes (6144 hyper-threaded CPU cores, 768 GPUs). The results exhibit that our GPU-based implementation performed 87.04 ME/s on 230 (1.07 billion) vertices and 234 (17.2 billion) edges, and 1.52 times faster than the CPU-based naive implementation with 2^29 vertices and 233 edges. We also studied the performance characteristics of our implementation and load balance optimization technique.

A scalable implementation of a mapreduce-based graph processing algorithm for large-scale heterogeneous supercomputers

We realized a speed-up of an object detection neural network with GPU. We improved the object detection speed of faster R-CNN [1], which is one of the most commonly used detection networks [2]. The speed of the original faster R-CNN (py - faster - rcnn [3]) was 72.4ms per image on our GPU server 11OS: Ubuntu 14.04.5 LTS (GNU/Linux 4.2.0-42-generic x86_64), CPU: Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz, GPU: GPU (Tesla P100-PCIE-16GB) x 1, Libraries: MKL, CUDA 8.0, cuDNN v5.1.5. We accelerated the detection speed by implementing our new algorithms that are suitable for GPUs. The speed-up is realized without sacrificing the object detection accuracy (mAP). Our GPU-accelerated faster R-CNN can detect objects with 55.8ms per image. This is nearly 30% speed-up. In detection networks, the processes of building scored candidate regions, sorting and non-maximum-suppression (nms) are commonly used. In faster R-CNN, these processes are executed in proposal layer. We reduced the processing time of the proposal layer from 5.6ms to 2.2ms. This is 2.5 times as fast as the original one. We also evaluated the detection speed with larger batch sizes. By applying batch size 16, it is accelerated to 44.9ms per image. This is 1.6 times as fast as the original faster R-CNN (py-faster-rcnn). Since we realized a speed-up of common basic methods for detection networks, our speed-up methods are also applicable to other detection networks such as R-FCN [4], YOLO [5] [6] and SSD [7].

Koichi Shirahata

Papers

Hybrid Map Task Scheduling for GPU-Based Heterogeneous Clusters

Out-of-core GPU memory management for MapReduce-based large-scale graph processing

Memory reduction method for deep neural network training

A scalable implementation of a mapreduce-based graph processing algorithm for large-scale heterogeneous supercomputers

Speed-Up of Object Detection Neural Network with GPU