GreenGPU: A Holistic Approach to Energy Efficiency in GPU-CPU Heterogeneous Architectures

TLDR: This paper proposes Green GPU, a holistic energy management framework for GPU-CPU heterogeneous architectures that dynamically throttles the frequencies of GPU cores and memory in a coordinated manner, based on their utilizations, for maximized energy savings with only marginal performance degradation.

Abstract: In recent years, GPU-CPU heterogeneous architectures have been increasingly adopted in high performance computing, because of their capabilities of providing high computational throughput. However, the energy consumption is a major concern due to the large scale of such kind of systems. There are a few existing efforts that try to lower the energy consumption of GPU-CPU architectures, but they address either GPU or CPU in an isolated manner and thus cannot achieve maximized energy savings. In this paper, we propose Green GPU, a holistic energy management framework for GPU-CPU heterogeneous architectures. Our solution features a two-tier design. In the first tier, Green GPU dynamically splits and distributes workloads to GPU and CPU based on the workload characteristics, such that both sides can finish approximately at the same time. As a result, the energy wasted on idling and waiting for the slower side to finish is minimized. In the second tier, Green GPU dynamically throttles the frequencies of GPU cores and memory in a coordinated manner, based on their utilizations, for maximized energy savings with only marginal performance degradation. Likewise, the frequency and voltage of the CPU are scaled similarly. We implement Green GPU using the CUDA framework on a real physical test bed with Nvidia GeForce GPUs and AMD Phenom II CPUs. Experiment results show that Green GPU achieves 21.04% average energy savings and outperforms several well-designed baselines.