Effective management of ICT (information and communications technology) and cooling is critical in modern data centres for high energy efficiency. This survey paper gives an overview of the joint optimization between ICT and cooling management under conventional air-cooled technology in the data centre. We first review the enabling techniques of ICT and cooling management in the data centre, which provide the opportunity to dynamically control the server utilization and operate cooling equipment. We then present the coupling models between ICT and cooling management, which are the basis of the optimization approaches for green data centre. The joint optimization of ICT and cooling management is considered under a set of performance metrics, including thermal requirement, power consumption, and application delay. We summarize the workload scheduling algorithms designed from the optimization approaches. We also present some testbeds in the data centres. Finally, we discuss on the future trends of the management in the data centre.

/pdf/towards-joint-optimization-over-ict-and-cooling-systems-in-1yuxngvnfx.pdf

Towards Joint Optimization Over ICT and Cooling Systems in Data Centre: A Survey

Process variation—deviation in parameters from their nominal specifications—threatens to slow down and even pause technological scaling, and mitigation of it is the way to continue the benefits of chip miniaturization. In this article, we present a survey of architectural techniques for managing process variation (PV) in modern processors. We also classify these techniques based on several important parameters to bring out their similarities and differences. The aim of this article is to provide insights to researchers into the state of the art in PV management techniques and motivate them to further improve these techniques for designing PV-resilient processors of tomorrow.

https://dl.acm.org/doi/pdf/10.1145/2871167

A Survey of Architectural Techniques for Managing Process Variation

Exascale computing represents the next leap in the HPC race. Reaching this level of performance is subject to several engineering challenges such as energy consumption, equipment-cooling, reliability and massive parallelism. Model-based optimization is an essential tool in the design process and control of energy efficient, reliable and thermally constrained systems. However, in the Exascale domain, model learning techniques tailored to the specific supercomputer require real measurements and must therefore handle and analyze a massive amount of data coming from the HPC monitoring infrastructure. This becomes rapidly a “big data” scale problem. The common approach where measurements are first stored in large databases and then processed is no more affordable due to the increasingly storage costs and lack of real-time support. Nowadays instead, cloud-based machine learning techniques aim to build on-line models using real-time approaches such as “stream processing” and “in-memory” computing, that avoid storage costs and enable fastdata processing. Moreover, the fast delivery and adaptation of the models to the quick data variations, make the decision stage of the optimization loop more effective and reliable. In this paper we leverage scalable, lightweight and flexible IoT technologies, such as the MQTT protocol, to build a highly scalable HPC monitoring infrastructure able to handle the massive sensor data produced by next-gen HPC components. We then show how state-of-the art tools for big data computing and analysis, such as Apache Spark, can be used to manage the huge amount of data delivered by the monitoring layer and to build adaptive models in real-time using on-line machine learning techniques.

/pdf/continuous-learning-of-hpc-infrastructure-models-using-big-24w1i42ieo.pdf

Continuous learning of HPC infrastructure models using big data analytics and in-memory processing tools

Power consumption is a critical aspect for next generation High Performance Computing systems: Supercomputers are expected to reach Exascale in 2023 but this will require a significant improvement in terms of energy efficiency. In this domain, power-capping can significant increase the final energy-efficiency by cutting cooling effort and worst-case design margins. A key aspect for an optimal implementation of power capping is the ability to estimate the power consumption of HPC applications before they run on the real system. In this paper we propose a Machine-Learning approach, based on the user and application resource request, to accurately predict the power consumption of typical supercomputer workloads. We demonstrate our method on real production workloads executed on the Eurora supercomputer hosted at CINECA computing center in Bologna and we provide useful insights to apply our technique in other installations.

Predictive Modeling for Job Power Consumption in HPC Systems

Power consumption is a key factor in modern ICT infrastructure, especially in the expanding world of High Performance Computing, Cloud Computing and Big Data. Such consumption is bound to become an even greater issue as supercomputers are envisioned to enter the Exascale by 2020, granted that they obtain an order of magnitude energy efficiency gain. An important component in many strategies devised to decrease energy usage is "power capping", i.e., the possibility to constrain the system power consumption within certain power budget. In this paper we propose two novel approaches for power capped workload dispatching and we demonstrate them on a real-life high-performance machine: the Eurora supercomputer hosted at CINECA computing center in Bologna. Power capping is a feature not included in the commercial Portable Batch System PBS dispatcher currently in use on Eurora. The first method is based on a heuristic technique while the second one relies on a hybrid strategy which combines a CP and a heuristic approach. Both systems are evaluated and compared on simulated job traces.

/pdf/power-capping-in-high-performance-computing-systems-5g1lj6vu69.pdf

Power capping in high performance computing systems

Eurora (EURopean many integrated cORe Architecture) is today the most energy efficient supercomputer in the world. Ranked 1st in the Green500 in July 2013, is a prototype built from Eurotech and Cineca toward next-generation Tier-0 systems in the PRACE 2IP EU project. Eurora's outstanding energy-efficiency is achieved by adopting a direct liquid cooling solution and a heterogeneous architecture with best-in-class general purpose HW components (Intel Xeon E5, Intel Xeon Phi and NVIDIA Kepler K20). In this paper we present a novel, low-overhead monitoring infrastructure capable to track in detail and in real-time the thermal and power characteristics of Eurora's components with fine-grained resolution. Our experiments give insights on Eurora's thermal/power trade-offs and highlight opportunities for run-time power/thermal management and optimization.

Unveiling eurora - thermal and power characterization of the most energy-efficient supercomputer in the world

Supercomputers, nowadays, aggregate a large number of nodes sharing the same nominal HW components (eg. processors and GPGPUS). In real-life machines, the chips populating each node are subject to a wide range of variability sources, related to performance and temperature operating points (i.e. ACPI p-states) as well as process variations and die binning. Eurora is a fully operational supercomputer prototype that topped July 2013 Green500 and it represents a unique ‘living lab’ for next-generation ultra-green supercomputers. In this paper we evaluate and quantify the impact of variability on Eurora's energy-performance tradeoffs under a wide range of workload intensity.

Quantifying the impact of variability on the energy efficiency for a next-generation ultra-green supercomputer

Hot-water liquid cooling is a key technology in future green supercomputers as it maximizes the cooling efficiency and energy reuse. However the cooling system still is responsible for a significant percentage of modern HPC power consumption. Standard design of liquid-cooling control relies on rules based on worst-case scenarios, or on CFD simulation of portion of the entire system, which cannot account for all the real supercomputer working conditions (workload and ambient temperature). In this work we first introduce an analytical model, based on lumped parameters, which can effectively describe the cooling components and dynamics, and can be used for analysis and control purposes. We then use it to design an energy-optimal control strategy which is capable to minimize the pump and chiller power consumption while, meeting the supercomputer cooling requirements. We validate the method with simulation tests, taking data from a real HPC cooling mechanism, and comparing the results with state-of-the-art commercial cooling system control strategies.

Giampietro Tecchiolli

Papers

Unveiling eurora - thermal and power characterization of the most energy-efficient supercomputer in the world

Quantifying the impact of variability on the energy efficiency for a next-generation ultra-green supercomputer

Energy-aware cooling for hot-water cooled supercomputers