SciNet, one of seven regional HPC consortia operating under the Compute Canada umbrella, runs Canada's first and third fastest computers (as of June 2010) in a state-of-the-art, highly energy-efficient datacentre with a Power Usage Effectiveness (PUE) design-point of 1.16. Power efficiency, computational bang for the buck and system capability for a handful of flagship science projects were important criteria in choosing the nature of the computers and the data centre itself. Here we outline some of the lessons learned in putting together the systems and the data centre that hosts Canada's fastest computer to date.

http://iopscience.iop.org/article/10.1088/1742-6596/256/1/012026/pdf

SciNet: Lessons Learned from Building a Power-efficient Top-20 System and Data Centre

In the last decade Grid Computing Technology, an innovative extension of distributed computing, is becoming an enabler for computing resource sharing among the participants in "Virtual Organisations" (VO) [1]. Although there exist enormous research efforts on grid-based collaboration technologies, most of them are concentrated on large research and business institutions. In this paper we are considering the adoption of Grid Computing Technology in a VO of small to medium Further Education (FE) and Higher-Education (HE) institutions. We will concentrate on the resource sharing among the campuses of The University of Huddersfield in Yorkshire and colleges in Lancashire, UK, enabled by the Grid. Within this context, it is important to focus on standards that support resource and information sharing, toolkits and middleware solutions that would promote Grid adoption among the FE/HE institutions in the Virtual HE organisation.

Huddersfield University Campus Grid: QGG of OSCAR Clusters

Checkpointing is a fault-tolerance mechanism commonly used in High Throughput Computing (HTC) environments to allow the execution of long-running computational tasks on compute resources subject to hardware and software failures and interruptions from resource owners. With increasing scrutiny of the energy consumption of IT infrastructures, it is important to understand the impact of checkpointing on the energy consumption of HTC environments. In this paper we demonstrate through trace-driven simulation on real-world datasets that existing checkpointing strategies are inadequate at maintaining an acceptable level of energy consumption whilst reducing the makespan of tasks. Furthermore, we identify factors important in deciding whether to employ checkpointing within an HTC environment, and propose novel strategies to curtail the energy consumption of checkpointing approaches.

/pdf/on-energy-efficient-checkpointing-in-high-throughput-cycle-1qnanbi76y.pdf

On Energy-efficient Checkpointing in High-throughput Cycle-stealing Distributed Systems

This paper targets the challenges of research data management with a focus on High Performance Computing (HPC) and simulation data. Main challenges are discussed: The Big Data qualities of HPC research data, technical data management, organizational and administrative challenges. Emerging from these challenges, requirements for a feasible HPC research data management are derived and an alternative data life cycle is proposed. The requirement analysis includes recommendations which are based on a modified OAIS architecture: To meet the HPC requirements of a scalable system, metadata and data must not be stored together. Metadata keys are defined and organizational actions are recommended. Moreover, this paper contributes by introducing the role of a Scientific Data Manager, who is responsible for the institution’s data management and taking stewardship of the data.

Challenges of Research Data Management for High Performance Computing

The deluge of dark data is about to happen. Lacking data management capabilities, especially in the field of supercomputing, and missing data documentation (i.e., missing metadata annotation) constitute a major source of dark data. The present work contributes to addressing this challenge by presenting ExtractIng, a generic automated metadata extraction toolkit. Existing metadata information of simulation output files scattered through the file system, can be aggregated, parsed and converted to the EngMeta metadata model. Use cases from computational engineering are considered to demonstrate the viability of ExtractIng. The evaluation results show that the metadata extraction is simulation-code independent in the sense that it can handle data outputs from various fields of science, is easy to integrate into simulation workflows and compatible with a multitude of computational environments.

Like a rainbow in the dark: metadata annotation for HPC applications in the age of dark data

In this paper we present a cluster middleware, designed to implement a Linux-Windows Hybrid HPC Cluster, which not only holds the characteristics of both operating system, but also accepts and schedules jobs in both environments. Beowulf Clusters have become an economical and practical choice for small-and-medium-sized institutions to provide High Performance Computing (HPC)resources. The HPC resources are required for running simulations, image rendering and other calculations, and to support the software requiring a specific operating system. To support the software, small-scale computer clusters would have to be divided in two or more clusters if they are to run on a single operating system. The x86 virtualisation technology would help running multiple operating systems on one computer, but only with the latest hardware which many legacy Beowulf clusters do not have. To aid the institutions, who rely on legacy non-virtualisation-supported facilities rather than high-end HPC resources, we have developed and deployed a bi-stable hybrid system built around Linux CentOS 5.5 with the improved OSCAR middleware, and Windows Server 2008 and Windows HPC 2008 R2. This hybrid cluster is utilised as part of the University of Hudders field campus grid.

/pdf/hybrid-computer-cluster-with-high-flexibility-37ebsajm5l.pdf

Hybrid Computer Cluster with High Flexibility

The advent of open source software leading to Beowulf clusters has enabled small to medium sized Higher and Further education institutions to remove the “computational power” factor from research ventures. In an effort to catch up with leading Universities in the realm of research, many Universities are investing in small departmental HPC clusters to help with simulations, renders and calculations. These small HE/FE institutions have in the past benefited from cheaper software and operating system licenses. This raises the question as to which platform Linux of Windows should be implemented on the cluster. As the smaller/medium Universities move into research, many Linux based applications and code better suit their research needs, but the teaching base still keeps the department tied to Windows based applications. In such institutions, where it is usually recycled machines that are linked to form the clusters, it is not often feasible to setup more than one cluster.
This paper will propose a method to implement a Linux-Windows Hybrid HPC Cluster that seamlessly and automatically accepts and schedules jobs in both domains. Using Linux CentOS 5.4 with OSCAR 5.2 beta 2 middleware with Windows Server 2008 and Windows HPC 2008 R2 (beta) a bi-stable hybrid system has been deployed at the University of Huddersfield. This hybrid cluster is known as the Queensgate Cluster. We will also examine innovative solutions and practices that are currently being followed in the academic world as well as those that have been recommended by Microsoft® Corp.

Hybrid HPC – Establishing a Bi-Stable Dual Boot Cluster for Linux with OSCAR middleware and Windows HPC 2008 R2

Scientific research activities generate a large amount of data, which varies in format, volume, structure and ownership. Although there are revision control systems and databases developed for data archiving, the traditional data management methods are not suitable for High-Performance Computing (HPC) systems. The files in such systems do not have semantic annotations and cannot be archived and managed for public dissemination.

/pdf/icurate-a-research-data-management-system-2f9djj1wal.pdf

iCurate: A Research Data Management System

High Throughput Computing (HTC) systems are designed to utilise available resources on a network of idle machines in an institution or organization by cycle stealing. It provides an additional ‘free’ resource from the existing computing and networking infrastructure for modelling and simulation requiring a large number of small jobs, such as applications from biology, chemistry, physics, and digital signal processing. At the University of Huddersfield, there are thousands of idle laboratory machines that could be used to run serial/parallel jobs by cycle stealing. Our HTC system, implemented in Condor [1], is part of the Queensgate Campus Grid (QGG) [2] that consists of a number of dedicated departmental and university computer clusters.
Condor is an excellent HTC tool that excels in cycle stealing and job scheduling on idle machines. However, only idle powered machines can be used from a networked pool. Many organizations deploy power saving mechanisms to try to reduce energy consumption in their systems, and power down idle resources, using rigid and inflexible power management policies. The University of Huddersfield Computing Services use the Energy Star EZ GPO power saving tool that runs as a Windows service and detects how long the computer has been idle. Then it allows the computer to first turn off the screen and then go into hibernation.
Our research and development work is focused on implementing a HTC system using Condor to work within a “green IT” policy of a higher education institutions that conform to green IT challenges for a multi-platform, multi-discipline user/ resource base. This system will allow Condor to turn on machines that may have gone to sleep due to lack of usage when there is a large queue of pending jobs. The decision to utilise dormant resources will be made on a variety of factors such as job priority, job requirements, user priority, time of day, flocking options, queue conditions etc. Good practice scheduling policies would need to be devised that would work within this “green IT” pool.

Shuo Liang

Papers

Hybrid Computer Cluster with High Flexibility

Hybrid HPC – Establishing a Bi-Stable Dual Boot Cluster for Linux with OSCAR middleware and Windows HPC 2008 R2

iCurate: A Research Data Management System

Implementing a Condor pool using a Green-IT policy