Kan Qiao

TL;DR: This paper proposes a monitoring-based weakly consistent resource stealing technique to achieve resource balancing in distributed HPC job launch, and implements the technique in Slurm++, a distributed workload manager directly extended from theSlurm centralized production system.

TL;DR: A distributed storage middleware right on the compute nodes, which allows I/O to effectively leverage the high bi-section bandwidth of the high-speed interconnect of massively parallel high-end computing systems, and a 2-phase mechanism to cache the hot data for parallel applications, called 2-Layer Scheduling (2LS).

TL;DR: An analytical suboptimal upper bound is devised of the proposed data‐aware work‐stealing technique to optimize both load balancing and data locality and results show that the technique is not only scalable but can achieve performance within 15% of theSuboptimal solution.

TL;DR: A stochastic performance model is built to quantify the impact of failures on MapReduce applications and to investigate its effectiveness under different computing environments, and results show that data replication is an effective approach even when failure rate is high, and the task migration mechanism of Map Reduce works well in balancing the reliability difference among individual nodes.

TL;DR: This paper introduces a concept called virtual chunks aiming to support efficient random accesses to the compressed scientific data without sacrificing its compression ratio, and implemented virtual chunks in two forms: a middleware to the GPFS parallel file system, and a module in the FusionFS distributed file system.