Proceedings ArticleDOI
GoldRush: resource efficient in situ scientific data analytics using fine-grained interference aware execution
Fang Zheng,Hongfeng Yu,Can Hantas,Matthew Wolf,Greg Eisenhauer,Karsten Schwan,Hasan Abbasi,Scott Klasky +7 more
- pp 78
Reads0
Chats0
TLDR
Experiments with representative science applications at large scales show that resources harvested on compute nodes can be leveraged to perform useful analytics, significantly improving resource efficiency, reducing data movement costs incurred by alternate solutions, and posing negligible impact on scientific simulations.Abstract:
Severe I/O bottlenecks on High End Computing platforms call for running data analytics in situ. Demonstrating that there exist considerable resources in compute nodes un-used by typical high end scientific simulations, we leverage this fact by creating an agile runtime, termed GoldRush, that can harvest those otherwise wasted, idle resources to efficiently run in situ data analytics. GoldRush uses fine-grained scheduling to "steal" idle resources, in ways that minimize interference between the simulation and in situ analytics. This involves recognizing the potential causes of on-node resource contention and then using scheduling methods that prevent them. Experiments with representative science applications at large scales show that resources harvested on compute nodes can be leveraged to perform useful analytics, significantly improving resource efficiency, reducing data movement costs incurred by alternate solutions, and posing negligible impact on scientific simulations.read more
Citations
More filters
Fast parallel algorithms for short-range molecular dynamics
TL;DR: Comparing the results to the fastest reported vectorized Cray Y-MP and C90 algorithm shows that the current generation of parallel machines is competitive with conventional vector supercomputers even for small problems.
Proceedings ArticleDOI
In-situ feature extraction of large scale combustion simulations using segmented merge trees
Aaditya G. Landge,Valerio Pascucci,Attila Gyulassy,Janine C. Bennett,Hemanth Kolla,Jacqueline H. Chen,Peer-Timo Bremer +6 more
TL;DR: This paper presents two variants of in-situ feature extraction techniques using segmented merge trees, which encode a wide range of threshold based features, and presents a fast, low communication cost technique that generates an exact solution but has limited scalability.
Proceedings ArticleDOI
Performance analysis, design considerations, and applications of extreme-scale in situ infrastructures
Utkarsh Ayachit,Andrew Bauer,Earl P. N. Duque,Greg Eisenhauer,Nicola J. Ferrier,Junmin Gu,Kenneth E. Jansen,B. Loring,Zarija Lukić,Suresh Menon,Dmitriy Morozov,Patrick O'Leary,Reetesh Ranjan,Michel Rasquin,Christopher Stone,Venkat Vishwanath,Gunther H. Weber,Brad Whitlock,Matthew Wolf,K. John Wu,E. Wes Bethel +20 more
TL;DR: This paper examines several key design and performance issues related to the idea of in situ processing at extreme scale on modern platforms: scalability, overhead, performance measurement and analysis, comparison and contrast with a traditional post hoc approach, and interfacing with simulation codes.
Proceedings ArticleDOI
Smart: a MapReduce-like framework for in-situ scientific analytics
TL;DR: A novel MapReduce-like framework that supports efficient in-situ scientific analytics, and is designed so that analytics can be launched from the parallel code region of a simulation program.
Proceedings ArticleDOI
Achieving Performance Isolation with Lightweight Co-Kernels
TL;DR: Pisces is presented, a system software architecture that enables the co-existence of multiple independent and fully isolated OS/Rs, or enclaves, that can be customized to address the disparate requirements of next generation HPC workloads.
References
More filters
Journal ArticleDOI
Fast parallel algorithms for short-range molecular dynamics
TL;DR: In this article, three parallel algorithms for classical molecular dynamics are presented, which can be implemented on any distributed-memory parallel machine which allows for message-passing of data between independently executing processors.
Fast parallel algorithms for short-range molecular dynamics
TL;DR: Comparing the results to the fastest reported vectorized Cray Y-MP and C90 algorithm shows that the current generation of parallel machines is competitive with conventional vector supercomputers even for small problems.
Proceedings ArticleDOI
Condor-a hunter of idle workstations
TL;DR: The design, implementation, and performance of the Condor scheduling system, which operates in a workstation environment, are presented and a performance profile of the system is presented that is based on data accumulated from 23 stations during one month.
Journal ArticleDOI
Addressing shared resource contention in multicore processors via scheduling
TL;DR: This study is the first to provide a comprehensive analysis of contention-mitigating techniques that use only scheduling, and finds a classification scheme that addresses not only contention for cache space, but contention for other shared resources, such as the memory controller, memory bus and prefetching hardware.
Proceedings ArticleDOI
Hybrid MPI/OpenMP Parallel Programming on Clusters of Multi-Core SMP Nodes
TL;DR: It is shown that machine topology has a signi¿cant impact on performance for all parallelization strategies and that topology awareness should be built into all applications in the future.