Showing papers by "Sanjeev Setia published in 1999"

Availability and utility of idle memory in workstation clusters

Abstract: In this paper, we examine the availability and utility of idle memory in workstation clusters. We attempt to answer the following questions. First, how much of the total memory in a workstation cluster can be expected to be idle? This provides an estimate of the opportunity for hosting guest data. Second, how much memory can be expected to be idle on individual workstations? This helps determine the recruitment policy – how much memory should be recruited on individual hosts? Third, what is the distribution of memory idle-times? This indicates how long guest data can be expected to survive; applications that access their data-sets frequently within the expected life-time of guest data are more likely to benefit from exploiting idle memory. Fourth, how much performance improvement can be achieved for off-the-shelf clusters without customizing the operating system and/or the processor firmware? Finally, how long and how frequently might a user have to wait to reclaim her machine if she volunteers to host guest pages on her machine? This helps answer the question of social acceptability. To answer the questions relating to the availability of idle memory, we have analyzed two-week long traces from two workstation pools with different sizes, locations, and patterns of use. To evaluate the expected benefits and costs, we have simulated five data-intensive applications (0.5 GB-5 GB) on these workstation pools.

Dodo: a user-level system for exploiting idle memory in workstation clusters

Abstract: In this paper, we present the design and implementation of Dodo, an efficient user-level system for harvesting idle memory in off-the-shelf clusters of workstations. Dodo enables data-intensive applications to use remote memory in a cluster as an intermediate cache between local memory and disk. It requires no modifications to the operating system and/or processor firmware and is hence portable to multiple platforms. Further, the memory recruitment policy used by Dodo is designed to minimize any delays experienced by the owner of desktop machines whose memory is harvested by Dodo. Our implementation of Dodo is operational and currently runs on Linux 2.0.35. For communication, Dodo can use either UDP/IP or U-Net, the low-latency user-level network architecture developed by von Eicken et al. (1995). We evaluated the performance improvements that can be achieved by using Dodo for two real applications and three synthetic benchmarks. Our results show that speedups obtained for an application are highly dependent on its I/O access pattern and data set sizes. Significant speedups (between 2 and 3) were obtained for applications whose working sets are larger than the local memory on a workstation but smaller than aggregate memory available on the cluster and for applications that can benefit from the zero-seek nature of remote memory.

The impact of job memory requirements on gang-scheduling performance

Abstract: Almost all previous research on gang-scheduling has ignored the impact of real job memory requirements on the performance of the policy. This is despite the fact that on parallel supercomputers, because of the problems associated with demand paging, executing jobs are typically allocated enough memory so that their entire address space is memory-resident. In this paper, we examine the impact of job memory requirements on the performance of gang-scheduling policies. We first present an analysis of the memory-usage characteristics of jobs in the production workload on the Cray T3E at the San Diego Supercomputer Center. We also characterize the memory usage of some of the applications that form part of the workload on the LLNL ASCI supercomputer. Next, we examine the issue of long-term scheduling on MPPs, i.e., we study policies for deciding which jobs among a set of competing jobs should be allocated memory and thus should be allowed to execute on the processors of the system. Using trace-driven simulation, we evaluate the impact of using different long-term scheduling policies on the overall performance of Distributed Hierarchical Control (DHC), a gang-scheduling policy that has been studied extensively in the research literature.