Showing papers by "Enis Afgan published in 2008"

Embarrassingly parallel jobs are not embarrassingly easy to schedule on the grid

Abstract: Embarrassingly parallel applications represent an important workload in today's grid environments. Scheduling and execution of this class of applications is considered mostly a trivial and well-understood process on homogeneous clusters. However, while grid environments provide the necessary computational resources, associated resource heterogeneity represents a new challenge for efficient task execution for these types of applications across multiple resources. This paper presents a set of examples illustrating how execution characteristics of individual tasks, and consequently a job, are affected by the choice of task execution resources, task invocation parameters, and task input data attributes. It is the aim of this work to highlight this relationship between an application and an execution resource to promote development of better metascheduling techniques for the grid. By exploiting this relationship, application throughput can be maximized, also resulting in higher resource utilization. In order to achieve such benefits, a set of job scheduling and execution concerns is derived leading toward a computational pipeline for scheduling embarrassingly parallel applications in grid environments.

Experiences with developing and deploying dynamic BLAST

Abstract: Basic Local Alignment Search Tool (BLAST) is a heavily used bioinformatics application that has gotten significant attention from the high performance computing community. The authors have taken BLAST execution a step further and enabled it to execute on grid resources. Adapting BLAST to execute on the grid brings up concerns regarding grid resource heterogeneity, which inevitably cause difficulty with application availability, fault tolerance, interoperability, and variability in performance of individual segments that are being distributed throughout grid resources. In addition difficulties arise because of tools, technologies, and middleware dependencies that an application developer must deal with. This paper describes Dynamic BLAST and experiences with developing and deploying the application on grid resources over a two year period. Dynamic BLAST is a BLAST-specific metascheduler, a multithreaded, master-worker type application that handles all aspects of a BLAST job submission on the grid for the user. It was developed with the goal of bringing the grid closer to a typical scientist by eliminating the initial learning curve necessary for use of many grid applications. Associated research and development have resulted in the authors' extensive experience in dealing with grid related issues with respect to available tools and technologies. Lessons learned and suggestions are also presented in this paper.

A common application platform for the SURAgrid (CAP)

Abstract: From our experience in developing and deploying research applications on a regional grid infrastructure (SURAgrid, www.sura.org/suragrid), we observe that there are significant entry-barriers to "grid-enable" applications, and therefore, to realize the full benefit of a grid environment. In order to increase both the number and the variety of applications that can run on the SURAgrid, we propose to develop an integrated environment that can directly support MPI-based* applications on a subset of SURAgrid resources. Our goal is to emulate the environment of a single Beowulf-style cluster on SURAgrid for easy access by MPI-based distributed-memory applications that are readily available in almost all fields of science and engineering. Although a single application can engineer a similar environment and performance with the various grid services, the creation of a persistent environment such as CAP can significantly reduce the complexity for deploying an application while extending the benefits to a much larger set of applications.This paper describes the architecture and initial implementation of the Common Application Platform (CAP) on SURAgrid. CAP provides an integrated platform for scheduling and execution of sequential and parallel jobs on the CAP-enabled resources in a user-friendly environment. In particular, we aim to provide the following capabilities:Meta-scheduling: Co-scheduling capability is needed for applications with large-scale memory requirements that can be met only by simultaneous use of resources at multiple sites. Automatic resource selections across multiple sites will enhance the overall utilization of the grid. The scheduling and job management capabilities in CAP are provided by the GridWay metascheduler.Orchestration: For parallel applications on CAP, the orchestration capabilities are provided by MPICH-G2, a grid-enabled version of the popular MPI implementation -- MPICH. Both GridWay and MPICH-G2 depend on the Globus Toolkit to provide the basic grid functionalities.Fast data transfer: High-performance networks, such as the National Lambda Rail that connects many of SURAgrid resources can be exploited by performing striped (parallel) file transfers, and will be explored in this project for enhancing inter-cluster communications.There are significant issues and challenges in several areas affecting practical deployment of CAP: load balancing, routing, resource heterogeneity, and network performance. In order to explore these issues, a prototype involving the SURAgrid resources at Old Dominion University and University of Alabama at Birmingham will be developed. The prototype leverages existing software solutions in a coordinated infrastructure to minimize development efforts, utilize SURAgrid infrastructure to provide simplified job control through the SURAgrid portal, and enable inter-institutional resource allocation through the SURAgrid authentication and authorization mechanism.This paper will disseminate lessons learned through the construction of this prototype as well as share experiences and perspectives towards next-step development. The goal of CAP is to provide immediately useful benefits to the growing SURAgrid application community in a way that also serves as a model for effective integration of existing technologies for the grid use and development community at large.

G-BLAST: A Grid Service for BLAST.

Abstract: This paper described the design and implementation of G-BLAST – a Grid Service for one of the most widely used bioinformatics application Basic Local Alignment Search Tool (BLAST). G-BLAST uses the factory design pattern to provide application developers a common interface to incorporate multiple implementations of BLAST. The process of application selection, resource selection, scheduling, and monitoring is completely hidden from the end-user through the web-based user interfaces and the programmatic interfaces enable users to employ G-BLAST as part of a bioinformatics pipeline. G-BLAST uses an adaptive scheduler to select the best application and the best set of resources available that will provide the shortest turnaround time when executed in a grid environment. G-BLAST is successfully deployed on a campus and regional grid and several BLAST applications are tested for different combinations of input parameters and computational resources. Experimental results illustrate the overall performance improvements obtained with G-BLAST.