With the advent of Grid and application technologies, scientists and engineers are building more and more complex applications to manage and process large data sets, and execute scientific experiments on distributed resources. Such application scenarios require means for composing and executing complex workflows. Therefore, many efforts have been made towards the development of workflow management systems for Grid computing. In this paper, we propose a taxonomy that characterizes and classifies various approaches for building and executing workflows on Grids. We also survey several representative Grid workflow systems developed by various projects world-wide to demonstrate the comprehensiveness of the taxonomy. The taxonomy not only highlights the design and engineering similarities and differences of state-of-the-art in Grid workflow systems, but also identifies the areas that need further research.

A Taxonomy of Workflow Management Systems for Grid Computing

Cloud computing environments facilitate applications by providing virtualized resources that can be provisioned dynamically. However, users are charged on a pay-per-use basis. User applications may incur large data retrieval and execution costs when they are scheduled taking into account only the ‘execution time’. In addition to optimizing execution time, the cost arising from data transfers between resources as well as execution costs must also be taken into account. In this paper, we present a particle swarm optimization (PSO) based heuristic to schedule applications to cloud resources that takes into account both computation cost and data transmission cost. We experiment with a workflow application by varying its computation and communication costs. We compare the cost savings when using PSO and existing ‘Best Resource Selection’ (BRS) algorithm. Our results show that PSO can achieve: a) as much as 3 times cost savings as compared to BRS, and b) good distribution of workload onto resources.

A Particle Swarm Optimization-Based Heuristic for Scheduling Workflow Applications in Cloud Computing Environments

We present the ASKALON environment whose goal is to simplify the development and execution of workflow applications on the Grid. ASKALON is centered around a set of high-level services for transparent and effective Grid access, including a Scheduler for optimized mapping of workflows onto the Grid, an Enactment Engine for reliable application execution, a Resource Manager covering both computers and application components, and a Performance Prediction service based on training phase and statistical methods. A sophisticated XML-based programming interface that shields the user from the Grid middleware details allows the high-level composition of workflow applications. ASKALON is used to develop and port scientific applications as workflows in the Austrian Grid project. We present experimental results using two real-world scientific applications to demonstrate the effectiveness of our approach.

/pdf/askalon-a-grid-application-development-and-computing-1ixjykai0h.pdf

ASKALON: A Grid Application Development and Computing Environment

Modern scientific applications often need to be distributed across Grids. Increasingly applications rely on services, such as job submission, data transfer or data portal services. We refer to such services as Grid services. While the invocation of Grid services could be hard coded in theory, scientific users want to orchestrate service invocations more flexibly. In enterprise applications, the orchestration of web services is achieved using emerging orchestration standards, most notably the Business Process Execution Language (BPEL). We describe our experience in orchestrating scientific workflows using BPEL. We have gained this experience during an extensive case study that orchestrates Grid services for the automation of a polymorph prediction application. Using this example, we explain the extent with which the BPEL language supports the definition of scientific workflows. We then describe the reliability, performance and scalability that can be achieved by executing a complex scientific workflow with ActiveBPEL, an industrial strength but freely available BPEL engine.

/pdf/grid-service-orchestration-using-the-business-process-1naefbuafs.pdf

Grid Service Orchestration Using the Business Process Execution Language (BPEL)

Process management is an extremely important concept in both business and scientific communities. Several workflow management tools have been proposed in recent years offering advanced functionality in various domains. In the business world, workflow vendors offer commercial and customized solutions targeting specific users. In the scientific world, several open-source workflow management tools are freely available. However they are directed toward service aggregation rather than distributed process management. Little consideration is given to the needs of the client in terms of mapping the process flow of the client. In the grid community it is essential that the grid users have such a tool available enabling them to orchestrate complex work-flows on the fly without substantial help from the service providers. At the same time it is important that the grid user not be burdened with the intricacies of the workflow system. With the perspective of the grid user in mind, an extensible client-side workflow management system, called GridAnt, has been developed. This paper discusses the design principles, functionality, and application of the proposed GridAnt workflow manager.

/pdf/gridant-a-client-controllable-grid-workflow-system-2dqaj7wz9m.pdf

A. Rossi

Papers

GridAnt: a client-controllable grid workflow system