Characterizing and profiling scientific workflows

Cloud computing is the latest distributed computing paradigm and it offers tremendous opportunities to solve large-scale scientific problems. However, it presents various challenges that need to be addressed in order to be efficiently utilized for workflow applications. Although the workflow scheduling problem has been widely studied, there are very few initiatives tailored for cloud environments. Furthermore, the existing works fail to either meet the user's quality of service (QoS) requirements or to incorporate some basic principles of cloud computing such as the elasticity and heterogeneity of the computing resources. This paper proposes a resource provisioning and scheduling strategy for scientific workflows on Infrastructure as a Service (IaaS) clouds. We present an algorithm based on the meta-heuristic optimization technique, particle swarm optimization (PSO), which aims to minimize the overall workflow execution cost while meeting deadline constraints. Our heuristic is evaluated using CloudSim and various well-known scientific workflows of different sizes. The results show that our approach performs better than the current state-of-the-art algorithms.

/pdf/deadline-based-resource-provisioningand-scheduling-algorithm-1u7jr6hf6b.pdf

Deadline Based Resource Provisioningand Scheduling Algorithm for Scientific Workflows on Clouds

Deadline-constrained workflow scheduling algorithms for Infrastructure as a Service Clouds

Resource scheduling in cloud is a challenging job and the scheduling of appropriate resources to cloud workloads depends on the QoS requirements of cloud applications. In cloud environment, heterogeneity, uncertainty and dispersion of resources encounters problems of allocation of resources, which cannot be addressed with existing resource allocation policies. Researchers still face troubles to select the efficient and appropriate resource scheduling algorithm for a specific workload from the existing literature of resource scheduling algorithms. This research depicts a broad methodical literature analysis of resource management in the area of cloud in general and cloud resource scheduling in specific. In this survey, standard methodical literature analysis technique is used based on a complete collection of 110 research papers out of large collection of 1206 research papers published in 19 foremost workshops, symposiums and conferences and 11 prominent journals. The current status of resource scheduling in cloud computing is distributed into various categories. Methodical analysis of resource scheduling in cloud computing is presented, resource scheduling algorithms and management, its types and benefits with tools, resource scheduling aspects and resource distribution policies are described. The literature concerning to thirteen types of resource scheduling algorithms has also been stated. Further, eight types of resource distribution policies are described. Methodical analysis of this research work will help researchers to find the important characteristics of resource scheduling algorithms and also will help to select most suitable algorithm for scheduling a specific workload. Future research directions have also been suggested in this research work.

A Survey on Resource Scheduling in Cloud Computing: Issues and Challenges

Techniques are described for managing the execution of programs on a plurality of computing systems, such as computing systems organized into multiple groups. A program execution service manages the program execution on behalf of multiple customers or other users, and selects appropriate computing systems to execute one or more instances of program, such as based in part on locations of one or more previously stored copies of the program from which copies of the program to execute may be acquired. For example, in some situations the selection of an appropriate computing system to execute an instance of a program is based in part on physical or logical proximity to other resources, such as stored copies of the program, executing copies of the program, and/or available computing systems.

Managing execution of programs by multiple computing systems

Cost optimized provisioning of elastic resources for application workflows

Recently grid computing has started to leverage Web services technology by proposing OGSI-standard. OGSI standard defines the grid service, which presents unified interfaces to every participant of grid. In current Glubus Toolkit3(GT3),which is an implementation of OGSI, grid service factories should be deployed manually into resources to provide grid services. However, it is necessary to dynamically allocate proper amount of resource, since the demand for resource of service provider changes over time. In this paper, we propose a architecture to enable on-demand resource provisioning. We develop universal factory service (UFS) that provides a dynamic grid service deployment mechanism and a resource broker called door service. Through the experiments, we show that grid services can adaptively exploit resources according to the request rates.

A dynamic grid services deployment mechanism for on-demand resource provisioning

https://cs.kaist.ac.kr/upload_files/report/1282287447.pdf

BTS: Resource capacity estimate for time-targeted science workflows

Workflow technologies have become a major vehicle for the easy and efficient development of science applications At the same time new computing environments such as the Cloud are now available A challenge is to determine the right amount of resources to provision for an application This paper introduces an algorithm named balanced time scheduling (BTS), which estimates the minimum number of virtual processors required to execute a workflow within a user-specified finish time The resource estimate of BTS is abstract, so it can be easily integrated with any resource description language or any resource provisioning system The experimental results with a number of synthetic workflows demonstrate that BTS can estimate the computing capacity close to the optimal The algorithm is scalable so that its turnaround time is only tens of seconds even with workflows having thousands of tasks and edges

/pdf/estimating-resource-needs-for-time-constrained-workflows-215avmpgys.pdf

Estimating Resource Needs for Time-Constrained Workflows

Large-scale Grid is a computing environment composed of Internet-wide distributed resources shared by a number of applications. Although WSRF and Java-based hosting environment can successfully deal with the heterogeneity of resources and the diversity of applications, the current Grid systems have several limitations to support the dynamic nature of large-scale Grid. This paper proposes DynaGrid, a new framework for building large-scale Grid for WSRF-compliant applications. Compared to the existing Grid systems, DynaGrid provides three new mechanisms: dynamic service deployment, resource migration, and transparent request dispatching. Two core components, ServiceDoor and dynamic service launcher (DSL), have been implemented as WSRF-compliant Web services to realize DynaGrid, which are applicable to any Java-based WSRF hosting environment. We construct a real testbed with DynaGrid on the Globus Toolkit 4 and evaluate the effectiveness of our framework using two practical applications. The evaluation results show that dynamic service deployment and resource migration in DynaGrid bring many advantages to large-scale Grid in terms of performance and reliability with minimal overhead.

/pdf/dynagrid-a-dynamic-service-deployment-and-resource-migration-4b8wp4ifbg.pdf

Eun-Kyu Byun

Papers

Cost optimized provisioning of elastic resources for application workflows

A dynamic grid services deployment mechanism for on-demand resource provisioning

BTS: Resource capacity estimate for time-targeted science workflows

Estimating Resource Needs for Time-Constrained Workflows

DynaGrid: A dynamic service deployment and resource migration framework for WSRF-compliant applications