Currently Gridapplication developers often configure available application components intoa workflow oftasks that they cansubmit forexecuting on theGrid. Inthis paper, wepresent anAbstract Grid Workflow Language (AGWL)fordescribing Grid workflow applications atahighlevel ofabstraction. AGWL hasbeendesigned suchthattheusercan concentrate onspecifying Gridapplications without dealing witheither thecomplexity oftheGridorany specific implementation technology (e.g. Webservice). AGWL isanXML-based language whichallows a programmer todefine agraph ofactivities thatrefer mostly tocomputational tasks. Activities areconnected bycontrol anddata flowlinks. A rich setofconstructs (compound activities) isprovided tosimplify the specification ofGridworkflow applications which includes compound activities suchasif,forEach andwhi1e loops aswellasadvanced compound activities including parallel sections, parallel loops andcollection iterators. Moreover, AGWLsupports a generic highlevelaccessmechanism to data repositories. AGWL isthemaininterface tothe ASKALONGridapplication development environment andhasbeenapplied tonumerousrealworld applications. Wedescribe amaterial science workflow thathasbeensuccessfully portedto a Grid infrastructure based onanAGWLspecification. Only a dozenAGWL activities areneededtodescribe a workflow withseveral hundred activity instances.

Specification ofGridWorkflow Applications withAGWL:AnAbstract Grid Workflow Language*

HyperFlow: A model of computation, programming approach and enactment engine for complex distributed workflows

This paper presents HyperFlow: an approach to workflow programming which combines the advantages of a declarative workflow description and low-level scripting programming. The workflow execution model of HyperFlow is based on a formal model of computation -- Process Networks. The execution environment is implemented on the basis of a widely adopted runtime platform node.js. Workflow programming benefits from such an approach in multiple ways, including leveraging a large programming ecosystem with many developers, reusable software packages and learning resources; elimination of shim nodes from the workflow graph; and increased reusability of workflow processing components. The HyperFlow workflow programming approach and its advanced capabilities are presented. The HyperFlow engine is also briefly described. Four example workflow applications from various domains, including flood threat assessment, are studied as a demonstration of the HyperFlow programming approach and a comparison with related solutions.

Increasing scientific workflow programming productivity with HyperFlow

Composing, execution and sharing of multiscale applications

The artic le presents a technology for dynamic knowledge -based building of Do main -Specific Languages (DSL) to describe data-intensive scientific discovery tasks using BigData technology The proposed technology supports high level abstract definit ion of analytic and simulat ion parts of the task as well as integration into the composite scientific solutions Automatic translation of the abstract task definition enables seamless integration of various data sources within single solution

A Technology for BigData Analysis Task Description using Domain-Specific Languages

For programming and executing complex applications on grid infrastructures, scientific workflows have been proposed as convenient high-level alternative to solutions based on general-purpose programming languages, APIs and scripts. GridSpace is a collaborative programming and execution environment, which is based on a scripting approach and it extends Ruby language with a high-level API for invoking operations on remote resources. In this paper we describe a tool which enables to convert the GridSpace application source code into a workflow representation which, in turn, may be used for scheduling, provenance, or visualization. We describe how we addressed the issues of analyzing Ruby source code, resolving variable and method dependencies, as well as building workflow representation. The solutions to these problems have been developed and they were evaluated by testing them on complex grid application workflows such as CyberShake, Epigenomics and Montage. Evaluation is enriched by representing typical workflow control flow patterns.

/pdf/constructing-workflows-from-script-applications-e4rkvx3ug5.pdf

Constructing workflows from script applications

Workflow management systems are widely used to solve scientific problems as they enable orchestration of remote and lo- cal services such as database queries, job submission and running an application. To extend the role that workflow systems play in data-intensive science, we propose a solution that integrates WMS and MapReduce model. In this paper, we discuss possible solution of combining MapReduce and workflow applications, we describe the implementation of chosen solution based on metaprogramming approach in Ruby programming language and evaluate it with an example of word count application.

/pdf/mapreduce-operations-with-ws-vlam-workflow-management-system-4vxg1faigj.pdf

MapReduce Operations with WS-VLAM Workflow Management System

Cloud service providers support many different standards in authentication and resource management. Libraries and APIs provided to operate in a cloud environment are not consistent and require deep understanding of internal technical aspects. Thus, we propose a framework that provides high-level language to develop applications and, thanks to its layered structure, API to modify low-level operation.

Cookery: A framework for developing cloud applications

Applying suitable application models is essential to achieve efficient execution of the applications, effective development process and assure application portability and reusability. We observe that execution environments and the cloud environment in particular, lack of tools that would make it more available from a programmer perspective. Complexity and variety of libraries and authentication methods do not correspond with simplistic functionality of those services. We approach this issue at several fields - workflow systems, domain specific languages and cloud computing. Our work focus on two points, optimizing existing application models by supplementing them with additional tools and providing new features, and develop new models that are based on our research on applications in the cloud environment.

Data and process abstractions for cloud computing

Bibliography [1] A. Belloum, M. Inda, D. Vasunin, V. Korkhov, Z. Zhao, H. Rauwerda, T. Breit, M. Bubak, L. Hertzberger, Collaborative e-science experiments and scientific workflows, Internet Computing, IEEE 15 (2011) 39–47. [2] C. Olston, B. Reed, U. Srivastava, R. Kumar, A. Tomkins, Pig Latin: a not-so-foreign language for data processing, in: Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data, SIGMOD ’08, ACM, New York, NY, USA, 2008, 1099–1110. [3] R. Pike, S. Dorward, R. Griesemer, S. Quinlan, Interpreting the data: Parallel analysis with Sawzall, Scientific Programming 13 (2005) 277. [4] M. Baranowski, A. Belloum, M. Bubak, M. Malawski, Constructing workflows from script applications, Scientific Programming 20 (2012) 359–377 [5] M. Baranowski, A. Belloum, M. Bubak, MapReduce operations with WS-VLAM Workflow Management System, Procedia Computer Science 18 (2013) 2599-2602 Objective Provide an easy to use and efficient Domain Specific Language for defining MapReduce operations in workflows.

/pdf/enhancing-vlam-workflow-model-with-mapreduce-operations-1v286cxtvz.pdf

Mikolaj Baranowski

Papers

Constructing workflows from script applications

MapReduce Operations with WS-VLAM Workflow Management System

Cookery: A framework for developing cloud applications

Data and process abstractions for cloud computing

Enhancing VLAM Workflow Model with MapReduce Operations