scispace - formally typeset

Journal ArticleDOI

A component-based framework for certification of components in a cloud of HPC services

01 Jun 2020-Science of Computer Programming (Elsevier)-Vol. 191, pp 102379

TL;DR: A Verification-as-a-Service (VaaS) framework for component certification on HPC Shelf is presented, aimed at providing higher confidence that components of parallel computing systems of HPC shelf behave as expected according to one or more requirements expressed in their contracts.

AbstractHPC Shelf is a proposal of a cloud computing platform to provide component-oriented services for High Performance Computing (HPC) applications. This paper presents a Verification-as-a-Service (VaaS) framework for component certification on HPC Shelf . Certification is aimed at providing higher confidence that components of parallel computing systems of HPC Shelf behave as expected according to one or more requirements expressed in their contracts. To this end, new abstractions are introduced, starting with certifier components. They are designed to inspect other components and verify them for different types of functional, non-functional and behavioral requirements. The certification framework is naturally based on parallel computing techniques to speed up verification tasks.

Topics: Cloud computing (61%), Certification (54%), Component-based software engineering (54%), Component (UML) (53%), Supercomputer (52%)

Summary (8 min read)

1. Introduction

  • HPC Shelf is a cloud computing platform aimed at addressing domain-specific, computationally intensive problems typically emerging from computational science and engineering domains.
  • In HPC Shelf, they must be able to identify and combine components to form parallel computing systems.
  • Through the proposed framework, components called certifiers may use a set of different certification tools to certify that the components of parallel computing systems meet a certain set of requirements.
  • The case studies used to demonstrate the proposed certification framework are particularly focused on functional and behavioral requirements that can be verified through automated verification methods and tools, such as theorem provers and model checkers.
  • From this assessment, the following outstanding features and contributions have been identified in favor of the certification framework of HPC Shelf: .

2. HPC Shelf

  • HPC Shelf is a cloud computing platform that provides HPC services for providers of domain-specific applications.
  • An application is a problem-solving environment through which specialist users, the end users of HPC Shelf, specify problems and obtain computational solutions for them.
  • It is assumed that these solutions are computationally intensive, thus demanding the use of large-scale parallel computing infrastructure, i.e. comprising multiple parallel computing platforms engaged in a single computational task.
  • Applications generate computational solutions as component-oriented parallel computing systems.
  • To do so, these components comply to Hash [8], a parallel component model whose components may exploit parallel processing in distributed-memory parallel computing platforms.

2.1. Component kinds of parallel computing systems

  • Component platforms that comply to the Hash component model distinguish components according to a set of component kinds.
  • Action bindings connect a set of action ports belonging to computation and connector components.
  • It may be programmed by using a general-purpose programming language (currently, C#) or SAFeSWL (SAFe Scientific Workflow Language), an XML-based orchestration language designed for activating the computational tasks of the solution components in a prescribed order [1].
  • In a MapReduce parallel computing system, mappers, reducers, splitters, and shufflers may be connected through bindings between their compatible collect_pairs and feed_pairs ports.
  • In turn, the connectors have only the read_chunk/finish_chunk and chunk_ready action names, distributed in their facets.

2.3. Stakeholders

  • The following stakeholders work around HPC Shelf: The specialists (end users) use applications for specifying problems using a domain-specific interface.
  • They do not handle directly with components, which are hidden behind the domain-specific abstractions of the application interface.
  • The providers create and deploy applications, by designing their high-level interfaces and by programming the generation of parallel computing systems.
  • For that, they are experts in parallel computer architectures and parallel programming.
  • Through contextual contracts, they may specify the architectural features of the virtual platforms they support.

2.4. Architecture

  • The multilayer cloud architecture of HPC Shelf for servicing applications comprises the three elements in Fig. 5: Frontend, Core and Backend.
  • The Frontend is SAFe (Shelf Application Framework) [1], a collection of classes and design patterns used by providers to build applications.
  • It supports SAFeSWL as a language for specifying parallel computing systems.
  • In turn, using the orchestration subset, the provider may program its workflow, by specifying the order in which action names must be activated.
  • Applications access the services of the Core for resolving contextual contracts and deploying the selected components on virtual platforms.

2.5. Contextual contracts

  • Since both abstract components named Mapper and Reducer are derived from MRComputation, function is used to specify the custom map and reduce functions that they will execute in particular MapReduce computations.
  • A contextual contract is an abstract component whose context parameters have particular execution context assumptions associated to each one of them.
  • In the classification phase, the list of candidate system components is ordered taking into account the best fulfillment of the contract requirements and the resource allocation policies of HPC Shelf.

3. The certification framework

  • For the purpose of leveraging component certification in HPC Shelf, a certification framework is introduced in this section.
  • It encompasses a set of component kinds, composition rules and design patterns.
  • They provide an environment where certification tools can be encapsulated into components to provide some level of assurance to application providers and component developers that components of parallel computing systems meet a predetermined set of requirements prior to their instantiation.
  • Each certifier associated with a certifiable component may impose its own set of obligations on compatible certifiable components, such as the use of certain programming languages, design patterns, code conventions, annotations, etc.
  • The service interface determines which kind of ad hoc properties are supported and how they are specified.

3.1. Parallel certification systems

  • Certifier components are implemented as parallel certification systems, comprising the following architectural elements, as depicted in Fig. 6: A set of tactical components; A certification-workflow component that orchestrates the tactical ones; A set of bindings, connecting the tactical components to the certification-workflow component.
  • The certification-workflow component performs a certification procedure on the certifiable components connected to the certifier.
  • Parallel certification systems are analogous to parallel computing systems, but aimed at certification purposes.
  • In turn, tactical components play the role of solution components.
  • For this reason, they must be seen as special kinds of virtual platforms on which the proof infrastructure is installed and ready to run verification tasks.

3.2. Tactical components

  • As stated earlier, a tactical component encapsulates a certification infrastructure comprising one or more certification tools.
  • An action port with action names perform, conclusive and inconclusive, where the latter two are alternative; .
  • When the certification subroutine terminates, either conclusive or inconclusive may be activated by the tactical component.
  • In the current implementation, the contextual signature of Tactical, the component type from which specific tactical components are derived, is similar to that of EnvironmentPortCertification.

3.3. Certifier components

  • In the orchestration code, an activation of the action certify will instantiate a parallel certification system for each certifier, which will certificate the certifiable component in parallel.
  • Each one may be reused to certify all certifiable components associated with the same certifier, when their certify actions are activated.
  • After the certification procedure, a certifiable component is considered certified if all default, contractual, component and ad hoc properties have been checked by the certifier.
  • Regarding the action ports, the certification-workflow component has ports to be connected to the action ports of its tactical components.
  • The units may run on different data partitions and synchronize by exchanging messages in a discipline akin to the MPI programming model [15].

4.1. Tactical components for C4

  • The verification of computation components may resort to two different classes of methods and tools.
  • The first class is based on deductive program verification, which partially automate axiomatic program verification based on some variant of the Floyd-Hoare logic.
  • The alternative approach explores the space of reachable states of a system through model checking.

4.1.1. Deductive tactical components

  • Tactical components for deductive verification require the target component programs to be annotated with assertions in the style of the Floyd-Hoare logic or its extensions, namely, separation logic [16], for mutable data structures, Owicki-Gries reasoning [17], for shared-memory parallel programs, and Apt’s reasoning [18], for distributed-memory components.
  • Actually, only ParTypes [19] can verify C/MPI programs, annotated in the syntax of VCC [20], against a high-level communication protocol stored by the certifier as an ad hoc property.
  • In such a case, they are equipped with reasonably complex interfaces for editing, searching and choosing suitable proof procedures and heuristics.
  • Using this approach, the application may either automatically interact with the tactical component or require some intervention of the specialist user to proceed the verification subroutine.

4.1.2. Model checking tactical components

  • Model checking provides a powerful alternative to deductive verification tools to establish properties of MPI programs.
  • In the context of the certification framework discussed in this article, the following tools were explored: ISP (In-situ Partial Order) [34] and CIVL (Concurrency Intermediate Verification Language) [35].
  • Both verify a fixed , although sufficiently expressive set of safety properties.
  • The former handles deadlock absence, assertion violations, MPI object leaks, and communication races (i.e. unexpected communication matches) in components written in C, C++ or C#, carrying MPI/OpenMP directives.
  • CIVL, on its turn, is also able to establish functional equivalence between programs and is able to discharge verification conditions to the provers Z3, CVC3 and CVC4.

4.2. Contextual contracts and architecture

  • Thus, C4 certifiers may prescribe the host programming language on which the computation component is written, as well as the message passing library for communication between the units of the computation component.
  • Also, certifiers can determine whether or not the ad hoc properties are supported.
  • The tactical components of C4MPIComplex are ISP, JWFA and CZ, whose abstract components restrict the bounds of the context parameters of Tactical to define the interface types through which they talk to the certification-workflow component.
  • Like in most scientific workflows management systems, such as Askalon [37], BPEL Sedna [38], Kepler [39], Pegasus [40], Taverna [41] and Triana [42], SAFeSWL workflows are usually represented by components and execution dependencies among them, usually adopting abstract descriptions of components and abstracting away from the computing platforms on which they run.
  • At an appropriate time of the workflow execution, a resolution procedure may be triggered for discovering an appropriate component implementation, making it relevant to ensure that the activation of computational actions of components is made after their effective resolution.

5.2. Architecture and contextual contracts

  • SWC2 prescribes two default properties: Deadlock absence; Obedience to the protocol in which lifecycle actions must be activated, for each component, presented in Section 2.1.
  • It supports all the default properties prescribed for SWC2 certifiers, as well as ad hoc properties.
  • There are two concrete components of mCRL2Certifier.
  • MCRL2Certifier certifiers may be able to exploit parallelism by initiating different verification tasks on distinct processing nodes of the tactical component.
  • For simplifying formulas, mCRL2 allows the use of regular expressions over the set of actions as possible labels of both necessity and eventuality modalities.

5.3.1. The translation process

  • The translation process follows directly the operational rules (Fig. 10) defined for an abstract grammar of a formal specification of the orchestration subset of SAFeSWL (Fig. 9).
  • Rule finish indicates that an action asynchronously activated can actually occur, having its handle registered in F and emitting an action to the system ((a,h)).
  • Rules select-left and select-right indicate the need for the creation of mCRL2 processes that control the state (enabled/disabled) of actions.
  • Rules repeat, continue and break indicate, respectively, the need for the creation of a mCRL2 process that manages a repetition task in order to detect the need for a new iteration, the return back to the beginning of the iteration, or the end of the iteration.
  • The first part of the conjunction states that a deploy may not be performed before a resolve.

6. Case studies

  • Three case studies demonstrate the certification framework of HPC Shelf, as well as the use of C4 and SWC2 certifiers.
  • The workflow maintains three versions of each image, which overlap the center of the Pleiades cluster, each corresponding to a different color band: red, infrared and blue.
  • Fig. 12(a) presents the architecture of a single subworkflow.
  • To do this, the application provider must configure certification bindings between these component instances and one or more instances of C4MPISimple.
  • Fig. 15 reports execution times for this certification case study, by varying the number of processing nodes and cores per node involved in the execution of the tactical component ISP.11.

6.2.1. The non-iterative system with three stages

  • They are the intermediate stages of a pipeline.
  • It is required an intermediate for communication between reducer_2 and application, since they are placed in distinct virtual platforms.
  • The workflow of the non-iterative system initially performs the lifecycle action activation sequence (resolve, deploy, instantiate, and run) for all components, because, in a pipeline pattern, they are required from the beginning of the computation.
  • Then, computations and connectors that will be placed on these virtual platforms.
  • After all the iterations are terminated, the parallel activation completes and all components are released.

6.2.2. The iterative system with a single stage

  • In the iterative workflow (Fig. 17), the single stage consists of a shuffler and a pair of parallel reducers.
  • In turn, before the next iterations, the following code is executed to enable the collector facets that receive pairs of the reducers and disable the collector facet that receive pairs of source: 0 <parallel> 1 <sequence> 2 <invoke port="task_shuffle_collector_active_status_0" action="CHANGE_STATUS_BEGIN.
  • Mappers and reducers receive chunks of input pairs in the first iteration (read_chunk/finish_chunk activation) and process them (invocation to the mapping or reduction function) when the action perform is activated.
  • The former describes precedences of execution between two distinct components or component actions.
  • Fig. 18 depicts the average certification times for both workflows by varying the number of units (processing nodes) of the tactical component from 1 to 16.

6.3. Parallel sorting

  • Parallel sorting is often used in HPC systems when dealing with huge amounts of data [49].
  • The contextual signature of Sorting declares a set of context parameters that may guide the choice of a sorting component that implement the supposedly better algorithm according to the contextual contract.
  • Sorting_place states whether internal or external sorting must be employed, also known as They are described below.
  • Contrariwise, it may employ a noncomparison-based algorithm.
  • In turn, number_nodes, muticore, and accelerator_type are so-called platform parameters, since they describe properties of the underlying parallel computing platform that must be taken into account in the component implementation.

6.3.1. Certifying parallel sorting components

  • Let QuickSortImpl and MergeSortImpl be two concrete components of Sorting that implement parallel versions of the well-known Quicksort and Mergesort algorithms, respectively.
  • Also, virtual platforms containing 2 processing nodes have been chosen for all tactical components.
  • For both QuickSortImpl and MergeSortImpl, all default properties of C4MPIComplex have been proved successfully.
  • The parallel times calculated for this case study makes it possible to conclude that, in general, the smallest times happen for tactical components with a single unit running in a processing node with many cores.

6.4. Discussion

  • The case studies with Montage, MapReduce and Integer Sorting are primarily aimed at demonstrating the feasibility of certifying components of distinct kinds using the certification framework of HPC Shelf.
  • Once all the certification processes involved in the case studies have been completed, the experiment has been successful to demonstrate this.
  • Therefore, it is important to emphasize that the experiments whose results are evaluated in this article do not have the ambition to constitute a definitive validation study of the certification framework of HPC Shelf.
  • The case studies have also shown how the inherent parallelism supported by the certification framework, using the parallel computing infrastructure of HPC Shelf itself, may be used to accelerate certification tasks, even if the underlying certification tools have not been developed with parallelism in mind, which is the case of the theorem provers and model checkers used in the experiments.
  • To reinforce this expectation, it is worth noting that, despite the current implementation of the certification framework is not optimal in relation to performance, the certification times achieved in the experiments, varying between 20 seconds and 12 minutes, are not influenced by possible implementation overheads.

7.1. Certification of software components

  • The certification of software components is an active research area in component-based software engineering (CBSE) since the 1990’s [3–5,7].
  • From the current literature, the authors define the certification of software components as the study and application of methods and techniques intended to provide a well-defined level of confidence that the components of a system meet a given set of requirements.
  • The literature does not mention other proposals of general-purpose certification artifacts in the context of CBHPC13 research, which could be directly compared to the certification framework of HPC Shelf.
  • In such applications, incorrect results and execution failures may cause unsustainable increases in project costs and schedules.

7.2. Verification-as-a-service (VaaS)

  • As pointed out earlier, the kind of certification focused on this paper is the verification of functional and behavioral properties of components of parallel computing systems in a cloud environment through formal methods, automated by deductive and model-checking tools.
  • The authors have systematically searched for related work on VaaS in the most comprehensive databases of scientific literature in computer science: IEEE,14 Scopus,15 ACM16 and Science Direct,17 applying the search string “(platform OR framework) AND service AND formal AND verification AND component AND cloud” to title, abstract and keywords fields.
  • Most of the discarded papers do not propose platforms or frameworks for the intended purpose.
  • The column Total 1 represents the number of distinct papers found, i.e. after removing redundancies.
  • The following sections (7.2.1 and 7.2.2) describe the papers classified in these two groups, respectively.

7.2.1. Verification of cloud administration concerns

  • Evangelidis et al. propose a probabilistic verification scheme aimed at dynamically evaluating auto-scaling policies of IaaS and PaaS virtual machines in Amazon EC2 and Microsoft Azure [59].
  • For that, it applies a Markov model implemented in the PRISM model checker [60].
  • Zhou et al. propose a formal framework for resource provisioning as a service [61].
  • Di Cosmo et al. propose the Aeolus component model [63].
  • The architecture and methodology for enabling SDV to operate in Azure, as well as the results of SDV on single drivers and driver suites using various configurations of the cloud relative to a local machine are reported.

7.2.2. Verification of functional requirements

  • Nezhad et al. propose COOL, a framework for provider-side design of cloud solutions based on formal methods and model-driven engineering [70].
  • Klai and Ochi address the problem of abstracting and verifying the correctness of integrating service-based business processes (SBPs) [74].
  • Skowyra et al. present Verificare, a verification platform for applications based on Software-Defined Networks (SDN) [82].
  • Three layers compose the framework: graphical layer, which uses sequence diagrams for system modeling; formal specification layer, which uses π -calculus to formalize the UML sequence diagram; and verification layer, in which π -calculus processes are verified by NuSMV.
  • It parallelizes symbolic execution, a popular model checking technique, to run on large shared-nothing clusters of computers, such as Amazon EC2.

7.3. Discussion

  • In comparison with the related work described above, the certification framework of HPC Shelf has the following distinguishing characteristics: .
  • It is a general-purpose framework that can be used for automatic certification of a wide range of requirements, including both functional and non-functional, while other works address a particular requirement.
  • For that, it may employ the same parallel computing infrastructure where certifiable components perform their tasks.
  • In turn, certifier selection is a component developer responsibility, using contextual contracts.
  • Also, when designing certifier components, certification authorities may provide high-level interfaces to facilitate the interaction of application providers and component developers with the underlying verification tools.

Did you find this useful? Give us your feedback

...read more

Content maybe subject to copyright    Report

Science of Computer Programming 191 (2020) 102379
Contents lists available at ScienceDirect
Science of Computer Programming
www.elsevier.com/locate/scico
A component-based framework for certification
of components in a cloud of HPC services
Allberson Bruno de Oliveira Dantas
a
, Francisco Heron de Carvalho Junior
b,
,
Luis Soares Barbosa
c
a
IEAD, Universidade da Integração Internacional da Lusofonia Afro-Brasileira, Campus da Liberdade, Redenção, Brazil
b
Mestrado e Doutorado em Ciência da Computação, Universidade Federal do Ceará, Brazil
c
HASLab INESC TEC & Universidade do Minho, Campus de Gualtar, Braga, Portugal
a r t i c l e i n f o a b s t r a c t
Article history:
Received
16 March 2018
Received
in revised form 29 November 2019
Accepted
20 December 2019
Available
online 2 January 2020
Keywords:
Verification-as-a-Service
Formal
verification
High
Performance Computing
Software
components
Scientific
workflows
HPC Shelf is a proposal of a cloud computing platform to provide component-oriented
services for High Performance Computing (HPC) applications. This paper presents a
Verification-as-a-Service (VaaS) framework for component certification on HPC Shelf.
Certification is aimed at providing higher confidence that components of parallel computing
systems of HPC Shelf behave as expected according to one or more requirements
expressed in their contracts. To this end, new abstractions are introduced, starting with
certifier components. They are designed to inspect other components and verify them for
different types of functional, non-functional and behavioral requirements. The certification
framework is naturally based on parallel computing techniques to speed up verification
tasks.
© 2020 Elsevier B.V. All rights reserved.
1. Introduction
HPC Shelf is a cloud computing platform aimed at addressing domain-specific, computationally intensive problems
typically emerging from computational science and engineering domains. For this purpose, it provides a range of High
Performance Computing (HPC) services based on parallel computing systems. They are built from the orchestration of par-
allel
components representing both software and hardware elements of HPC systems. The hardware elements represent
distributed memory parallel computing platforms such as clusters and MPPs.
1
Software components, representing parallel
computations, attempt to extract the best performance of them.
Parallel
computing systems are managed by SAFe (Shelf Application Framework) [1]. By means of SAFe, application
providers build applications, through which domain specialists access the services of HPC Shelf. Applications are domain-
specific
problem-solving environments, such as web portals [2]. They provide a high-level interface through which specialists
specify problems. Computational solutions to these problems are automatically generated according to rules programmed
by application providers, in the form of parallel computing systems.
Application
providers must have technical background to create computational solutions to problems in their application
domains. In HPC Shelf, they must be able to identify and combine components to form parallel computing systems. Thus,
background on parallel computing platforms and programming for such platforms is not required for application providers.
*
Corresponding author.
E-mail
addresses: allberson@unilab.edu.br (A.B. de Oliveira Dantas), heron@lia.ufc.br (F.H. de Carvalho Junior), lsb@di.uminho.pt (L.S. Barbosa).
1
Massively Parallel Processing systems.
https://doi.org/10.1016/j.scico.2019.102379
0167-6423/
© 2020 Elsevier B.V. All rights reserved.

2 A.B. de Oliveira Dantas et al. / Science of Computer Programming 191 (2020) 102379
This is a requirement for component developers. Combined with the inherent complexity of parallel system design, this fact
implies the need for effective mechanisms to ensure that, in parallel computing systems, components and interactions be-
tween
them behave as expected by application providers and predicted by component developers. In software engineering,
this is a problem known as certification of software components [37]. In the context of HPC Shelf, the certification problem
can be seen from two perspectives. In the component perspective, each component implementation is verified against the
functional, non-functional, and behavioral requirements declared in its published interface. In the system perspective, typical
safety and liveness properties of the component orchestration workow should be ensured.
The
need for rigorous validation, leading to some form of system certification, brings formal methods into the picture
in order to identify, and possibly rule out, faulty behaviors in applications. Such methods are, however, not so common in
the domain of HPC systems, due to their inherent complexity and the difficulty of their concrete implementation. Indeed,
HPC systems are defined by heterogeneous computing platforms composed concurrently. In fact, in spite of more than three
decades of active research in formal methods for software development and verification, we are still far from what should
be the practice of a true engineering discipline, supported by a stable and sound mathematical basis. In most cases, testing
and a posteriori empirical error detection are still dominant, even in scenarios where formal verification is a requirement
(e.g. safety-critical systems).
The
work reported in this article presents the proposal for a cloud-based general-purpose certification framework for
HPC Shelf. Through the proposed framework, components called certifiers may use a set of different certification tools
to certify that the components of parallel computing systems meet a certain set of requirements. The case studies used
to demonstrate the proposed certification framework are particularly focused on functional and behavioral requirements
that can be verified through automated verification methods and tools, such as theorem provers and model checkers. The
certification process becomes integrated with the parallel computing systems in a highly modular way, so that new certifier
components may be inserted according to the verification tasks required in the certification process.
The
certification process may be carried on in parallel. For this, certifier components are defined as parallel certification
systems, analogous to parallel computing systems. Parallel certification systems contain a certification-workflow component
and
a set of tactical components, each one providing the access to an existing certification tool or infrastructure running
in a parallel computing platform. Thus, within tactical components, parallel computing may help exploit the maximum
performance of the underlying verification infrastructures to accelerate the certification process.
Summing
up, the main artifacts produced by the work whose results are reported in this article are the following ones:
A general-purpose certification framework for HPC Shelf;
A class of certifier components, named C4, for the certification of computation components of HPC Shelf;
Another class of certifier components, named SWC2, for the certification of workflow components;
A set of tactical components to make the bridge between the above certifier components and existing formal verification
tools.
C4 and SWC2 have the purpose of helping proof-of-concept validation of the certification framework of HPC Shelf.
It has also been evaluated in the context of other cloud-based software certification initiatives, with emphasis on works
related to VaaS, HPC, and automatic software-verification tools. From this assessment, the following outstanding features
and contributions have been identified in favor of the certification framework of HPC Shelf:
It is general purpose, in the sense that it is not intended to certify a particular requirement, although the case studies
presented in this article focus on the verification of functional and behavioral properties through deductive program
verification and model checking tools.
It does not certify only software components, but any kind of component, including components representing hardware
elements, such as parallel computing platforms in HPC Shelf.
It is fully component-oriented, with seamless integration with the environment, in the sense that certification is intro-
duced
by certifier and tactical components that may encapsulate certification tools.
It is the first certification framework in the context of component-based high performance computing (CBHPC), where
certification may avoid the wasting of time and financial resources due to delays, crashes, and wrong outputs in the
execution of long-running computations.
It is the first VaaS framework applied in the context of HPC.
It introduces new ideas for VaaS framework design, such as:
–the
use of component-orientation to support a higher level of abstraction with respect to underlying formal verifica-
tion
tools;
–the
clear role separation among certification authorities, component developers, and system builders (application
providers);
It presents a general method for exploring parallel processing to speedup certification tasks at several levels, by explor-
ing
the parallel computing infrastructure where parallel components subject to certification runs.
Article
structure. After a description of HPC Shelf in Section 2, its certification framework is introduced in Section 3. Sec-
tions 4 and
5 detail, respectively, the architecture of C4 and SWC2 certifiers. Next, Section 6 presents some case studies to

A.B. de Oliveira Dantas et al. / Science of Computer Programming 191 (2020) 102379 3
demonstrate the use of C4 and SWC2 certifiers in parallel computing systems. A discussion about related works obtained
through a systematic search in scientific databases is presented in Section 7, with emphasis in certification of component-
based
software systems and VaaS. Finally, Section 8 presents concluding remarks, pointing to further works.
2. HPC Shelf
HPC Shelf is a cloud computing platform that provides HPC services for providers of domain-specific applications. An ap-
plication
is a problem-solving environment through which specialist users, the end users of HPC Shelf, specify problems and
obtain computational solutions for them. It is assumed that these solutions are computationally intensive, thus demanding
the use of large-scale parallel computing infrastructure, i.e. comprising multiple parallel computing platforms engaged in a
single computational task.
Applications
generate computational solutions as component-oriented parallel computing systems. They are built by com-
position
and orchestration of a set of parallel components that represent hardware and software elements, addressing
functional and non-functional concerns. To do so, these components comply to Hash [8], a parallel component model whose
components may exploit parallel processing in distributed-memory parallel computing platforms. For that, they are formed
by a set of units, each one placed on one of their processing nodes, so that parallelism concerns may be confined to a
single component. Parallel components that are compatible with Hash may be combined hierarchically through overlapping
composition [9].
2.1. Component kinds of parallel computing systems
Component platforms that comply to the Hash component model distinguish components according to a set of com-
ponent
kinds. A component kind represents a set of components with similar deployment and interaction models, possibly
representing building-block abstractions of an application domain. HPC Shelf supports the following component kinds:
virtual platforms, representing distributed-memory parallel computing platforms (e.g. clusters and MPPs);
computations, representing parallel algorithm implementations that may accelerate performance by exploiting the ar-
chitectural
characteristics and features of a class of virtual platforms;
data sources, representing repositories of data required by computations, possibly BigData ones;
connectors, formed by a set of facets, co-located with computations and data sources located at different virtual plat-
forms,
aimed at orchestrating them and/or supporting choreographs involving them;
service bindings, aimed at binding user and provider ports exported by components for communication with the envi-
ronment
and among them, as well as supporting non-functional services;
action bindings, aimed at binding action ports of computations and connectors for orchestration of computational,
communication and synchronization tasks supported by them.
Through
a service binding, a component may consume a service offered by another component. This is only possible if
the former component has a user port and the later one has a provider port, whose types must be compatible with the type
of the service binding. Therefore, the service binding may act either as a simple service dispatcher or as a service adapter.
Action
bindings connect a set of action ports belonging to computation and connector components. These ports are typed
by a set of action names, which label computational, communication or synchronization tasks. A component may export one
or more action ports. The orchestration of action names defines the workflow of a parallel computing system.
An
action port offers, to computation and connectors, an interface to control the activation of action names (Listing 1).
The activation of an action name n completes in an action binding if there is a pending activation of n in each action port.
Otherwise, it remains blocked.
The
components of HPC Shelf have a default lifecycle action port, referred with the own component instance identifier
in a parallel computing system. It has the following reflexive action names:
resolve, which attempts to select a component implementation that best fits the requirements of its contextual contract
(Section 2.5);
deploy, which deploys the selected component implementation in the virtual platform where it will be instantiated for
starting execution;
instantiate, which makes a component ready for computation and interaction with other components through its service
and action ports;
run, which starts the internal orchestration logic of the component, i.e. the orchestration of action names of their action
ports;
release, which releases the resources allocated for the component instance, after its computation has finished.
In
the workflow of a parallel computing system, the activation of the lifecycle action names must respect the following
protocol (regular expression):

4 A.B. de Oliveira Dantas et al. / Science of Computer Programming 191 (2020) 102379
public interface ITaskBinding : IActivateKind , GoPort
{
// synchronous , simple , no reaction code
void invoke( object action );
// synchronous , multiple , no reaction code
object invoke( object [] action);
// asynchronous , simple , no reaction code
void invoke( object action , out IActionFuture f );
// asynchronous , multiple , no reaction code
void invoke( object [] action , out IActionFuture f );
// asynchronous , simple , reaction code
void invoke( object action , Action reaction , out IActionFuture f );
// asynchronous , multiple , reaction code
void invoke( object [] action , Action[] reaction , out IActionFuture f );
}
public interface IActionFuture
{
void wait (); // waits for completion of the action
bool test (); // tests the completion of an action
IActionFutureSet createSet (); // creates a set of pending actions
object Action { get; } // retrieves the reaction code to be executed
void registerWaitingSet ( AutoResetEvent waiting_set );
void unregisterWaitingSet (AutoResetEvent waiting_set );
}
public interface IActionFutureSet: IEnumerable<IActionFuture>
{
void addAction(IActionFuture f ); // add a new activation action
void waitAll (); // waits for completion of all pending actions
IActionFuture waitAny(); // waits for completion of any pending action
bool testAll (); // tests the completion of all pending actions
IActionFuture testAny(); // tests the completion of any pending action
IActionFuture [] Pending { get; } // get the list of pending actions
}
Listing 1: ITaskBinding interface (C#).
resolve · deploy · instantiate · run · release ·
(
resolve · deploy
)
?
· instantiate · run · release
The protocol specifies that the workflow must resolve the contextual contract of each component before its instantiation.
However, it may invoke contract resolution multiple times. After each resolution, the component must be deployed again,
before instantiation. It is worth remembering that a distinct component implementation may be selected in each invocation
to the contract resolution service. Finally, all instantiated components must be released before workflow termination, freeing
resources.
Parallel
computing systems. A parallel computing system consists of two special components, respectively called application
and
workflow, and a set of solution components of the previously described kinds. The workflow component represents an or-
chestration
engine that will drive the overall computation. It may be programmed by using a general-purpose programming
language (currently, C#) or SAFeSWL (SAFe Scientific Workow Language), an XML-based orchestration language designed
for activating the computational tasks of the solution components in a prescribed order [1]. Indeed, an application may
generate SAFeSWL code by dynamically building computational solutions to problems specified by specialist users. Through
a provenance mechanism recently developed, automatically generated parallel computing systems may be saved in the
component catalog by treating them as components. The XSD grammar of SAFeSWL is presented in the Appendix A. The
application component makes communication between the application frontend and solution components possible through
service bindings.
2.2. Parallel computing systems through an example: MapReduce
MapReduce is the parallel processing model of a number of large-scale parallel processing frameworks [10]. A user must
specify: a map function, which is applied by a set of parallel mapper processes to each element of an input list of key/value
pairs
(KV-pair
2
) and returns a set of elements in an intermediary list of KV-pairs; and a reduce function, which is applied by
a set of parallel reducer processes to each element of an intermediate list of key/multi-value pairs (KMV-pair
3
), and yield a
list of output KV-pairs.
Fig. 1 illus
trates a MapReduce computation for counting the frequencies of words green, yellow, blue and pink in a text.
At the end of the computation, the expected output is the number of occurrences of each color in the text.
A
framework of components for MapReduce computations has been designed for HPC Shelf [11]. It comprises a pair of
component types of kind computation, named Mapper and Reducer, which represent mapping and reducing agents, and
another pair of component types of kind connector, named Splitter and Shuffler, which intermediate communication of
KV/KMV-pairs among mappers, reducers and data sources.
2
A KV-pair is a pair
k, v
, where k is a key and v is a value.
3
A KMV-pair is a pair
k,
[
v
1
v
2
... v
n
]
, where k is a key, mapped to values v
i
, for i
{
1, 2,...,n
}
.

A.B. de Oliveira Dantas et al. / Science of Computer Programming 191 (2020) 102379 5
Fig. 1. A classic example of counting word frequencies with MapReduce. (For interpretation of the colors in the figure(s), the reader is referred to the web
version of this article.)
Fig. 2. The building blocks of MapReduce parallel computing systems.
Fig. 3. Multiple facets of Splitter and Shuffler.
Fig. 2 depicts the service ports of Mapper, Reducer, Splitter and Shuffler. Their action ports will be introduced later.
They have a pair of service ports named collect_pairs and feed_pairs. In the connectors, these ports are placed at different
facets. Since they are multiple facets, instead of single facets, connectors may have a set of collect_pairs and another set of
feed_pairs ports,
as illustrated in Fig. 3. Thus, mappers, reducers, splitters and shufflers receive a list of either KV-pairs
(type
[
(
K , V
)
]
,
highlighted in blue) or KMV-pairs (type
[
(
K ,
[
V
]
)
]
,
highlighted in red) through each collect_pairs binding,
and send a list of either KV-pairs or KMV-pairs through each feed_pairs binding. In particular:
mappers receive a list of input KV-pairs of type
(
IK, IV
)
, apply a map function to each input pair in order to yield a list
of output KV-pairs of type
(
TK, TV
)
, and return the list of yielded output KV-pairs;
reducers receive a list of input KMV-pairs of type
(
TK,
[
TV
]
)
, apply a reduce function to each input pair in order to
yield an output KV-pair of type
[
(
OK, OV
)
]
,
and return the list of yielded output KV-pairs;
splitters receive a set of lists of input KV-pairs of type
(
IK, IV
)
, each one associated to a collector facet, redistribute the
input KV-pairs across a set of output KV-pairs lists, each one associated to a feeder facet, and send them through the
corresponding feed_pairs port;
shufflers receive a set of lists of input KV-pairs of type
(
TK, TV
)
, each one associated to a collector facet, group the pairs
with the same key in a single KMV-pair of type
[
(
TK,
[
TV
]
)
]
,
distribute the KMV-pairs across a set of output KMV-pair
lists, each one associated to a feeder facet, and send them through the corresponding feed_pairs port.
In
a MapReduce parallel computing system, mappers, reducers, splitters, and shufflers may be connected through bind-
ings
between their compatible collect_pairs and feed_pairs ports. For instance, Fig. 4 depicts the architecture of a simple

Citations
More filters

Journal ArticleDOI
10 Jun 2020
Abstract: Existe la necesidad de incorporar componentes personalizados en plataformas empresariales cuyas ediciones al publico en general son libres y limitados de caracteristicas que solo cuentan las ediciones premium o pagadas. Esto se ha vuelto un tema de incomodo por parte de los desarrolladores de software de empresas publicas y privadas porque al desarrollar sus aplicaciones con estas soluciones de TI se ven restringidos al momento de extender funcionalidades de acuerdo al modelo de negocio. Por consecuente, con la crisis economica a nivel mundial, las organizaciones optan por manejar versiones libres para evitar elevados costos de licencias, pero esto obliga a los programadores a tener un desarrollo de software de una manera no convencional para extender las funcionalidades.

1 citations


Proceedings Article
01 Jan 2015
Abstract: Cloud environments are being increasingly used for deploying and executing business processes and particularly service-based business processes (SBPs). In this paper, we propose a bottom-up approach to check the correct interaction between different SBPs distributed over a Cloud environment and which may be provided by various organizations. The whole system's model being unavailable, an up-down analysis approach is not appropriate. To check the correctness of the composition of several SBPs communicating asynchronously and sharing resources (hardware, platform, and software), we consider temporal properties that can be expressed with the LTL logic. Each part of the whole composite SBP exposes its abstract model, represented by a Symbolic Observation Graph (SOG), to allow the correct collaboration with possible partners in the Cloud. The SOG is adapted in order to reduce the verification of the entire composite model to the verification of the composition of the SOG-based abstractions.

References
More filters

Journal ArticleDOI
Jeffrey Dean1, Sanjay Ghemawat1
TL;DR: This presentation explains how the underlying runtime system automatically parallelizes the computation across large-scale clusters of machines, handles machine failures, and schedules inter-machine communication to make efficient use of the network and disks.
Abstract: MapReduce is a programming model and an associated implementation for processing and generating large datasets that is amenable to a broad variety of real-world tasks. Users specify the computation in terms of a map and a reduce function, and the underlying runtime system automatically parallelizes the computation across large-scale clusters of machines, handles machine failures, and schedules inter-machine communication to make efficient use of the network and disks. Programmers find the system easy to use: more than ten thousand distinct MapReduce programs have been implemented internally at Google over the past four years, and an average of one hundred thousand MapReduce jobs are executed on Google's clusters every day, processing a total of more than twenty petabytes of data per day.

17,078 citations


"A component-based framework for cer..." refers methods in this paper

  • ...MapReduce is the parallel processing model of a number of large-scale parallel processing frameworks [10]....

    [...]


Book ChapterDOI
29 Mar 2008
TL;DR: Z3 is a new and efficient SMT Solver freely available from Microsoft Research that is used in various software verification and analysis applications.
Abstract: Satisfiability Modulo Theories (SMT) problem is a decision problem for logical first order formulas with respect to combinations of background theories such as: arithmetic, bit-vectors, arrays, and uninterpreted functions. Z3 is a new and efficient SMT Solver freely available from Microsoft Research. It is used in various software verification and analysis applications.

5,796 citations


Additional excerpts

  • ...Examples include Alt-Ergo [25], CVC3 [26], CVC4 [27] and Z3 [28]....

    [...]



Journal ArticleDOI
TL;DR: The a-calculus is presented, a calculus of communicating systems in which one can naturally express processes which have changing structure, including the algebraic theory of strong bisimilarity and strong equivalence, including a new notion of equivalence indexed by distinctions.
Abstract: We present the a-calculus, a calculus of communicating systems in which one can naturally express processes which have changing structure. Not only may the component agents of a system be arbitrarily linked, but a communication between neighbours may carry information which changes that linkage. The calculus is an extension of the process algebra CCS, following work by Engberg and Nielsen, who added mobility to CCS while preserving its algebraic properties. The rr-calculus gains simplicity by removing all distinction between variables and constants; communication links are identified by names, and computation is represented purely as the communication of names across links. After an illustrated description of how the n-calculus generalises conventional process algebras in treating mobility, several examples exploiting mobility are given in some detail. The important examples are the encoding into the n-calculus of higher-order functions (the I-calculus and combinatory algebra), the transmission of processes as values, and the representation of data structures as processes. The paper continues by presenting the algebraic theory of strong bisimilarity and strong equivalence, including a new notion of equivalence indexed by distinctions-i.e., assumptions of inequality among names. These theories are based upon a semantics in terms of a labeled transition system and a notion of strong bisimulation, both of which are expounded in detail in a companion paper. We also report briefly on work-in-progress based upon the corresponding notion of weak bisimulation, in which internal actions cannot be observed. 0 1992 Academic Press, Inc.

3,028 citations


Book
01 Jan 2002
TL;DR: This presentation discusses Functional Programming in HOL, which aims to provide students with an understanding of the programming language through the lens of Haskell.
Abstract: Elementary Techniques.- 1. The Basics.- 2. Functional Programming in HOL.- 3. More Functional Programming.- 4. Presenting Theories.- Logic and Sets.- 5. The Rules of the Game.- 6. Sets, Functions, and Relations.- 7. Inductively Defined Sets.- Advanced Material.- 8. More about Types.- 9. Advanced Simplification, Recursion, and Induction.- 10. Case Study: Verifying a Security Protocol.

2,797 citations


"A component-based framework for cer..." refers background in this paper

  • ...Well known examples include Coq [32] and Isabelle/HOL [33]....

    [...]


Frequently Asked Questions (1)
Q1. What are the contributions in "A component-based framework for certification of components in a cloud of hpc services" ?

In this paper, the authors propose a certification framework for HPC Shelf, a cloud-based general-purpose certification framework.