scispace - formally typeset
Search or ask a question

Showing papers in "Concurrency and Computation: Practice and Experience in 2015"


Journal ArticleDOI
TL;DR: FireWorks has been used to complete over 50 million CPU‐hours worth of computational chemistry and materials science calculations at the National Energy Research Supercomputing Center, and its implementation strategy that rests on Python and NoSQL databases (MongoDB) is discussed.
Abstract: This paper introduces FireWorks, a workflow software for running high-throughput calculation workflows at supercomputing centers. FireWorks has been used to complete over 50 million CPU-hours worth of computational chemistry and materials science calculations at the National Energy Research Supercomputing Center. It has been designed to serve the demanding high-throughput computing needs of these applications, with extensive support for i concurrent execution through job packing, ii failure detection and correction, iii provenance and reporting for long-running projects, iv automated duplicate detection, and v dynamic workflows i.e., modifying the workflow graph during runtime. We have found that these features are highly relevant to enabling modern data-driven and high-throughput science applications, and we discuss our implementation strategy that rests on Python and NoSQL databases MongoDB. Finally, we present performance data and limitations of our approach along with planned future work. Copyright © 2015 John Wiley & Sons, Ltd.

405 citations


Journal ArticleDOI
TL;DR: An experimental evaluation of OpenStack Neat and several dynamicVM consolidation algorithms on a five‐node testbed shows significant benefits of dynamic VM consolidation resulting in up to 33% energy savings.
Abstract: Dynamic consolidation of virtual machines VMs is an efficient approach for improving the utilization of physical resources and reducing energy consumption in cloud data centers. Despite the large volume of research published on this topic, there are very few open-source software systems implementing dynamic VM consolidation. In this paper, we propose an architecture and open-source implementation of OpenStack Neat, a framework for dynamic VM consolidation in OpenStack clouds. OpenStack Neat can be configured to use custom VM consolidation algorithms and transparently integrates with existing OpenStack deployments without the necessity of modifying their configuration. In addition, to foster and encourage further research efforts in the area of dynamic VM consolidation, we propose a benchmark suite for evaluating and comparing dynamic VM consolidation algorithms. The proposed benchmark suite comprises OpenStack Neat as the base software framework, a set of real-world workload traces, performance metrics and evaluation methodology. As an application of the proposed benchmark suite, we conduct an experimental evaluation of OpenStack Neat and several dynamic VM consolidation algorithms on a five-node testbed, which shows significant benefits of dynamic VM consolidation resulting in up to 33% energy savings. Copyright © 2014 John Wiley & Sons, Ltd.

154 citations


Journal ArticleDOI
TL;DR: This paper defines a lifecycle for Big Data processing and classifies various available tools and technologies in terms of the lifecycle phases of Big Data, which include data acquisition, data storage, data analysis, and data exploitation of the results.
Abstract: Big Data encompasses large volume of complex structured, semi-structured, and unstructured data, which is beyond the processing capabilities of conventional databases. The processing and analysis of Big Data now play a central role in decision making, forecasting, business analysis, product development, customer experience, and loyalty, to name but a few. In this paper, we examine the distinguishing characteristics of Big Data along the lines of the 3Vs: variety, volume, and velocity. Accordingly, the paper provides an insight into the main processing paradigms in relation to the 3Vs. It defines a lifecycle for Big Data processing and classifies various available tools and technologies in terms of the lifecycle phases of Big Data, which include data acquisition, data storage, data analysis, and data exploitation of the results. This paper is first of its kind that reviews and analyzes current trends and technologies in relation to the characteristics, evolution, and processing of Big Data. Copyright © 2014 John Wiley & Sons, Ltd.

105 citations


Journal ArticleDOI
TL;DR: Through a large‐scale survey, the extent and characteristics of the gateway community (reliance on gateways and nature of existing resources) are measured to understand useful services and support for builders and users.
Abstract: Science gateways are digital interfaces to advanced technologies that support science/engineering research/education. Frequently implemented as Web and mobile applications, they provide access to community resources such as software, data, collaboration tools, instrumentation, and high‐performance computing. We anticipate opportunities for growth within a fragmented community. Through a large‐scale survey, we measured the extent and characteristics of the gateway community (reliance on gateways and nature of existing resources) to understand useful services and support for builders and users. We administered an online survey to nearly 29,000 principal investigators, senior administrators, and people with gateway affiliations. Nearly 5000 respondents represented diverse expertise and geography. The majority of researchers/educators indicated that specialized online resources were important to their work. They choose technologies by asking colleagues and looking for documentation, demonstrated reliability, and technical support; adaptability via customizing or open‐source standards was another priority. Research groups commonly provide their own resources, but public/academic institutions and commercial services also provide substantial offerings. Application creators and administrators welcome external services providing guidance such as technology selection, sustainability planning, evaluation, and specialized expertise (e.g., quality assurance and design). Technologies are diverse, so flexibility and ongoing community input are essential, as is offering specific, easy‐to‐access training, community support, and professional development. Copyright © 2015 John Wiley & Sons, Ltd.

93 citations


Journal ArticleDOI
TL;DR: Three genetic algorithms as energy‐aware grid schedulers are developed and empirically evaluated in three grid size scenarios in static and dynamic modes to address the independent batch scheduling in computational grid as a bi‐objective global minimization problem with Makespan and energy consumption as the main criteria.
Abstract: In today's highly parametrized distributed computational environments, such as green grid clusters and clouds, the growing power and cooling rates are becoming the dominant part of the users' and system managers' budgets. Computational grids, owing to their sheer sizes, still require advanced methodologies and strategies for supporting the scheduling of the users' tasks and applications to the distributed resources. The efficient resource allocation becomes even more challenging when energy utilization, beyond the conventional scheduling criteria, such as Makespan, is treated as first-class additional scheduling objective. In this paper, we address the independent batch scheduling in computational grid as a bi-objective global minimization problem with Makespan and energy consumption as the main criteria. We apply the dynamic voltage and frequency scaling model for the management of the cumulative power energy utilized by the grid resources. We develop three genetic algorithms as energy-aware grid schedulers, which were empirically evaluated in three grid size scenarios in static and dynamic modes. The simulation results confirmed the effectiveness of the proposed genetic algorithm-based schedulers in the reduction of the energy consumed by the whole system and in dynamic load balancing of the resources in grid clusters, which is sufficient to maintain the desired quality levels. Copyright © 2012 John Wiley & Sons, Ltd.

77 citations


Journal ArticleDOI
TL;DR: An overview and roadmap of the Apache Airavata software system for science gateways shows how the open community governance model is as important as its software base and how it may be applicable to other distributed computing infrastructure and cyberinfrastructure efforts.
Abstract: Summary This paper provides an overview and roadmap of the Apache Airavata software system for science gateways. Gateways use Airavata to manage application and workflow executions on a range of backend resources (grids, computing clouds, and local clusters). Airavata's design goal is to provide component abstractions for major tasks required to provide gateway application management. Components are not directly accessed but are instead exposed through component programming interfaces. This design allows gateway developers to take full advantage of Airavata's capabilities and Airavata developers (including those interested in middleware research) to modify Airavata's implementations and behavior. This is particularly important as Airavata evolves to become a scalable, elastic ‘platform as a service’ for science gateways. We illustrate the capabilities of Airavata through the discussion of usage vignettes. As an Apache Software Foundation project, Airavata's open community governance model is as important as its software base. We discuss how this works within Airavata and how it may be applicable to other distributed computing infrastructure and cyberinfrastructure efforts. Copyright © 2015 John Wiley & Sons, Ltd.

68 citations


Journal ArticleDOI
TL;DR: This paper studies a virtual machine (VM) migration problem in roadside cloudlet‐based vehicular network and proposes a heuristic algorithm with polynomial time to tackle the complexity of solving mixed‐integer quadratic programming.
Abstract: Vehicle Ad-Hoc Networks VANET enable all components in intelligent transportation systems to be connected so as to improve transport safety, relieve traffic congestion, reduce air pollution, and enhance driving comfort. The vision of all vehicles connected poses a significant challenge to the collection, storage, and analysis of big traffic-related data. Vehicular cloud computing, which incorporates cloud computing into vehicular networks, emerges as a promising solution. Different from conventional cloud computing platform, the vehicle mobility poses new challenges to the allocation and management of cloud resources in roadside cloudlet. In this paper, we study a virtual machine VM migration problem in roadside cloudlet-based vehicular network and unfold that 1 whether a VM shall be migrated or not along with the vehicle moving and 2 where a VM shall be migrated, in order to minimize the overall network cost for both VM migration and normal data traffic. We first treat the problem as a static off-line VM placement problem and formulate it into a mixed-integer quadratic programming problem. A heuristic algorithm with polynomial time is then proposed to tackle the complexity of solving mixed-integer quadratic programming. Extensive simulation results show that it produces near-optimal performance and outperforms other related algorithms significantly. Copyright © 2015 John Wiley & Sons, Ltd.

62 citations


Journal ArticleDOI
TL;DR: This article examines enabling technologies for adding semantics to the IoT, and analyzes data formats, which enable IoT applications consume semantic IoT data in a straightforward and general fashion, and evaluates resource usage of different alternatives with a sensor system.
Abstract: The development of Internet of Things IoT applications can be facilitated by encoding the meaning of the data in the messages sent by IoT nodes, but the constrained resources of these nodes challenge the common Semantic Web solutions for doing this. In this article, we examine enabling technologies for adding semantics to the IoT. Especially, we analyze data formats, which enable IoT applications consume semantic IoT data in a straightforward and general fashion, and evaluate resource usage of different alternatives with a sensor system. Our experiment illustrates encoding and decoding of different data formats and shows how big a difference a data format can make in energy consumption. Copyright © 2014 John Wiley & Sons, Ltd.

62 citations


Journal ArticleDOI
TL;DR: The capabilities of this platform are introduced and representative applications are reviewed, which makes it easy for individuals, teams, and institutions to create collaborative applications such as science gateways for science communities.
Abstract: Globus, developed as Software-as-a-Service (SaaS) for research data management, also provides APIs that constitute a flexible and powerful Platform-as-a-Service (PaaS) to which developers can outsource data management activities such as transfer and sharing, as well as identity, profile and group management. By providing these frequently important but always challenging capabilities as a service, accessible over the network, Globus PaaS streamlines web application development and makes it easy for individuals, teams, and institutions to create collaborative applications such as science gateways for science communities. We introduce the capabilities of this platform and review representative applications.

53 citations


Journal ArticleDOI
TL;DR: These demands and how the e‐Infrastructure and gateway is being designed and implemented to accommodate this diversity of requirements are described, both from the user/researcher perspective and from the data provider perspective.
Abstract: The $20m Australian Urban Research Infrastructure Network (AURIN) project (www.aurin.org.au) began in July 2010. AURIN has been tasked with developing a secure,Web-based virtual environment (e-Infrastructure) offering seamless, secure access to diverse, distributed and extremely heterogeneous data sets from numerous agencies with an extensive portfolio of targeted analytical and visualization tools. This is being provisioned for Australia-wide urban and built environment researchers – itself a highly heterogeneous collection of research communities with diverse demands, through a unified urban research gateway. This paper describes these demands and how the e-Infrastructure and gateway is being designed and implemented to accommodate this diversity of requirements, both from the user/researcher perspective and from the data provider perspective. The scaling of the infrastructure is presented and the way in which it copes with the spectrum of big data challenges (volume, veracity, variability and velocity) and associated big data analytics. The utility of the e-Infrastructure is also demonstrated through a range of scenarios illustrating and reflecting the interdisciplinary urban research now possible.

49 citations


Journal ArticleDOI
TL;DR: A domain‐independent, cloud‐based science gateway platform, the Globus Galaxies platform, which overcomes this gap by providing a set of hosted services that directly address the needs of science gateway developers.
Abstract: Summary The use of public cloud computers to host sophisticated scientific data and software is transforming scientific practice by enabling broad access to capabilities previously available only to the few. The primary obstacle to more widespread use of public clouds to host scientific software (‘cloud-based science gateways’) has thus far been the considerable gap between the specialized needs of science applications and the capabilities provided by cloud infrastructures. We describe here a domain-independent, cloud-based science gateway platform, the Globus Galaxies platform, which overcomes this gap by providing a set of hosted services that directly address the needs of science gateway developers. The design and implementation of this platform leverages our several years of experience with Globus Genomics, a cloud-based science gateway that has served more than 200 genomics researchers across 30 institutions. Building on that foundation, we have implemented a platform that leverages the popular Galaxy system for application hosting and workflow execution; Globus services for data transfer, user and group management, and authentication; and a cost-aware elastic provisioning model specialized for public cloud resources. We describe here the capabilities and architecture of this platform, present six scientific domains in which we have successfully applied it, report on user experiences, and analyze the economics of our deployments. Published 2015. This article is a U.S. Government work and is in the public domain in the USA. Concurrency and Computation: Practice and Experience published by John Wiley & Sons Ltd.

Journal ArticleDOI
TL;DR: Progress made on furthering this research is presented by investigating what attacks are available to insiders together with the damage and implications of such attacks.
Abstract: Cloud computing offers the potential for significant cost reductions and increased agility for users. However, security concerns continue to be raised as a potential barrier to uptake for private, community and public clouds. A report from the European Network and Information Security Agency on the Priorities for Research on Current and Emerging Network Technologies highlighted trusted cloud models as one of its top priorities for further research. More recently-September 2012-Carnegie Mellon University's computer emergency response team have released a paper describing insider threats to cloud computing as a direction for new research. Further, a project completed at the University of Warwick in 2010 investigated security aspects of cloud computing and in particular the potential for cascade effects. This research involved a detailed modelling of the threat and vulnerability landscape, including the incentives and motivations that might drive attackers. One of the conclusions is that insider threats potentially pose the most significant source of risk. This paper presents the progress made on furthering this research by investigating what attacks are available to insiders together with the damage and implications of such attacks. Copyright © 2014 John Wiley & Sons, Ltd.

Journal ArticleDOI
TL;DR: An artificial bee colony based energy‐aware resource utilization technique corresponding to the model has been designed to allocate jobs to the resources in a cloud environment and outperforms the existing techniques by minimizing energy consumption and execution time of applications submitted to the cloud.
Abstract: Cloud computing is a form of distributed computing, which promises to deliver reliable services through next-generation data centers that are built on virtualized compute and storage technologies. It is becoming truly ubiquitous and with cloud infrastructures becoming essential components for providing Internet services, there is an increase in energy-hungry data centers deployed by cloud providers. As cloud providers often rely on large data centers to offer the resources required by the users, the energy consumed by cloud infrastructures has become a key environmental and economical concern. Much energy is wasted in these data centers because of under-utilized resources hence contributing to global warming. To conserve energy, these under-utilized resources need to be efficiently utilized and to achieve this, jobs need to be allocated to the cloud resources in such a way so that the resources are used efficiently and there is a gain in performance and energy efficiency. In this paper, a model for energy-aware resource utilization technique has been proposed to efficiently manage cloud resources and enhance their utilization. It further helps in reducing the energy consumption of clouds by using server consolidation through virtualization without degrading the performance of users' applications. An artificial bee colony based energy-aware resource utilization technique corresponding to the model has been designed to allocate jobs to the resources in a cloud environment. The performance of the proposed algorithm has been evaluated with the existing algorithms through the CloudSim toolkit. The experimental results demonstrate that the proposed technique outperforms the existing techniques by minimizing energy consumption and execution time of applications submitted to the cloud. Copyright © 2014 John Wiley & Sons, Ltd.

Journal ArticleDOI
TL;DR: The requirements and the improvements made in the Swiss Grid Proteomics Portal, now called iPortal, are detailed, and an outlook on future directions is given.
Abstract: Summary Development of workflow and data management systems are challenging because of the need to provide a service for users with varying degrees of expertise from novices to experts with more knowledge than the developers. We have received feedback from users and developers on the functionality and usability of the Swiss Grid Proteomics Portal, now called iPortal, during its first year of operation. Monitoring and interaction with the production system under heavy use have provided further information on how to improve efficiency and stability. In a second, highly upgraded version of iPortal, we have introduced several new concepts based on this feedback. In this paper, we detail the requirements and the improvements we have made, and also give an outlook on future directions. Copyright © 2014 John Wiley & Sons, Ltd.

Journal ArticleDOI
Zhaobin Liu1, Wenyu Qu1, Weijiang Liu1, Zhiyang Li1, Yujie Xu1 
TL;DR: A new directed acyclic graph based scheduling algorithm called earliest finish time duplication algorithm for heterogeneous cloud systems is presented, which attempts to insert suitable immediate parent nodes of the current selected node in order to reduce its waiting time on the processor.
Abstract: Cloud computing came into being and is currently an essential infrastructure of many commerce facilities. To achieve the promising potentials of cloud computing, effective and efficient scheduling algorithms are fundamentally important. However, conventional scheduling methodology encounters a number of challenges. During the tasks scheduling in cloud systems, how to make full use of resources and how to effectively select resources are also important factors. At the same time, communication delay also plays an important role in cloud scheduling, which not only leads to waiting between tasks but also results in much idle interval time between processing units. In this paper, a fuzzy clustering method is used to effectively preprocess the cloud resources. Combining the list scheduling with the task duplication scheduling scheme, a new directed acyclic graph based scheduling algorithm called earliest finish time duplication algorithm for heterogeneous cloud systems is presented. Earliest finish time duplication attempts to insert suitable immediate parent nodes of the current selected node in order to reduce its waiting time on the processor. The case study and experimental results illustrate that the algorithm proposed in this paper is better than the popular heterogeneous earliest finish time algorithms. Copyright © 2014 John Wiley & Sons, Ltd.

Journal ArticleDOI
TL;DR: The method first computes neighborhoods of users and services based on their locations which provide a basis for data smoothing and then combines user‐based and service‐based collaborative filtering techniques to make QoS predictions.
Abstract: To assess the quality of services QoS in service selection, collaborative service QoS prediction has recently garnered increasing attention. They focus on exploring the historical QoS information generated by interactions between users and services. However, they may suffer from the data sparsity issue because interactions between users and services are usually sparse in real scenarios. They also seldom consider the network environments of users and services, which surely will affect cloud service QoS. To address the data sparsity issue and improve the QoS prediction accuracy, the following paper proposes a collaborative QoS prediction method with location-based data smoothing. The method first computes neighborhoods of users and services based on their locations which provide a basis for data smoothing. It then combines user-based and service-based collaborative filtering techniques to make QoS predictions. Experiments conducted using a real service invocation dataset validate the performance of the proposed QoS prediction method. Copyright © 2015 John Wiley & Sons, Ltd.

Journal ArticleDOI
TL;DR: This paper proposes an approach to parallelize data mining procedures in the form of compiled software or R scripts developed by biology communities of practice, and relies on a distributed network of web services to expose the algorithms as‐a‐Service, to be invoked by remote thin clients.
Abstract: Data mining is being increasingly used in biology. Biologists are adopting prototyping languages, like R and Matlab, to facilitate the application of data mining algorithms to their data. As a result, their scripts are becoming increasingly complex and also require frequent updates. Application to large datasets becomes impractical and the time-to-paper increases. Furthermore, even if there are various systems that can be used to efficiently process large datasets, for example, using Cloud and High Performance Computing, they usually require procedures to be translated into specific languages or to be adapted to a certain computing platform. Such modifications can speed up the processing, but translation is not automatic, especially in complex cases, and can require a large amount of programming effort and accurate validation. In this paper, we propose an approach to parallelize data mining procedures in the form of compiled software or R scripts developed by biology communities of practice. Our approach requires minimal alteration of the original code. In many cases, there is no need for code modification. Furthermore, it allows for fast updating when a new version is ready. We clarify the constraints and the benefits of our method and report a practical use case to demonstrate such benefits compared with a standard execution. Our approach relies on a distributed network of web services and ultimately exposes the algorithms as-a-Service, to be invoked by remote thin clients. Copyright © 2014 John Wiley & Sons, Ltd.

Journal ArticleDOI
TL;DR: This study evaluated the application of simulated annealing in a multi‐cloud system serving a workload of processes with low parallelism but with high arrival rates and highly variant run‐times and indicated that significant gains both in performance and in cost can be achieved using the proposed scheduling technique.
Abstract: The Internet of Things IoT and Cloud Computing are two novel paradigms both rapidly evolving in their particular areas of application. The former is enabled through several technologies ranging from communication systems to distributed intelligence, whereas the latter provides the means for massively parallel computation on demand. Therefore, we can include Cloud Computing as an enabling factor in the greater picture of the IoT. Given the complexity of IoT's computing concepts, it would be prudent to assume that this complexity will eventually provide computing workloads that are different in kind to the current ones. Thus, it becomes important to study how current scheduling optimization techniques can be adapted to such computing tasks. In this study, we have evaluated the application of simulated annealing in a multi-cloud system serving a workload of processes with low parallelism but with high arrival rates and highly variant run-times. A discrete event simulator was used in order to assess both the performance and the cost of the system. Simulation results indicate that significant gains both in performance and in cost can be achieved using the proposed scheduling technique in this context. Copyright © 2013 John Wiley & Sons, Ltd.

Journal ArticleDOI
TL;DR: The system architecture of the new gateway has been designed to have additional data and meta‐data management facilities to access and manage (biomedical) data servers, and to provide data‐centric user interaction.
Abstract: Summary Science gateways provide UIs and high-level services to access and manage applications and data collections on distributed resources. They facilitate users to perform data analysis on distributed computing infrastructures without getting involved into the technical details. The e-BioInfra Gateway is a science gateway for biomedical data analysis on a national grid infrastructure, which has been successfully adopted for neuroscience research. This paper describes the motivation, requirements, and design of a new generation of e-BioInfra Gateway, which is based on the grid and cloud user support environment (also known as WS-PGRADE/gUSE framework) and supports heterogeneous infrastructures. The new gateway has been designed to have additional data and meta-data management facilities to access and manage (biomedical) data servers, and to provide data-centric user interaction. We have implemented and deployed the new gateway for the computational neuroscience research community of the Academic Medical Center of the University of Amsterdam. This paper presents the system architecture of the new gateway, highlights the improvements that have been achieved, discusses the choices that we have made, and reflects on those based on initial user feedback. Copyright © 2014 John Wiley & Sons, Ltd.

Journal ArticleDOI
TL;DR: A chemical reaction‐inspired computational model using the concepts of graphs and reflection is presented, which attempts to address the complexities associated with the visualisation, modelling, interaction, analysis and abstraction of information in the IoT.
Abstract: The evolution of communication protocols, sensory hardware, mobile and pervasive devices, alongside social and cyber-physical networks, has made the Internet of things IoT an interesting concept with inherent complexities as it is realised. Such complexities range from addressing mechanisms to information management and from communication protocols to presentation and interaction within the IoT. Although existing Internet and communication models can be extended to provide the basis for realising IoT, they may not be sufficiently capable to handle the new paradigms that IoT introduces, such as social communities, smart spaces, privacy and personalisation of devices and information, modelling and reasoning. With interaction models in IoT moving from the orthodox service consumption model, towards an interactive conversational model, nature-inspired computational models appear to be candidate representations. Specifically, this research contests that the reactive and interactive nature of IoT makes chemical reaction-inspired approaches particularly well suited to such requirements. This paper presents a chemical reaction-inspired computational model using the concepts of graphs and reflection, which attempts to address the complexities associated with the visualisation, modelling, interaction, analysis and abstraction of information in the IoT. Copyright © 2013 John Wiley & Sons, Ltd.

Journal ArticleDOI
TL;DR: There are still many open challenges with regard to RS based on Linked Data in order to be efficient for real applications, including personalization of recommendations, use of more datasets considering the heterogeneity introduced, and creation of new hybrid RS for adding information.
Abstract: Recommender systems RS are software tools that use analytic technologies to suggest different items of interest to an end user. Linked Data is a set of best practices for publishing and connecting structured data on the Web. This paper presents a systematic literature review to summarize the state of the art in RS that use structured data published as Linked Data for providing recommendations of items from diverse domains. It considers the most relevant research problems addressed and classifies RS according to how Linked Data have been used to provide recommendations. Furthermore, it analyzes contributions, limitations, application domains, evaluation techniques, and directions proposed for future research. We found that there are still many open challenges with regard to RS based on Linked Data in order to be efficient for real applications. The main ones are personalization of recommendations, use of more datasets considering the heterogeneity introduced, creation of new hybrid RS for adding information, definition of more advanced similarity measures that take into account the large amount of data in Linked Data datasets, and implementation of testbeds to study evaluation techniques and to assess the accuracy scalability and computational complexity of RS. Copyright © 2015 John Wiley & Sons, Ltd.

Journal ArticleDOI
TL;DR: PENNANT is a mini‐app that operates on general unstructured meshes (meshes with arbitrary polygons), and is designed for advanced architecture research, giving an indication of how PENNANT can be a useful tool for studies of new architectures and programming models.
Abstract: This paper describes PENNANT, a mini-app that operates on general unstructured meshes meshes with arbitrary polygons, and is designed for advanced architecture research. It contains mesh data structures and physics algorithms adapted from the Los Alamos National Laboratory radiation-hydrodynamics code FLAG and gives a sample of the typical memory access patterns of FLAG. The basic capabilities and optimization approaches of PENNANT are presented. Results are shown from sample performance experiments run on serial, multicore, and graphics processing unit implementations, giving an indication of how PENNANT can be a useful tool for studies of new architectures and programming models. Copyright © 2014 John Wiley & Sons, Ltd.

Journal ArticleDOI
TL;DR: This paper presents a new method for scheduling efficiently parallel applications with m CPUs and k GPUs, where each task of the application can be processed either on a core (CPU) or on a GPU.
Abstract: More and more computers use hybrid architectures combining multi-core processors and hardware accelerators such as graphics processing units GPUs. We present in this paper a new method for scheduling efficiently parallel applications with m CPUs and k GPUs, where each task of the application can be processed either on a core CPU or on a GPU. The objective is to minimize the maximum completion time makespan. The corresponding scheduling problem is Non-deterministic Polynomial NP-time hard, Copyright © 2014 John Wiley & Sons, Ltd.

Journal ArticleDOI
TL;DR: This work investigates a general tile‐based approach to facilitating fast alignment by deeply exploring the powerful compute capability of CUDA‐enabled GPUs and presents GSWABE, a graphics processing unit (GPU)‐accelerated pairwise sequence alignment algorithm for a collection of short DNA sequences.
Abstract: In this paper, we present GSWABE, a graphics processing unit GPU-accelerated pairwise sequence alignment algorithm for a collection of short DNA sequences. This algorithm supports all-to-all pairwise global, semi-global and local alignment, and retrieves optimal alignments on Compute Unified Device Architecture CUDA-enabled GPUs. All of the three alignment types are based on dynamic programming and share almost the same computational pattern. Thus, we have investigated a general tile-based approach to facilitating fast alignment by deeply exploring the powerful compute capability of CUDA-enabled GPUs. The performance of GSWABE has been evaluated on a Kepler-based Tesla K40 GPU using a variety of short DNA sequence datasets. The results show that our algorithm can yield a performance of up to 59.1 billions cell updates per second GCUPS, 58.5 GCUPS and 50.3 GCUPS for global, semi-global and local alignment, respectively. Furthermore, on the same system GSWABE runs up to 156.0 times faster than the Streaming SIMD Extensions SSE-based SSW library and up to 102.4 times faster than the CUDA-based MSA-CUDA the first stage in terms of local alignment. Compared with the CUDA-based gpu-pairAlign, GSWABE demonstrates stable and consistent speedups with a maximum speedup of 11.2, 10.7, and 10.6 for global, semi-global, and local alignment, respectively. Copyright © 2014 John Wiley & Sons, Ltd.

Journal ArticleDOI
TL;DR: This paper shows a cloud broker architecture for deploying virtualized servers across available clouds that uses a scheduling module to obtain an optimal placement of a virtual infrastructure while making it transparent for users.
Abstract: Summary The number of providers in the cloud computing market is increasing at a rapid pace. They offer a wide range of pricing schemes, different types of instance, or even different value-added features to differ from other competitors, making the cloud market more complex. Cloud brokering enable users to choose the best cloud provider for their needs and avoid particular providers' lock in. Moreover, the use of multiple clouds offers several benefits such as scalability of services, improvement in fault tolerance, or cost reduction. However, users still find difficult the decision of where to deploy their resources as they have to handle too much information. The use of cloud brokering mechanisms is useful to reduce the complexity of using a multi-cloud environment. In this paper, we show a cloud broker architecture for deploying virtualized servers across available clouds. This architecture uses a scheduling module to obtain an optimal placement of a virtual infrastructure while making it transparent for users. We focus our investigation on dynamic cloud scenarios in terms of variable resource prices, taking into account the virtual machine migration overhead issue. We evaluate the performance of several use cases applied in real scenarios, and we show the improvement potential of using brokering mechanisms in dynamic deployments compared with static ones. Copyright © 2012 John Wiley & Sons, Ltd.

Journal ArticleDOI
TL;DR: This approach allows to obtain performance predictions of classical dense linear algebra kernels accurate within a few percents and in a matter of seconds, which allows both runtime and application designers to quickly decide which optimization to enable or whether it is worth investing in higher‐end graphics processing units or not.
Abstract: Summary Multi-core architectures comprising several graphics processing units (GPUs) have become mainstream in the field of high-performance computing. However, obtaining the maximum performance of such heterogeneous machines is challenging as it requires to carefully off-load computations and manage data movements between the different processing units. The most promising and successful approaches so far build on task-based runtimes that abstract the machine and rely on opportunistic scheduling algorithms. As a consequence, the problem gets shifted to choosing the task granularity, task graph structure, and optimizing the scheduling strategies. Trying different combinations of these different alternatives is also itself a challenge. Indeed, obtaining accurate measurements requires reserving the target system for the whole duration of experiments. Furthermore, observations are limited to the few available systems at hand and may be difficult to generalize. In this article, we show how we crafted a coarse-grain hybrid simulation/emulation of StarPU, a dynamic runtime for hybrid architectures, over SimGrid, a versatile simulator of distributed systems. This approach allows to obtain performance predictions of classical dense linear algebra kernels accurate within a few percents and in a matter of seconds, which allows both runtime and application designers to quickly decide which optimization to enable or whether it is worth investing in higher-end graphics processing units or not. Additionally, it allows to conduct robust and extensive scheduling studies in a controlled environment whose characteristics are very close to real platforms while having reproducible behavior. Copyright © 2015 John Wiley & Sons, Ltd.

Journal ArticleDOI
TL;DR: This work proposes a technique for the grid scheduling problem using genetic algorithm with the objective to maximize availability, and results reveal the effectiveness of the model.
Abstract: Computational grid provides a wide distributed platform for high-end compute intensive applications. Grid scheduling is often carried out to schedule the submitted jobs on the nodes of the grid so that some characteristic parameter is optimized. Availability of the computational nodes is one of the important characteristic parameters and measures the probability of the node availability for job execution. This paper addresses the availability of the grid computational nodes for the job execution and proposes a model to maximize it. As such, the task scheduling problem in grid is nondeterministic polynomial-time hard, and often, metaheuristics techniques are applied to solve it. Genetic algorithm, a metaheuristic technique based on evolutionary computation, has been used to solve such complex optimization problem. This work proposes a technique for the grid scheduling problem using genetic algorithm with the objective to maximize availability. Simulation experiment, to evaluate the performance of the proposed algorithm, is conducted, and results reveal the effectiveness of the model. A comparative study has also been performed. Copyright © 2014 John Wiley & Sons, Ltd.

Journal ArticleDOI
TL;DR: A scheme that allocates interdependent tasks and aims to reduce task completion time and the amount of energy consumed in transmission of data is presented, based on a hybrid architecture that results in effective allocation decisions, reduces the communication cost associated with the exchange of control information, and distributes the processing burden among the nodes.
Abstract: A mobile ad hoc computational grid is a distributed computing infrastructure that allows mobile nodes to share computing resources in a mobile ad hoc environment. Compared to traditional distributed systems such as grids and clouds, resource allocation in mobile ad hoc computational grids is not straightforward because of node mobility, limited battery power and an infrastructure-less network environment. The existing schemes are either based on a decentralized architecture that results in poor allocation decisions or assume independent tasks. This paper presents a scheme that allocates interdependent tasks and aims to reduce task completion time and the amount of energy consumed in transmission of data. This scheme comprises two key algorithms: resource selection and resource allocation. The resource selection algorithm is designed to select nodes that remain connected for a longer period, whereas the resource assignment or allocation algorithm is developed to allocate interdependent tasks to the nodes that are accessible at the minimum transmission power. The scheme is based on a hybrid architecture that results in effective allocation decisions, reduces the communication cost associated with the exchange of control information, and distributes the processing burden among the nodes. The paper also investigates the relationship between the data transfer time and transmission energy consumption and presents a power-based routing protocol to reduce data transfer costs and transmission energy consumption. Copyright © 2014 John Wiley & Sons, Ltd.

Journal ArticleDOI
TL;DR: The education and outreach activities in CyberGIS Gateway are presented to illustrate the impact of CyberGis Gateway in GIScience communities and the effective collaboration within the science gateway community.
Abstract: This paper describes CyberGIS Gateway, an online problem-solving environment, for multiple science communities to conduct data-rich geospatial research and education.

Journal ArticleDOI
TL;DR: This work describes how JS4Cloud has been integrated within the data mining cloud framework (DMCF), a system supporting the scalable execution of data analysis workflows on Cloud platforms by exploiting parallelism to enable their scalable execution on Clouds.
Abstract: Workflows are an effective paradigm to model complex data analysis processes, such as knowledge discovery in databases applications, which can be efficiently executed on distributed computing systems such as a Cloud platform. Data analysis workflows can be designed through visual programming, which is a convenient design approach for high-level users. On the other hand, script-based workflows are a useful alternative to visual workflows, because they allow expert users to program complex applications more effectively. In order to provide Cloud users with an effective script-based data analysis workflow formalism, we designed the JS4Cloud language. The main benefits of JS4Cloud are as follows: i it extends the well-known JavaScript language while using only its basic functions arrays, functions, and loops; ii it implements both a data-driven task parallelism that automatically spawns ready-to-run tasks to the Cloud resources and data parallelism through an array-based formalism; and iii these two types of parallelism are exploited implicitly so that workflows can be programmed in a fully sequential way, which frees users from duties like work partitioning, synchronization, and communication. We describe how JS4Cloud has been integrated within the data mining cloud framework DMCF, a system supporting the scalable execution of data analysis workflows on Cloud platforms. In particular, we describe how data analysis workflows modeled as JS4Cloud scripts are processed by DMCF by exploiting parallelism to enable their scalable execution on Clouds. Finally, we present some data analysis workflows developed with JS4Cloud and the performance results obtained by executing such workflows on DMCF. Copyright © 2015John Wiley & Sons, Ltd.