scispace - formally typeset
Search or ask a question
Proceedings ArticleDOI

Nimrod/G: an architecture for a resource management and scheduling system in a global computational grid

14 May 2000-Vol. 1, pp 283-289
TL;DR: The proposed Nimrod/G grid-enabled resource management and scheduling system builds on the earlier work on Nimrod and follows a modular and component-based architecture enabling extensibility, portability, ease of development, and interoperability of independently developed components.
Abstract: The availability of powerful microprocessors and high-speed networks as commodity components has enabled high-performance computing on distributed systems (wide-area cluster computing). In this environment, as the resources are usually distributed geographically at various levels (department, enterprise or worldwide), there is a great challenge in integrating, coordinating and presenting them as a single resource to the user, thus forming a computational grid. Another challenge comes from the distributed ownership of resources, with each resource having its own access policy, cost and mechanism. The proposed Nimrod/G grid-enabled resource management and scheduling system builds on our earlier work on Nimrod (D. Abramson et al., 1994, 1995, 1997, 2000) and follows a modular and component-based architecture enabling extensibility, portability, ease of development, and interoperability of independently developed components. It uses the GUSTO (GlobUS TOolkit) services and can be easily extended to operate with any other emerging grid middleware services. It focuses on the management and scheduling of computations over dynamic resources scattered geographically across the Internet at department, enterprise or global levels, with particular emphasis on developing scheduling schemes based on the concept of computational economy for a real testbed, namely the Globus testbed (GUSTO).

Content maybe subject to copyright    Report

Citations
More filters
Proceedings ArticleDOI
01 Nov 2008
TL;DR: In this article, the authors compare and contrast cloud computing with grid computing from various angles and give insights into the essential characteristics of both the two technologies, and compare the advantages of grid computing and cloud computing.
Abstract: Cloud computing has become another buzzword after Web 2.0. However, there are dozens of different definitions for cloud computing and there seems to be no consensus on what a cloud is. On the other hand, cloud computing is not a completely new concept; it has intricate connection to the relatively new but thirteen-year established grid computing paradigm, and other relevant technologies such as utility computing, cluster computing, and distributed systems in general. This paper strives to compare and contrast cloud computing with grid computing from various angles and give insights into the essential characteristics of both.

3,132 citations

Journal ArticleDOI
TL;DR: This work states that clusters, Grids, and peer‐to‐peer (P2P) networks have emerged as popular paradigms for next generation parallel and distributed computing and introduces a number of resource management and application scheduling challenges in the domain of security, resource and policy heterogeneity, fault tolerance, continuously changing resource conditions, and politics.
Abstract: SUMMARY Clusters, Grids, and peer-to-peer (P2P) networks have emerged as popular paradigms for next generation parallel and distributed computing. They enable aggregation of distributed resources for solving largescale problems in science, engineering, and commerce. In Grid and P2P computing environments, the resources are usually geographically distributed in multiple administrative domains, managed and owned by different organizations with different policies, and interconnected by wide-area networks or the Internet. This introduces a number of resource management and application scheduling challenges in the domain of security, resource and policy heterogeneity, fault tolerance, continuously changing resource conditions, and politics. The resource management and scheduling systems for Grid computing need to manage resources and application execution depending on either resource consumers’ or owners’ requirements, and continuously adapt to changes in resource availability. The management of resources and scheduling of applications in such large-scale distributed systems is a complex undertaking. In order to prove the effectiveness of resource brokers and associated scheduling algorithms, their performance needs to be evaluated under different scenarios such as varying number of resources and users with different requirements. In a Grid environment, it is hard and even impossible to perform scheduler performance evaluation in a repeatable and controllable manner as resources and users are distributed across multiple organizations with their own policies. To overcome this limitation, we have developed a Java-based discrete-event Grid simulation toolkit called GridSim. The toolkit supports modeling and simulation of heterogeneous Grid resources (both time- and space-shared), users and application models. It provides primitives for creation of application tasks, mapping of tasks to resources, and their management. To demonstrate suitability of the GridSim toolkit, we have simulated a Nimrod-G

1,604 citations


Cites background from "Nimrod/G: an architecture for a res..."

  • ...ully manage them. Therefore, in [9][10][11], we investigated on the use of economics as a metaphor for management of resources in gr id computing environments. A grid resource broker, called Nimrod-G [8], has been developed that performs scheduling of parameter sweep, task-farming applications on geographically distributed resources. It supports deadline and budget based scheduling driven by market -...

    [...]

  • ...id libraries or legacy applications that can be grid enabled using user -level middleware tools. The user essentially interacts with a resource broker that hides the complexities of grid computing [7][8]. The broker discovers resources that the user can access using information services, negotiates for access costs using trading services, maps tasks to resources (scheduling), stages the application a...

    [...]

Journal ArticleDOI
TL;DR: The results of improving application performance through workflow restructuring which clusters multiple tasks in a workflow into single entities are presented.
Abstract: This paper describes the Pegasus framework that can be used to map complex scientific workflows onto distributed resources. Pegasus enables users to represent the workflows at an abstract level without needing to worry about the particulars of the target execution systems. The paper describes general issues in mapping applications and the functionality of Pegasus. We present the results of improving application performance through workflow restructuring which clusters multiple tasks in a workflow into single entities. A real-life astronomy application is used as the basis for the study.

1,324 citations


Cites background from "Nimrod/G: an architecture for a res..."

  • ...Nimrod-G [34] is a cost and deadline based resource management and scheduling system....

    [...]

Journal ArticleDOI
TL;DR: In this article, an abstract model and a comprehensive taxonomy for describing resource management architectures is developed, which is used to identify approaches followed in the implementation of existing resource management systems for very large-scale network computing systems known as Grids.
Abstract: The resource management system is the central component of distributed network computing systems. There have been many projects focused on network computing that have designed and implemented resource management systems with a variety of architectures and services. In this paper, an abstract model and a comprehensive taxonomy for describing resource management architectures is developed. The taxonomy is used to identify approaches followed in the implementation of existing resource management systems for very large-scale network computing systems known as Grids. The taxonomy and the survey results are used to identify architectural approaches and issues that have not been fully explored in the research. Copyright © 2001 John Wiley & Sons, Ltd.

993 citations

Journal ArticleDOI
TL;DR: A computational economy framework for resource allocation and for regulating supply and demand in Grid computing environments is proposed and some of the economic models in resource trading and scheduling are demonstrated using the Nimrod/G resource broker.
Abstract: The accelerated development in peer-to-peer and Grid computing has positioned them as promising next-generation computing platforms. They enable the creation of virtual enterprises for sharing resources distributed across the world. However, resource management, application development and usage models in these environments is a complex undertaking. This is due to the geographic distribution of resources that are owned by different organizations or peers. The resource owners of each of these resources have different usage or access policies and cost models, and varying loads and availability. In order to address complex resource management issues, we have proposed a computational economy framework for resource allocation and for regulating supply and demand in Grid computing environments. This framework provides mechanisms for optimizing resource provider and consumer objective functions through trading and brokering services. In a real world market, there exist various economic models for setting the price of services based on supply-and-demand and their value to the user. They include commodity market, posted price, tender and auction models. In this paper, we discuss the use of these models for interaction between Grid components to decide resource service value, and the necessary infrastructure to realize each model. In addition to usual services offered by Grid computing systems, we need an infrastructure to support interaction protocols, allocation mechanisms, currency, secure banking and enforcement services. We briefly discuss existing technologies that provide some of these services and show their usage in developing the Nimrod-G grid resource broker. Furthermore, we demonstrate the effectiveness of some of the economic models in resource trading and scheduling using the Nimrod/G resource broker, with deadline and cost constrained scheduling for two different optimization strategies, on the World-Wide Grid testbed that has resources distributed across five continents.

961 citations


Cites background or methods from "Nimrod/G: an architecture for a res..."

  • ...Nimrod-G [1][2] (Monash University) It supports economy models such as commodity market, spot market, and contract-net for price establishment....

    [...]

  • ...For example, implementation specific details of our Nimrod/G resource broker [1] [2][4] vary from other related systems....

    [...]

  • ...In [2][3][4][5], we proposed and explored the usage of an economics based paradigm for managing resource allocation in Grid computing environments....

    [...]

  • ...A detailed discussion on the Nimrod system architecture and implementation [1][2], scheduling algorithms [4], and its ability to execute real world applications such as ionization chamber calibration [1] and drug design [34] on the Grid can be found elsewhere....

    [...]

  • ...They include Mariposa [8], Mungi [17], Popcorn [21], Java Market [18], Enhanced MOSIX [19], JaWS [30], Xenoservers [31], D’Agents [32], Rexec/Anemone [22], Spawn [20], Mojo Nation [24], and Nimrod-G [1][2]....

    [...]

References
More filters
Journal ArticleDOI
01 Jun 1997
TL;DR: The Globus system is intended to achieve a vertically integrated treatment of application, middleware, and net work, an integrated set of higher level services that enable applications to adapt to heteroge neous and dynamically changing metacomputing environ ments.
Abstract: The Globus system is intended to achieve a vertically integrated treatment of application, middleware, and net work. A low-level toolkit provides basic mechanisms such as communication, authentication, network information, and data access. These mechanisms are used to con struct various higher level metacomputing services, such as parallel programming tools and schedulers. The long- term goal is to build an adaptive wide area resource environment AWARE, an integrated set of higher level services that enable applications to adapt to heteroge neous and dynamically changing metacomputing environ ments. Preliminary versions of Globus components were deployed successfully as part of the I-WAY networking experiment.

3,450 citations


"Nimrod/G: an architecture for a res..." refers background in this paper

  • ...…scheduling of computations over dynamic resources scattered geographically across the Internet at department, enterprise, or global level with particular emphasis on developing scheduling schemes based on the concept of computational economy for a real test bed, namely, the Globus testbed (GUSTO)....

    [...]

Journal ArticleDOI
TL;DR: The current implementation of the NWS for Unix and TCP/IP sockets is described and examples of its performance monitoring and forecasting capabilities are provided.

1,414 citations

Book
01 May 1999
TL;DR: This book brings together contributions from more than 100 leading practitioners, offering a single source for up-to-the-minute information on virtually every key system-related issue in high performance cluster computing.
Abstract: From the Publisher: Rapid improvements in network and processor performance are revolutionizing high performance computing, transforming clustered commodity workstations into the supercomputing solution of choice. This book brings together contributions from more than 100 leading practitioners, offering a single source for up-to-the-minute information on virtually every key system-related issue in high performance cluster computing. The book contains expert coverage of "commodity supercomputing" systems and architectures; Internet-based wide area "metacomputing" systems; the role of Java; new applications and algorithms; advanced techniques for enhancing availability and throughput, and much more.

588 citations


"Nimrod/G: an architecture for a res..." refers background in this paper

  • ...1 The proposed Nimrod/G grid-enabled resource management and scheduling system builds on our earlier work on Nimrod and follows a modular and component-based architecture enabling extensibility, portability, ease of development, and interoperability of independently developed components....

    [...]

Proceedings Article
01 Jan 2000
TL;DR: The role of parametric modeling as an application for the global computing grid is examined, and some heuristics which make it possible to specific soft real time deadlines for larger computational experiments are explored.
Abstract: The evolution of a particular tool, Nimrod, from a local computing environment to the global computational grid is discussed. The various services of Globus and how these were applied in building a grid aware application are described. The algorithm used is both simple and adaptive to changes in the workload distribution on the grid and incorporates user requirements as well as system ones. Results show that it is possible to build an application which takes account of the highly dynamic and unpredictable nature of the grid.

517 citations


"Nimrod/G: an architecture for a res..." refers background in this paper

  • ...…scheduling of computations over dynamic resources scattered geographically across the Internet at department, enterprise, or global level with particular emphasis on developing scheduling schemes based on the concept of computational economy for a real test bed, namely, the Globus testbed (GUSTO)....

    [...]

  • ...The parameters to be considered include, • Resource Architecture and Configuration • Resource Capability (clock speed, memory size) • Resource State (such as CPU load, memory available, disk storage free) • Resource Requirements of an Application • Access Speed (such as disk access speed) • Free or…...

    [...]

  • ...The parameters to be considered include, • Resource Architecture and Configuration • Resource Capability (clock speed, memory size) • Resource State (such as CPU load, memory available, disk storage free) • Resource Requirements of an Application • Access Speed (such as disk access speed) • Free or Available Nodes • Priority (that the user has) • Queue Type and Length • Network Bandwidth, Load, and Latency (if jobs need to communicate) • Reliability of Resource and Connection • User Preference • Application Deadline • User Capacity/Willingness to Pay for Resource Usage • Resource Cost (in terms of dollars that the user need to pay to the resource owner) • Resource Cost Variation in terms of Time-scale (like high @ daytime and low @ night) • Historical Information, including Job Consumption Rate The important parameters of computational economy that can influence the way resource scheduling is done are: • Resource Cost (set by its owner) • Price (that the user is willing to pay) • Deadline (the period by which an application execution need to completed) The scheduler can use all sorts of information gathered by a resource discoverer and also negotiate with resource owners to get the best “value for money”....

    [...]

Book
01 Jan 1999
TL;DR: Rajkumar Buyya provides an authoritative overview of the field and its relevant state-of-the art research directions, offering in-depth coverage for scientists and engineers engaged in the research, development, and application of high-performance computing systems.
Abstract: IEEE Concurrency Because high-performance cluster computing is a relatively new area, few books successfully cover the topic. Rajkumar Buyya provides an authoritative overview of the field and its relevant state-of-the art research directions. Both volumes stem from his interaction with leading researchers, offering in-depth coverage for scientists and engineers engaged in the research, development, and application of high-performance computing systems. (Buyya’s Web site, www.dgs.monash.edu.au/~rajkumar/cluster/ index.html, offers a wealth of additional information.) Graduate students should also find these two books to be extremely useful, especially when exploring research topics.

516 citations


"Nimrod/G: an architecture for a res..." refers background in this paper

  • ...This technology opportunity leads to the possibility of using networks of computers as a single, unified computing resource popularly called as cluster computing [ 7 ]....

    [...]