A unified view of metaheuristics This book provides a complete background on metaheuristics and shows readers how to design and implement efficient algorithms to solve complex optimization problems across a diverse range of applications, from networking and bioinformatics to engineering design, routing, and scheduling. It presents the main design questions for all families of metaheuristics and clearly illustrates how to implement the algorithms under a software framework to reuse both the design and code. Throughout the book, the key search components of metaheuristics are considered as a toolbox for: Designing efficient metaheuristics (e.g. local search, tabu search, simulated annealing, evolutionary algorithms, particle swarm optimization, scatter search, ant colonies, bee colonies, artificial immune systems) for optimization problems Designing efficient metaheuristics for multi-objective optimization problems Designing hybrid, parallel, and distributed metaheuristics Implementing metaheuristics on sequential and parallel machines Using many case studies and treating design and implementation independently, this book gives readers the skills necessary to solve large-scale optimization problems quickly and efficiently. It is a valuable reference for practicing engineers and researchers from diverse areas dealing with optimization or machine learning; and graduate students in computer science, operations research, control, engineering, business and management, and applied mathematics.

/pdf/metaheuristics-from-design-to-implementation-1vqdfv8nk9.pdf

Metaheuristics: From Design to Implementation

The Globus Toolkit (GT) has been developed since the late 1990s to support the development of service-oriented distributed computing applications and infrastructures. Core GT components address, within a common framework, basic issues relating to security, resource access, resource management, data movement, resource discovery, and so forth. These components enable a broader “Globus ecosystem” of tools and components that build on, or interoperate with, core GT functionality to provide a wide range of useful application-level functions. These tools have in turn been used to develop a wide range of both “Grid” infrastructures and distributed applications. I summarize here the principal characteristics of the latest release, the Web services-based GT4, which provides significant improvements over previous releases in terms of robustness, performance, usability, documentation, standards compliance, and functionality.

https://cs.uwaterloo.ca/~brecht/courses/702/Possible-Readings/grid-computing/globus-v4-icnpc-2005.pdf

Globus toolkit version 4: software for service-oriented systems

Many believe that Cloud will reshape the entire ICT industry as a revolution. In this paper, we aim to pinpoint the challenges and issues of Cloud computing. We first discuss two related computing paradigms - Service-Oriented Computing and Grid computing, and their relationships with Cloud computing We then identify several challenges from the Cloud computing adoption perspective. Last, we will highlight the Cloud interoperability issue that deserves substantial further research and development.

Cloud Computing: Issues and Challenges

One of the many definitions of "cloud" is that of an infrastructure-as-a-service (IaaS) system, in which IT infrastructure is deployed in a provider's data center as virtual machines. With IaaS clouds' growing popularity, tools and technologies are emerging that can transform an organization's existing infrastructure into a private or hybrid cloud. OpenNebula is an open source, virtual infrastructure manager that deploys virtualized services on both a local pool of resources and external IaaS clouds. Haizea, a resource lease manager, can act as a scheduling back end for OpenNebula, providing features not found in other cloud software or virtualization-based data center management software.

Virtual Infrastructure Management in Private and Hybrid Clouds

Cloud computing is an emerging commercial infrastructure paradigm that promises to eliminate the need for maintaining expensive computing facilities by companies and institutes alike. Through the use of virtualization and resource time sharing, clouds serve with a single set of physical resources a large user base with different needs. Thus, clouds have the potential to provide to their owners the benefits of an economy of scale and, at the same time, become an alternative for scientists to clusters, grids, and parallel production environments. However, the current commercial clouds have been built to support web and small database workloads, which are very different from typical scientific computing workloads. Moreover, the use of virtualization and resource time sharing may introduce significant performance penalties for the demanding scientific computing workloads. In this work, we analyze the performance of cloud computing services for scientific computing workloads. We quantify the presence in real scientific computing workloads of Many-Task Computing (MTC) users, that is, of users who employ loosely coupled applications comprising many tasks to achieve their scientific goals. Then, we perform an empirical evaluation of the performance of four commercial cloud computing services including Amazon EC2, which is currently the largest commercial cloud. Last, we compare through trace-based simulation the performance characteristics and cost models of clouds and other scientific computing platforms, for general and MTC-based scientific computing workloads. Our results indicate that the current clouds need an order of magnitude in performance improvement to be useful to the scientific community, and show which improvements should be considered first to address this discrepancy between offer and demand.

/pdf/performance-analysis-of-cloud-computing-services-for-many-3e3awq09a7.pdf

Performance Analysis of Cloud Computing Services for Many-Tasks Scientific Computing

As cluster computers are used for a wider range of applications, we encounter the need to deliver resources at particular times, to meet particular deadlines, and/or at the same time as other resources are provided elsewhere. To address such requirements, we describe a scheduling approach in which users request resource leases, where leases can request either as-soon-as-possible ("best-effort") or reservation start times. We present the design of a lease management architecture, Haizea, that implements leases as virtual machines (VMs), leveraging their ability to suspend, migrate, and resume computations and to provide leased resources with customized application environments. We discuss methods to minimize the overhead introduced by having to deploy VM images before the start of a lease. We also present the results of simulation studies that compare alternative approaches. Using workloads with various mixes of best-effort and advance reservation requests, we compare the performance of our VM-based approach with that of non-VM-based schedulers. We find that a VM-based approach can provide better performance (measured in terms of both total execution time and average delay incurred by best-effort requests) than a scheduler that does not support task pre-emption, and only slightly worse performance than a scheduler that does support task pre-emption. We also compare the impact of different VM image popularity distributions and VM image caching strategies on performance. These results emphasize the importance of VM image caching for the workloads studied and quantify the sensitivity of scheduling performance to VM image popularity distribution.

/pdf/combining-batch-execution-and-leasing-using-virtual-machines-ekw1vpaw38.pdf

Combining batch execution and leasing using virtual machines

Clouds can be used to provide on-demand capacity as a utility. Although the realization of this idea can differ among various cloud providers (from Google App Engine to Amazon EC2), the most flexible approach is the provisioning of virtualized resources as a service. These virtualization-based clouds, like Amazon EC2 or the Science Clouds (which uses the Globus Virtual Workspace Service [4]), provide a way to build a large computing infrastructure by accessing remote computational, storage and network resources. Since a cloud typically comprises a large amount of virtual and physical servers, in the order of hundreds or thousands, efficiently managing this virtual infrastructure becomes a major concern. Several solutions, such as VMWare VirtualCenter, Platform Orchestrator, or Enomalism, have emerged to manage virtual infrastructures, providing a centralized control platform for the automatic deployment and monitoring of virtual machines (VMs) in resource pools. However, these solutions provide simple VM placement and load balancing policies. In particular, existing clouds use an immediate provisioning model, where virtualized resources are allocated at the time they are requested, without the possibility of requesting resources at a specific future time and, at most, being placed in a simple first-come-first-serve queue when no resources are available. However, service provisioning clouds, like the one being built by the RESERVOIR project, have requirements that cannot be supported within this model, such as resource requests that are subject to non-trivial policies, capacity reservations at specific times to meet peak capacity requirements, variable resource usage throughout a VM’s lifetime, and dynamic renegotiation of resources allocated to VMs. Additionally, smaller clouds with limited resources, where not all requests may be satisfiable immediately for lack of resources, could benefit from more complex VM placement strategies supporting queues, priorities, and advance reservations. In this work we explore extending the capacity provisioning model used in current clouds by using resource leases [3, 10, 9] as a fundamental provisioning abstraction. To do this, we have integrated the OpenNebula virtual infrastructure engine with the Haizea lease manager to produce a resource management system that can be used to support a variety of leases in clouds. We focus in this work on advance reservation leases, which can be used to satisfy capacity peaks known in advance, or for a variety of well-documented use cases where advance reservations are used (such as coscheduling of multiple resources [12, 5, 1, 2], urgent

/pdf/capacity-leasing-in-cloud-systems-using-the-opennebula-rs89x7gjul.pdf

Capacity Leasing in Cloud Systems using the OpenNebula Engine

Foreword / Preface / ((PART 1: KEY CONCEPTS)) / CH 1: Grid Computing / CH 2: OGSA, WSRF, and GT4 / CH 3: Web Services / CH 4: WSRF / CH 5: The Globus Toolkit 4 / ((PART II: GT JAVA WS CORE)) / CH 6: Writing Your First Stateful Web Service in 5 Simple Steps / CH 7: Singleton Resources / CH 8: Multiple Resources / CH 9: Logging / CH 10: Resource Properties / CH 11: Lifecycle Management / CH 12: Persistent Resources / CH 13: Notifications / CH 14: Implementing Your Own Operation Providers / ((PART III: GT4 SECURITY)) / CH 15: Fundamental Security Concepts / CH 16: GSI: Grid Security Concepts / CH 17: Writing a Secure Math Service / CH 18: The Security Descriptor / CH 19: Authentication / CH 20: Authorization / CH 21: Resource-Level Security / CH 22: Run-As Modes and Delegation / ((PART IV: The File Buy Application)) / CH 23: Design / CH 24: Implementation / ((Conclusion: The Next Step: Higher-Level Services)) / ((PART V: Appendices)) / Appendix A: Installing the Globus Toolkit 4 / Appendix B: A WSDL Primer / Appendix C: Command-line Clients / Appendix D: Examples / Appendix E: Globus-Build-Service Script Reference / Reference

Globus toolkit 4 : programming Java services

Using virtual machines as a resource provisioning mechanism offers multiple benefits, most recently exploited by "infrastructure-as-a-service" clouds, but also poses several scheduling challenges. More specifically, although we can use the suspend/resume/migrate capability of virtual machines to support advance reservation of resources efficiently, by using suspension/resumption as a preemption mechanism, this requires adequately modeling the time and resources consumed by these operations to ensure that preemptions are completed before the start of a reservation. In this work we present a model for predicting various runtime overheads involved in using virtual machines, allowing us to efficiently support advance reservations. We extend our lease management software, Haizea, to use this new model in its scheduling decisions, and we use Haizea with the OpenNebula virtual infrastructure manager so the scheduling decisions will be enacted in a Xen cluster. We present both physical and simulated experimental results showing the degree of accuracy of our model and the long-term effects of variables in our model on several workloads.

/pdf/resource-leasing-and-the-art-of-suspending-virtual-machines-2if0riy37y.pdf

Borja Sotomayor

Papers

Virtual Infrastructure Management in Private and Hybrid Clouds

Combining batch execution and leasing using virtual machines

Capacity Leasing in Cloud Systems using the OpenNebula Engine

Globus toolkit 4 : programming Java services

Resource Leasing and the Art of Suspending Virtual Machines