scispace - formally typeset
Search or ask a question

Showing papers on "Resource management published in 2011"


Journal ArticleDOI
TL;DR: The result of this case study proves that the federated Cloud computing model significantly improves the application QoS requirements under fluctuating resource and service demand patterns.
Abstract: Cloud computing is a recent advancement wherein IT infrastructure and applications are provided as ‘services’ to end-users under a usage-based payment model. It can leverage virtualized services even on the fly based on requirements (workload patterns and QoS) varying with time. The application services hosted under Cloud computing model have complex provisioning, composition, configuration, and deployment requirements. Evaluating the performance of Cloud provisioning policies, application workload models, and resources performance models in a repeatable manner under varying system and user configurations and requirements is difficult to achieve. To overcome this challenge, we propose CloudSim: an extensible simulation toolkit that enables modeling and simulation of Cloud computing systems and application provisioning environments. The CloudSim toolkit supports both system and behavior modeling of Cloud system components such as data centers, virtual machines (VMs) and resource provisioning policies. It implements generic application provisioning techniques that can be extended with ease and limited effort. Currently, it supports modeling and simulation of Cloud computing environments consisting of both single and inter-networked clouds (federation of clouds). Moreover, it exposes custom interfaces for implementing policies and provisioning techniques for allocation of VMs under inter-networked Cloud computing scenarios. Several researchers from organizations, such as HP Labs in U.S.A., are using CloudSim in their investigation on Cloud resource provisioning and energy-efficient management of data center resources. The usefulness of CloudSim is demonstrated by a case study involving dynamic provisioning of application services in the hybrid federated clouds environment. The result of this case study proves that the federated Cloud computing model significantly improves the application QoS requirements under fluctuating resource and service demand patterns. Copyright © 2010 John Wiley & Sons, Ltd.

4,570 citations


Journal ArticleDOI
TL;DR: The results show that by proper resource management, D2D communication can effectively improve the total throughput without generating harmful interference to cellular networks.
Abstract: We consider Device-to-Device (D2D) communication underlaying cellular networks to improve local services. The system aims to optimize the throughput over the shared resources while fulfilling prioritized cellular service constraints. Optimum resource allocation and power control between the cellular and D2D connections that share the same resources are analyzed for different resource sharing modes. Optimality is discussed under practical constraints such as minimum and maximum spectral efficiency restrictions, and maximum transmit power or energy limitation. It is found that in most of the considered cases, optimum power control and resource allocation for the considered resource sharing modes can either be solved in closed form or searched from a finite set. The performance of the D2D underlay system is evaluated in both a single-cell scenario, and a Manhattan grid environment with multiple WINNER II A1 office buildings. The results show that by proper resource management, D2D communication can effectively improve the total throughput without generating harmful interference to cellular networks.

1,093 citations


Journal ArticleDOI
TL;DR: Basic concepts of energy-efficient communications are first introduced and then existing fundamental works and advanced techniques for energy efficiency are summarized, including information-theoretic analysis, OFDMA networks, MIMO techniques, relay transmission, and resource allocation for signaling.
Abstract: With explosive growth of high-data-rate applications, more and more energy is consumed in wireless networks to guarantee quality of service. Therefore, energy-efficient communications have been paid increasing attention under the background of limited energy resource and environmental- friendly transmission behaviors. In this article, basic concepts of energy-efficient communications are first introduced and then existing fundamental works and advanced techniques for energy efficiency are summarized, including information-theoretic analysis, OFDMA networks, MIMO techniques, relay transmission, and resource allocation for signaling. Some valuable topics in energy-efficient design are also identified for future research.

753 citations


Proceedings ArticleDOI
04 Jul 2011
TL;DR: A model-predictive algorithm for workload forecasting that is used for resource auto scaling is developed and empirical results are provided that demonstrate that resources can be allocated and deal located by the algorithm in a way that satisfies both the application QoS while keeping operational costs low.
Abstract: Large-scale component-based enterprise applications that leverage Cloud resources expect Quality of Service(QoS) guarantees in accordance with service level agreements between the customer and service providers. In the context of Cloud computing, auto scaling mechanisms hold the promise of assuring QoS properties to the applications while simultaneously making efficient use of resources and keeping operational costs low for the service providers. Despite the perceived advantages of auto scaling, realizing the full potential of auto scaling is hard due to multiple challenges stemming from the need to precisely estimate resource usage in the face of significant variability in client workload patterns. This paper makes three contributions to overcome the general lack of effective techniques for workload forecasting and optimal resource allocation. First, it discusses the challenges involved in auto scaling in the cloud. Second, it develops a model-predictive algorithm for workload forecasting that is used for resource auto scaling. Finally, empirical results are provided that demonstrate that resources can be allocated and deal located by our algorithm in a way that satisfies both the application QoS while keeping operational costs low.

605 citations


Journal ArticleDOI
TL;DR: Key elements, processes, and issues in adaptive decision making are highlighted in terms of this framework, and special emphasis is given to the question of geographic scale, the difficulties presented by non-stationarity, and organizational challenges in implementing adaptive management.

556 citations


Proceedings ArticleDOI
10 Apr 2011
TL;DR: This first detailed measurement analysis of network resource usage and subscriber behavior using a large-scale data set collected inside a nationwide 3G cellular data network delivers important insights into network-wide resource usage.
Abstract: We conduct the first detailed measurement analysis of network resource usage and subscriber behavior using a large-scale data set collected inside a nationwide 3G cellular data network. The data set tracks close to a million subscribers over thousands of base stations. We analyze individual subscriber behaviors and observe a significant variation in network usage among subscribers. We characterize subscriber mobility and temporal activity patterns and identify their relation to traffic volume. We then investigate how efficiently radio resources are used by different subscribers as well as by different applications. We also analyze the network traffic from the point of view of the base stations and find significant temporal and spatial variations in different parts of the network, while the aggregated behavior appears predictable. Broadly, our observations deliver important insights into network-wide resource usage. We describe implications in pricing, protocol design and resource and spectrum management.

438 citations


Journal ArticleDOI
TL;DR: The conceptual underpinnings of adaptive management are simple; there will always be inherent uncertainty and unpredictability in the dynamics and behavior of complex social-ecological systems, but management decisions must still be made, and whenever possible, learning should incorporate learning into management.

433 citations


Journal ArticleDOI
TL;DR: This work outlines nine pathologies and challenges that can lead to failure in adaptive management programs and focuses on general sources of failures in adaptivemanagement, so that others can avoid these pitfalls in the future.

278 citations


Book
01 Aug 2011
TL;DR: In this paper, the social fabric of environmental governance is uncovered by uncovering the social networks and natural resource management in social networks, and natural resources management is used as a metaphor for environmental governance.
Abstract: Social networks and natural resource management : uncovering the social fabric of environmental governance

276 citations


Journal ArticleDOI
TL;DR: This work calculates the specific water consumption and land use for the production of 160 crops and crop groups, covering most harvested mass on global cropland, and quantifies indicators for land and water scarcity with high geospatial resolution.
Abstract: Global crop production is causing pressure on water and land resources in many places. In addition to local resource management, the related environmental impacts of commodities traded along international supply chains need to be considered and managed accordingly. For this purpose, we calculate the specific water consumption and land use for the production of 160 crops and crop groups, covering most harvested mass on global cropland. We quantify indicators for land and water scarcity with high geospatial resolution. This facilitates spatially explicit crop-specific resource management and regionalized life cycle assessment of processed products. The vast cultivation of irrigated wheat, rice, cotton, maize, and sugar cane, which are major sources of food, bioenergy, and fiber, drives worldwide water scarcity. According to globally averaged production, substituting biofuel for crude oil would have a lower impact on water resources than substituting cotton for polyester. For some crops, water scarcity impac...

243 citations


Journal ArticleDOI
TL;DR: The potential of MSE to transform terrestrial conservation is reviewed, emphasizing that the behavior of individual harvesters must be included because harvester compliance with management rules has been a major challenge in conservation.
Abstract: The poor management of natural resources has led in many cases to the decline and extirpation of populations. Recent advances in fisheries science could revolutionize management of harvested stocks by evaluating management scenarios in a virtual world by including stakeholders and by assessing its robustness to uncertainty. These advances have been synthesized into a framework, management strategy evaluation (MSE), which has hitherto not been used in terrestrial conservation. We review the potential of MSE to transform terrestrial conservation, emphasizing that the behavior of individual harvesters must be included because harvester compliance with management rules has been a major challenge in conservation. Incorporating resource user decision-making required to make MSEs relevant to terrestrial conservation will also advance fisheries science.

Proceedings ArticleDOI
12 Dec 2011
TL;DR: In this article, the authors present the vision, challenges, and architectural elements of SLA-oriented resource management for cloud computing systems, and propose an architecture that supports integration of market-based provisioning policies and virtualization technologies for flexible allocation of resources to applications.
Abstract: Cloud computing systems promise to offer subscription-oriented, enterprise-quality computing services to users worldwide With the increased demand for delivering services to a large number of users, they need to offer differentiated services to users and meet their quality expectations Existing resource management systems in data centers are yet to support Service Level Agreement (SLA)-oriented resource allocation, and thus need to be enhanced to realize cloud computing and utility computing In addition, no work has been done to collectively incorporate customer-driven service management, computational risk management, and autonomic resource management into a market-based resource management system to target the rapidly changing enterprise requirements of Cloud computing This paper presents vision, challenges, and architectural elements of SLA-oriented resource management The proposed architecture supports integration of market-based provisioning policies and virtualisation technologies for flexible allocation of resources to applications The performance results obtained from our working prototype system shows the feasibility and effectiveness of SLA-based resource provisioning in Clouds

Patent
Navendu Jain1, Ishai Menache1
27 Jun 2011
TL;DR: In this article, a system for managing allocation of resources based on service level agreements between application owners and cloud operators is proposed, where the cloud operator may have responsibility for managing resource allocation to the software application and may manage the allocation such that the application executes within an agreed performance level.
Abstract: A system for managing allocation of resources based on service level agreements between application owners and cloud operators. Under some service level agreements, the cloud operator may have responsibility for managing allocation of resources to the software application and may manage the allocation such that the software application executes within an agreed performance level. Operating a cloud computing platform according to such a service level agreement may alleviate for the application owners the complexities of managing allocation of resources and may provide greater flexibility to cloud operators in managing their cloud computing platforms.

Journal ArticleDOI
TL;DR: A minimum variance-based spectrum decision is proposed for real-time applications, which minimizes the capacity variance of the decided spectrum bands subject to the capacity constraints, and a dynamic resource management scheme is developed to coordinate the spectrum decision adaptively dependent on the time-varying cognitive radio network capacity.
Abstract: Cognitive radio networks have been proposed as a solution to both spectrum inefficiency and spectrum scarcity problems. However, they face to a unique challenge based on the fluctuating nature of heterogeneous spectrum bands as well as the diverse service requirements of various applications. In this paper, a spectrum decision framework is proposed to determine a set of spectrum bands by considering the application requirements as well as the dynamic nature of spectrum bands. To this end, first, each spectrum is characterized by jointly considering primary user activity and spectrum sensing operations. Based on this, a minimum variance-based spectrum decision is proposed for real-time applications, which minimizes the capacity variance of the decided spectrum bands subject to the capacity constraints. For best-effort applications, a maximum capacity-based spectrum decision is proposed where spectrum bands are decided to maximize the total network capacity. Moreover, a dynamic resource management scheme is developed to coordinate the spectrum decision adaptively dependent on the time-varying cognitive radio network capacity. Simulation results show that the proposed methods provide efficient bandwidth utilization while satisfying service requirements.

Proceedings ArticleDOI
05 Dec 2011
TL;DR: This paper considers the case of a single cloud provider and addresses the question how to best match customer demand in terms of both supply and price in order to maximize the providers revenue and customer satisfactions while minimizing energy cost.
Abstract: The advent of cloud computing promises to provide computational resources to customers like public utilities such as water and electricity. To deal with dynamically fluctuating resource demands, market-driven resource allocation has been proposed and recently implemented by public Infrastructure-as-a-Service (IaaS) providers like Amazon EC2. In this environment, cloud resources are offered in distinct types of virtual machines (VMs) and the cloud provider runs an auction-based market for each VM type with the goal of achieving maximum revenue over time. However, as demand for each type of VMs can fluctuate over time, it is necessary to adjust the capacity allocated to each VM type to match the demand in order to maximize total revenue while minimizing the energy cost. In this paper, we consider the case of a single cloud provider and address the question how to best match customer demand in terms of both supply and price in order to maximize the providers revenue and customer satisfactions while minimizing energy cost. In particular, we model this problem as a constrained discrete-time optimal control problem and use Model Predictive Control (MPC) to find its solution. Simulation studies using real cloud workloads indicate that under dynamic workload conditions, our proposed solution achieves higher net income than static allocation strategies and minimizes the average request waiting time.

Journal ArticleDOI
TL;DR: A simple channel sensing order for secondary users in multi-channel CRNs without a priori knowledge of primary user activities is proposed, and it is observed that the total throughput and resource utilization increase with the number of secondary user pairs due to increased transmission opportunities and multi-user diversity.
Abstract: In cognitive radio networks (CRNs), effective and efficient channel exploitation is imperative for unlicensed secondary users to seize available network resources and improve resource utilization In this paper, we propose a simple channel sensing order for secondary users in multi-channel CRNs without a priori knowledge of primary user activities By sensing the channels according to the descending order of their achievable rates with optimal stopping, we show that the proposed channel exploitation approach is efficient yet effective in elevating throughput and resource utilization Simulation results show that our proposed channel exploitation approach outperforms its counterparts by up to 18% in a single-secondary user pair scenario In addition, we investigate the probability of packet transmission collision in a multi-secondary user pair scenario, and show that the probability of collision decreases as the number of channels increases and/or the number of secondary user pairs decreases It is observed that the total throughput and resource utilization increase with the number of secondary user pairs due to increased transmission opportunities and multi-user diversity Our results also demonstrate that resource utilization can be further improved via the proposed channel exploitation approach when the number of secondary user pairs approaches the number of channels

Proceedings ArticleDOI
15 May 2011
TL;DR: The simulation results show that the proposed method is the optimal resource allocation method when the D2D pair locates at the most part of the cell area in both uplink and downlink.
Abstract: Device-to-Device (D2D) communication will become an important technology in future networks with the increase of the requirements of local communication services. The interference between cellular communication and D2D communication can be coordinated by proper power control and resource allocation. In this paper, we analyze the resource allocation methods for D2D communication underlaying cellular networks. A novel resource allocation method that D2D can reuse the resources of more than one cellular user is proposed. After that, we discuss the selection of the optimal resource allocation method from the proposed method and the conventional methods. Finally, the performance of different methods is evaluated through numerical simulation. The simulation results show that the proposed method is the optimal resource allocation method when the D2D pair locates at the most part of the cell area in both uplink and downlink. The proposed method can improve the sum throughput of cellular communication and D2D communication significantly.

Journal ArticleDOI
TL;DR: This paper outlines specific objectives in each of these three "pillars" that, if incorporated into a management plan, will improve the plan's likelihood of sustainability and encourages agency directors and policy-makers to consider sustainability principles when developing funding schemes, management agendas, and policy.

Proceedings ArticleDOI
11 Apr 2011
TL;DR: The performance results indicate that SmartSLA can successfully compute predictive models under different hardware resource allocations, as well as database specific resources, such as the number of replicas in the database systems.
Abstract: In a cloud computing environment, resources are shared among different clients. Intelligently managing and allocating resources among various clients is important for system providers, whose business model relies on managing the infrastructure resources in a cost-effective manner while satisfying the client service level agreements (SLAs). In this paper, we address the issue of how to intelligently manage the resources in a shared cloud database system and present SmartSLA, a cost-aware resource management system. SmartSLA consists of two main components: the system modeling module and the resource allocation decision module. The system modeling module uses machine learning techniques to learn a model that describes the potential profit margins for each client under different resource allocations. Based on the learned model, the resource allocation decision module dynamically adjusts the resource allocations in order to achieve the optimum profits. We evaluate SmartSLA by using the TPC-W benchmark with workload characteristics derived from real-life systems. The performance results indicate that SmartSLA can successfully compute predictive models under different hardware resource allocations, such as CPU and memory, as well as database specific resources, such as the number of replicas in the database systems. The experimental results also show that SmartSLA can provide intelligent service differentiation according to factors such as variable workloads, SLA levels, resource costs, and deliver improved profit margins.

Journal ArticleDOI
TL;DR: Resource allocation issues are investigated in this paper for multiuser wireless transmissions based on orthogonal frequency division multiplexing (OFDM) using convex and stochastic optimization tools.
Abstract: Resource allocation issues are investigated in this paper for multiuser wireless transmissions based on orthogonal frequency division multiplexing (OFDM). Relying on convex and stochastic optimization tools, the novel approach to resource allocation includes: i) development of jointly optimal subcarrier, power, and rate allocation for weighted sum-average-rate maximization; ii) judicious formulation and derivation of the optimal resource allocation for maximizing the utility of average user rates; and iii) development of the stochastic resource allocation schemes, and rigorous proof of their convergence and optimality. Simulations are also provided to demonstrate the merits of the novel schemes.

Proceedings ArticleDOI
04 Jun 2011
TL;DR: This paper is the first to propose mechanisms that both manage the shared resources of a multi-core chip to obtain high-performance and fairness, and also exploit prefetching, and it is shown that these mechanisms improve the performance of a 4-core system that uses network fair queuing, parallelism-aware batch scheduling, and fairness via source throttling.
Abstract: Chip multiprocessors (CMPs) share a large portion of the memory subsystem among multiple cores Recent proposals have addressed high-performance and fair management of these shared resources; however, none of them take into account prefetch requests Without prefetching, significant performance is lost, which is why existing systems prefetch By not taking into account prefetch requests, recent shared-resource management proposals often significantly degrade both performance and fairness, rather than improve them in the presence of prefetching This paper is the first to propose mechanisms that both manage the shared resources of a multi-core chip to obtain high-performance and fairness, and also exploit prefetching We apply our proposed mechanisms to two resource-based management techniques for memory scheduling and one source-throttling-based management technique for the entire shared memory system We show that our mechanisms improve the performance of a 4-core system that uses network fair queuing, parallelism-aware batch scheduling, and fairness via source throttling by 110%, 109%, and 113% respectively, while also significantly improving fairness

Journal ArticleDOI
TL;DR: In this paper, two separate mechanisms have been described in the literature, resource acquisition in strategic factor markets and internal resource accumulation, and they discuss several issues that are critical to developing a more complete theoretical and practical understanding of the creation of heterogeneous resource positions.

Journal ArticleDOI
TL;DR: In this article, the authors explore how environmental contingencies determine the way resources are accumulated in young technology-based firms and argue that growth paths are critically shaped at the nexus between resource management and the competitive environment, defined along its most important dimensions, namely stability and complexity.
Abstract: We explore how environmental contingencies determine the way resources are accumulated in young technology-based firms and argue that growth paths are critically shaped at the nexus between resource management and the competitive environment, defined along its most important dimensions, ‘stability’ and ‘complexity.’ We also build propositions about the way environmental conditions affect resource portfolio development or acquisition. We show how particular high-growth paths result from structuring resource portfolios in accordance with environmental demands and provide insights into why, based on six case studies of young technology-based high-growth firms, involving 27 interviews, 121 press releases, 605 press articles, and archival data. Copyright © 2011 Strategic Management Society.

Journal ArticleDOI
Jim Andersén1
TL;DR: In this article, the relationship between strategic resources and firm performance is examined and the traditional VRIO attributes have been the point of departure in most of the studies, including the one presented in this paper.
Abstract: Purpose – Numerous studies have set out to examine the relationship between strategic resources and firm performance. The traditional VRIO attributes have been the point of departure in most resour ...

Journal ArticleDOI
01 Jul 2011
TL;DR: This paper revisits several concepts and model frameworks that have been in the literature of human-machine interaction and control engineering for up to 50 years to sharpen distinctions between adaptive automation, level of automation, allocation authority, supervisory control, and adaptive control engineering.
Abstract: This paper revisits several concepts and model frameworks that have been in the literature of human-machine interaction and control engineering for up to 50 years. The purposes of the revisit are as follows: 1) to sharpen distinctions between adaptive automation, level of automation, allocation authority, supervisory control, and adaptive control engineering as the terms are currently used in the literature; 2) to define modes of human supervisory adaptation from the control engineering perspective; and 3) to suggest comparative taxonomies for adaptive automation in direct control and in supervisory control.

Journal ArticleDOI
TL;DR: Computationally efficient suboptimal algorithms are proposed for the downlink resource allocation problem, and then, they are extended to the uplink case and it is shown through simulation that the proposed algorithms exhibit near-optimal performance and significantly reduce the computational complexity compared to obtaining the optimal solution.
Abstract: This paper considers the problem of radio resource allocation in an orthogonal frequency-division multiple access based cognitive radio (CR) network that opportunistically operates within the licensed primary users (PUs) spectrum. The resource allocation algorithm aims at maximizing the CR network throughput under PUs interference constraints. The CR interference introduced into PUs subbands is modeled as a composite that consists of the following two parts: 1) CR out-of-band emissions and 2) the interference that arises as a result of imperfect spectrum sensing. We consider both downlink and uplink subcarrier and power allocation. In both cases, the resource allocation problem is a mixed-integer nonlinear programming problem, for which obtaining the optimal solution is known to be NP-hard. Computationally efficient suboptimal algorithms are proposed for the downlink resource allocation problem, and then, they are extended to the uplink case. We show through simulation that the proposed algorithms exhibit near-optimal performance and significantly reduce the computational complexity compared to obtaining the optimal solution.

Book ChapterDOI
24 Feb 2011
TL;DR: Preface school-based management (SBM) has become a very popular movement over the past decade as discussed by the authors, and the work of the International Bank for Reconstruction and Development (IBRD/The World Bank) is an example of such a movement.
Abstract: This volume is a product of the staff of the International Bank for Reconstruction and Development / The World Bank. The fi ndings, interpretations, and conclusions expressed in this volume do not necessarily refl ect the views of the Executive Directors of The World Bank or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgement on the part of The World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. Rights and Permissions The material in this publication is copyrighted. Copying and/or transmitting portions or all of this work without permission may be a violation of applicable law. The International Bank for Reconstruction and Development / The World Bank encourages dissemination of its work and will normally grant permission to reproduce portions of the work promptly. For permission to photocopy or reprint any part of this work, please send a request with complete information to the vii Preface School-based management (SBM) has become a very popular movement over the past decade. Our SBM work program emerged out of a need to defi ne the concept more clearly, review the evidence, support impact assessments in various countries, and provide some initial feedback to teams preparing education projects. During fi rst phase of the SBM work program, the team undertook a detailed stocktaking of the existing literature on SBM. At the same time we identifi ed several examples of SBM reforms that we are now supporting through ongoing impact assessments. An online toolkit providing some general principles that can broadly be applied to the implementation of SBM reforms has been developed and can be accessed on Mackintosh provided excellent editing of the content and Victoriano Arias formatted the document. The team received very useful feedback from Ruth Kagia and Robin Horn. The peer reviewers for this task were Luis Benveniste and Shantayanan Devarajan. Excellent comments were received for an informal, virtual review from Erik Bloom.

Journal ArticleDOI
TL;DR: In this paper, the authors identified four types of uncertainty that characterize problems in natural resource management and examined ways in which climate change is expected to exacerbate these uncertainties, as well as potential approaches to dealing with them.
Abstract: Climate change and its associated uncertainties are of concern to natural resource managers. Although aspects of climate change may be novel (e.g., system change and nonstationarity), natural resource managers have long dealt with uncertainties and have developed corresponding approaches to decision-making. Adaptive resource management is an application of structured decision-making for recurrent decision problems with uncertainty, focusing on management objectives, and the reduction of uncertainty over time. We identified 4 types of uncertainty that characterize problems in natural resource management. We examined ways in which climate change is expected to exacerbate these uncertainties, as well as potential approaches to dealing with them. As a case study, we examined North American waterfowl harvest management and considered problems anticipated to result from climate change and potential solutions. Despite challenges expected to accompany the use of adaptive resource management to address pr...

Proceedings ArticleDOI
14 Jun 2011
TL;DR: Light is shed on some of the key issues in building cloud-scale resource management systems, based on five years of research and shipping cluster resource management products and various techniques to provide large scale resource management.
Abstract: Managing resources at large scale while providing performance isolation and efficient use of underlying hardware is a key challenge for any cloud management software. Most virtual machine (VM) resource management systems like VMware DRS clusters, Microsoft PRO and Eucalyptus, do not currently scale to the number of hosts and VMs supported by cloud service providers. In addition to scale, other challenges include heterogeneity of systems, compatibility constraints between virtual machines and underlying hardware, islands of resources created due to storage and network connectivity and limited scale of storage resources.In this paper, we shed light on some of the key issues in building cloud-scale resource management systems, based on five years of research and shipping cluster resource management products. Furthermore, we discuss various techniques to provide large scale resource management, along with the pros and cons of each technique. We hope to motivate future research in this area to develop practical solutions to these issues.

Proceedings ArticleDOI
12 Nov 2011
TL;DR: TRACON is presented, a novel Task and Resource Allocation CONtrol framework that mitigates the interference effects from concurrent data-intensive applications and greatly improves the overall system performance.
Abstract: Large-scale data centers leverage virtualization technology to achieve excellent resource utilization, scalability, and high availability. Ideally, the performance of an application running inside a virtual machine (VM) shall be independent of co-located applications and VMs that share the physical machine. However, adverse interference effects exist and are especially severe for data-intensive applications in such vir-tualized environments. In this work, we present TRACON, a novel Task and Resource Allocation CONtrol framework that mitigates the interference effects from concurrent data-intensive applications and greatly improves the overall system performance. TRACON utilizes modeling and control techniques from statistical machine learning and consists of three major components: the interference prediction model that infers application performance from resource consumption observed from different VMs, the interference-aware scheduler that is designed to utilize the model for effective resource management, and the task and resource monitor that collects application characteristics at the runtime for model adaption. We simulate TRACON with a wide variety of data-intensive applications including bioinformatics, data mining, video processing, email and web servers, etc. The evaluation results show that TRACON can achieve up to 50% improvement on application runtime, and up to 80% on I/O throughput for data-intensive applications in virtu-alized data centers.