scispace - formally typeset
Book ChapterDOI

An Adaptive MapReduce Scheduler for Scalable Heterogeneous Systems

Mohammad Ghoneem, +1 more
- pp 603-611
Reads0
Chats0
TLDR
A lightweight adaptive scheduler is introduced in which the classifier is provided with information about jobs requirement and node capabilities and the scheduler assigns the tasks to appropriate nodes in the cluster to get highest performance.
Abstract: 
Hadoop MapReduce has been proved to be an efficient model for distributed data processing. This model is widely used by different service providers, which create a challenge of maintaining same efficiency and performance level in different systems. One of the most critical problems for this model is how to overcome heterogeneity and scalability in different systems. The decreases of performance in heterogeneous environment occur due to inefficient scheduling of Map and Reduce tasks. Another important problem is how to minimize master node overhead and network traffic created by scheduling algorithm. In this paper, we introduce a lightweight adaptive scheduler in which we provide the classifier with information about jobs requirement and node capabilities. The scheduler classifies jobs into executable and nonexecutable according to the nodes capabilities. Then the scheduler assigns the tasks to appropriate nodes in the cluster to get highest performance.

read more

Citations
More filters
Journal ArticleDOI

Load-balancing algorithms in cloud computing

Einollah Jafarnejad Ghomi, +2 more
- 15 Jun 2017 - 
TL;DR: This paper study the literature on the task scheduling and load-balancing algorithms and present a new classification of such algorithms, for example, Hadoop MapReduce load balancing category, Natural Phenomena-based load balancing categories, Agent-basedLoadBalancing category, General load balancingcategory, application-oriented category, network-aware category, and workflow specific category.
Journal ArticleDOI

Issues and Challenges of Load Balancing Techniques in Cloud Computing: A Survey

Pawan Kumar, +1 more
- 04 Feb 2019 - 
TL;DR: A state-of-the-art review of issues and challenges associated with existing load-balancing techniques for researchers to develop more effective algorithms is presented.
Journal ArticleDOI

Autonomic task scheduling algorithm for dynamic workloads through a load balancing technique for the cloud-computing environment

Fatemeh Ebadifard, +1 more
- 01 Jun 2021 - 
TL;DR: The results for the actual workloads of the NASA and Calgary servers and sample workload indicate that upon an increase in the requests and their variations together with heterogeneity of different VMs, this proposed algorithm can distribute the workload among them equally and allocate requests to appropriate VMs based on the required resources; thus, a decrease in the communication overheads and a increase in load balancing degree.
Proceedings ArticleDOI

Improved fuzzy Load-Balancing Algorithm for Cloud Computing System

Mostefa Hamdani, +2 more
TL;DR: This paper proposes a Load Balancing algorithm based on the weights of servers in the cloud platform, and suggests the use of the fuzzy logic to represent the weight of the different nodes.
Book ChapterDOI

Big Data Issues in SDN Based IoT: A Review

Syeda Zeenat Marshoodulla, +2 more
TL;DR: In this article, the authors have tried to highlight the evolution of conventional IoT architecture to SDN based IoT architecture which offers better provisions to resolve Big Data issues also, and also highlight the procedures, developed by investigators of the area, as to how Big data issues can be resolved in these architectures.
References
More filters
Journal ArticleDOI

MapReduce: simplified data processing on large clusters

Jeffrey Dean, +1 more
TL;DR: This paper presents the implementation of MapReduce, a programming model and an associated implementation for processing and generating large data sets that runs on a large cluster of commodity machines and is highly scalable.
Journal ArticleDOI

MapReduce: simplified data processing on large clusters

Jeffrey Dean, +1 more
- 01 Jan 2008 - 
TL;DR: This presentation explains how the underlying runtime system automatically parallelizes the computation across large-scale clusters of machines, handles machine failures, and schedules inter-machine communication to make efficient use of the network and disks.
Proceedings ArticleDOI

Improving MapReduce performance in heterogeneous environments

Matei Zaharia, +4 more
TL;DR: A new scheduling algorithm, Longest Approximate Time to End (LATE), that is highly robust to heterogeneity and can improve Hadoop response times by a factor of 2 in clusters of 200 virtual machines on EC2.
Proceedings ArticleDOI

Scheduling Hadoop Jobs to Meet Deadlines

Kamal Kc, +1 more
TL;DR: This paper develops criteria for scheduling jobs based on user specified deadline constraints and discusses the implementation and preliminary evaluation of a Deadline Constraint Scheduler for Hadoop which ensures that only jobs whose deadlines can be met are scheduled for execution.
Proceedings ArticleDOI

Locality-Aware Reduce Task Scheduling for MapReduce

Mohammad Hammoud, +1 more
TL;DR: LARTS attempts to collocate reduce tasks with the maximum required data computed after recognizing input data network locations and sizes and adopts a cooperative paradigm seeking a good data locality while circumventing scheduling delay, scheduling skew, poor system utilization, and low degree of parallelism.