scispace - formally typeset
Search or ask a question

Showing papers on "Load balancing (computing) published in 2016"


Journal ArticleDOI
TL;DR: In this article, the authors investigate and discuss serious limitations of the fourth generation (4G) cellular networks and corresponding new features of 5G networks, and present a comparative study of the proposed architectures that can be categorized on the basis of energy-efficiency, network hierarchy, and network types.

363 citations


Proceedings ArticleDOI
14 Mar 2016
TL;DR: HULA is presented, a data-plane load-balancing algorithm that outperforms a scalable extension to CONGA in average flow completion time and is designed for emerging programmable switches and programed in P4 to demonstrate that HULA could be run on such programmable chipsets, without requiring custom hardware.
Abstract: Datacenter networks employ multi-rooted topologies (e.g., Leaf-Spine, Fat-Tree) to provide large bisection bandwidth. These topologies use a large degree of multipathing, and need a data-plane load-balancing mechanism to effectively utilize their bisection bandwidth. The canonical load-balancing mechanism is equal-cost multi-path routing (ECMP), which spreads traffic uniformly across multiple paths. Motivated by ECMP's shortcomings, congestion-aware load-balancing techniques such as CONGA have been developed. These techniques have two limitations. First, because switch memory is limited, they can only maintain a small amount of congestion-tracking state at the edge switches, and do not scale to large topologies. Second, because they are implemented in custom hardware, they cannot be modified in the field. This paper presents HULA, a data-plane load-balancing algorithm that overcomes both limitations. First, instead of having the leaf switches track congestion on all paths to a destination, each HULA switch tracks congestion for the best path to a destination through a neighboring switch. Second, we design HULA for emerging programmable switches and program it in P4 to demonstrate that HULA could be run on such programmable chipsets, without requiring custom hardware. We evaluate HULA extensively in simulation, showing that it outperforms a scalable extension to CONGA in average flow completion time (1.6 x at 50% load, 3 x at 90% load).

322 citations


Journal ArticleDOI
TL;DR: A closer look at the current SFC architecture and a survey of the recent developments in SFC including its relevance with NFV to help determine the future research directions and the standardization efforts of SFC are provided.

251 citations


Journal ArticleDOI
TL;DR: A systematic literature review of the existing load balancing techniques proposed so far and the advantages and disadvantages associated with several load balancing algorithms have been discussed and the important challenges of these algorithms are addressed so that more efficientload balancing techniques can be developed in future.

202 citations


Proceedings ArticleDOI
22 Aug 2016
TL;DR: The OpenNetVM architecture is described, an efficient packet processing framework that greatly simplifies the development of network functions, as well as research into their management and optimization, and its performance is evaluated compared to existing NFV platforms.
Abstract: Network middleboxes are growing in number and diversity. Middleboxes have been deployed widely to complement the basic end-to-end functionality provided by the Internet Protocol suite that depends only on the minimal functionality of a best-effort network layer. The move from purpose-built hardware middleboxes to software appliances running in virtual machines provides much needed deployment flexibility, but significant challenges remain. Just as Software Defined Networking (SDN) research and product development was greatly accelerated with the release of several open source SDN platforms, we believe that Network Function Virtualization (NFV) research can see similar growth with the development of a flexible platform that enables high performance NFV prototypes. Towards this end we have been building OpenNetVM, an efficient packet processing framework that greatly simplifies the development of network functions, as well as research into their management and optimization. OpenNetVM runs network functions in lightweight Docker containers, enabling fast startup and reducing memory overheads. The OpenNetVM platform manager provides load balancing, flexible flow management, and service name abstractions. OpenNetVM efficiently routes packets through dynamically created service chains, achieving throughputs of 10 Gbps even when traversing a chain of 6 NFs. In this paper we describe our architecture and evaluate its performance compared to existing NFV platforms.

200 citations


Journal ArticleDOI
TL;DR: In this paper, a succinct overview is presented regarding the state of the art on the research on C-RAN with emphasis on fronthaul compression, baseband processing, medium access control, resource allocation, system-level considerations and standardization efforts.
Abstract: Cloud radio access network (C-RAN) refers to the visualization of base station functionalities by means of cloud computing. This results in a novel cellular architecture in which low-cost wireless access points, known as radio units or remote radio heads, are centrally managed by a reconfigurable centralized "cloud", or central, unit. C-RAN allows operators to reduce the capital and operating expenses needed to deploy and maintain dense heterogeneous networks. This critical advantage, along with spectral efficiency, statistical multiplexing and load balancing gains, make C-RAN well positioned to be one of the key technologies in the development of 5G systems. In this paper, a succinct overview is presented regarding the state of the art on the research on C-RAN with emphasis on fronthaul compression, baseband processing, medium access control, resource allocation, system-level considerations and standardization efforts.

193 citations


Journal ArticleDOI
TL;DR: An adaptive method aiming at spatial-temporal efficiency in a heterogeneous cloud environment based on an optimized Kernel-based Extreme Learning Machine algorithm is presented for faster forecast of job execution duration and space occupation and achieves 26.6% improvement over the original scheme.
Abstract: A heterogeneous cloud system, for example, a Hadoop 2.6.0 platform, provides distributed but cohesive services with rich features on large-scale management, reliability, and error tolerance. As big data processing is concerned, newly built cloud clusters meet the challenges of performance optimization focusing on faster task execution and more efficient usage of computing resources. Presently proposed approaches concentrate on temporal improvement, that is, shortening MapReduce time, but seldom focus on storage occupation; however, unbalanced cloud storage strategies could exhaust those nodes with heavy MapReduce cycles and further challenge the security and stability of the entire cluster. In this paper, an adaptive method is presented aiming at spatial-temporal efficiency in a heterogeneous cloud environment. A prediction model based on an optimized Kernel-based Extreme Learning Machine algorithm is proposed for faster forecast of job execution duration and space occupation, which consequently facilitates the process of task scheduling through a multi-objective algorithm called time and space optimized NSGA-II TS-NSGA-II. Experiment results have shown that compared with the original load-balancing scheme, our approach can save approximate 47-55i¾źs averagely on each task execution. Simultaneously, 1.254i¾ź of differences on hard disk occupation were made among all scheduled reducers, which achieves 26.6% improvement over the original scheme. Copyright © 2016 John Wiley & Sons, Ltd.

193 citations


Journal ArticleDOI
TL;DR: A new clustering algorithm, Distributed Unequal Clustering using Fuzzy logic (DUCF) which elects CHs using fuzzy approach is contributed which improves the network lifetime compared with its counterparts.

178 citations


Journal ArticleDOI
TL;DR: This paper proposes an energy-efficient multi-constraint rerouting algorithm, E2MR2, which uses the energy consumption model to set up the link weight for maximum energy efficiency and exploits rerouted strategy to ensure network QoS and maximum delay constraints.
Abstract: Many researches show that the power consumption of network devices of ICT is nearly 10% of total global consumption. While the redundant deployment of network equipment makes the network utilization is relatively low, which leads to a very low energy efficiency of networks. With the dynamic and high quality demands of users, how to improve network energy efficiency becomes a focus under the premise of ensuring network performance and customer service quality. For this reason, we propose an energy consumption model based on link loads, and use the network’s bit energy consumption parameter to measure the network energy efficiency. This paper is to minimize the network’s bit energy consumption parameter, and then we propose the energy-efficient minimum criticality routing algorithm, which includes energy efficiency routing and load balancing. To further improve network energy efficiency, this paper proposes an energy-efficient multi-constraint rerouting (E2MR2) algorithm. E2MR2 uses the energy consumption model to set up the link weight for maximum energy efficiency and exploits rerouting strategy to ensure network QoS and maximum delay constraints. The simulation uses synthetic traffic data in the real network topology to analyze the performance of our method. Simulation results that our approach is feasible and promising.

166 citations


Proceedings ArticleDOI
10 Apr 2016
TL;DR: This paper considers dynamic controller assignment so as to minimize the average response time of the control plane, and proposes a hierarchically two-phase algorithm that integrates key concepts from both matching theory and coalitional games to solve it efficiently.
Abstract: Software defined networking is becoming increasingly prevalent in data center networks for its programmability that enables centralized network configuration and management. However, since switches are statically assigned to controllers, traffic dynamics cause load imbalance among the controllers. As a result, some controllers are not fully utilized, while switches connected to overloaded controllers may experience long response times. In this paper, we consider dynamic controller assignment so as to minimize the average response time of the control plane. We formulate this problem as a stable matching problem with transfers, and propose a hierarchically two-phase algorithm that integrates key concepts from both matching theory and coalitional games to solve it efficiently. Theoretical analysis proves that our algorithm converges to a near-optimal Nash stable solution within tens of iterations. Extensive simulations show that our approach reduces response time by about 86%, and achieves better load balancing among controllers compared to static assignment.

166 citations


Journal ArticleDOI
TL;DR: This survey mainly categorizes and reviews the representative studies on task allocation and load balancing according to the general characteristics of varying distributed systems and makes a comprehensive taxonomy on them.
Abstract: In past decades, significant attention has been devoted to the task allocation and load balancing in distributed systems. Although there have been some related surveys about this subject, each of which only made a very preliminary review on the state of art of one single type of distributed systems. To correlate the studies in varying types of distributed systems and make a comprehensive taxonomy on them, this survey mainly categorizes and reviews the representative studies on task allocation and load balancing according to the general characteristics of varying distributed systems. First, this survey summarizes the general characteristics of distributed systems. Based on these general characteristics, this survey reviews the studies on task allocation and load balancing with respect to the following aspects: 1) typical control models; 2) typical resource optimization methods; 3) typical methods for achieving reliability; 4) typical coordination mechanisms among heterogeneous nodes; and 5) typical models considering network structures. For each aspect, we summarize the existing studies and discuss the future research directions. Through the survey, the related studies in this area can be well understood based on how they can satisfy the general characteristics of distributed systems.

Proceedings ArticleDOI
10 Apr 2016
TL;DR: This paper introduces a system model to capture the response times of offloaded tasks, and proposes a fast, scalable algorithm for the problem to find an optimal redirection of tasks between cloudlets such that the maximum of the average response times at cloudlets is minimized.
Abstract: With advances in wireless communication technology, more and more people depend heavily on portable mobile devices for businesses, entertainments and social interactions. Although such portable mobile devices can offer various promising applications, their computing resources remain limited due to their portable size. This however can be overcome by remotely executing computation-intensive tasks on clusters of near by computers known as cloudlets. As increasing numbers of people access the Internet via mobile devices, it is reasonable to envision in the near future that cloudlet services will be available for the public through easily accessible public wireless metropolitan area networks (WMANs). However, the outdated notion of treating cloudlets as isolated data-centers-in-a-box must be discarded as there are clear benefits to connecting multiple cloudlets together to form a network. In this paper we investigate how to balance the workload between multiple cloudlets in a network to optimize mobile application performance. We first introduce a system model to capture the response times of offloaded tasks, and formulate a novel optimization problem, that is to find an optimal redirection of tasks between cloudlets such that the maximum of the average response times of tasks at cloudlets is minimized. We then propose a fast, scalable algorithm for the problem. We finally evaluate the performance of the proposed algorithm through experimental simulations. The experimental results demonstrate the significant potential of the proposed algorithm in reducing the response times of tasks.

Journal ArticleDOI
TL;DR: The simulation results show that the framework of the fog computing after Cloud Atomization can build the system network flexibly, and dynamic load balancing mechanism can effectively configure system resources as well as reducing the consumption of node migration brought by system changes.
Abstract: Because of cloud computing's high degree of polymerization calculation mode, it can't give full play to the resources of the edge device such as computing, storage, etc. Fog computing can improve the resource utilization efficiency of the edge device, and solve the problem about service computing of the delay-sensitive applications. This paper researches on the framework of the fog computing, and adopts Cloud Atomization Technology to turn physical nodes in different levels into virtual machine nodes. On this basis, this paper uses the graph partitioning theory to build the fog computing's load balancing algorithm based on dynamic graph partitioning. The simulation results show that the framework of the fog computing after Cloud Atomization can build the system network flexibly, and dynamic load balancing mechanism can effectively configure system resources as well as reducing the consumption of node migration brought by system changes.

Journal ArticleDOI
Weina Wang1, Kai Zhu1, Lei Ying1, Jian Tan2, Li Zhang2 
TL;DR: A new queueing architecture is presented and a map task scheduling algorithm constituted by the Join the Shortest Queue policy together with the MaxWeight policy is proposed that is heavy-traffic optimal, i.e., it asymptotically minimizes the number of backlogged tasks as the arrival rate vector approaches the boundary of the capacity region.
Abstract: MapReduce/Hadoop framework has been widely used to process large-scale datasets on computing clusters. Scheduling map tasks with data locality consideration is crucial to the performance of MapReduce. Many works have been devoted to increasing data locality for better efficiency. However, to the best of our knowledge, fundamental limits of MapReduce computing clusters with data locality, including the capacity region and theoretical bounds on the delay performance, have not been well studied. In this paper, we address these problems from a stochastic network perspective. Our focus is to strike the right balance between data locality and load balancing to simultaneously maximize throughput and minimize delay. We present a new queueing architecture and propose a map task scheduling algorithm constituted by the Join the Shortest Queue policy together with the MaxWeight policy. We identify an outer bound on the capacity region, and then prove that the proposed algorithm can stabilize any arrival rate vector strictly within this outer bound. It shows that the outer bound coincides with the actual capacity region, and the proposed algorithm is throughput-optimal. Furthermore, we study the number of backlogged tasks under the proposed algorithm, which is directly related to the delay performance based on Little's law. We prove that the proposed algorithm is heavy-traffic optimal, i.e., it asymptotically minimizes the number of back-logged tasks as the arrival rate vector approaches the boundary of the capacity region. Therefore, the proposed algorithm is also delay-optimal in the heavy-traffic regime. The proofs in this paper deal with random processing times with heterogeneous parameters and nonpreemptive task execution, which differentiate our work from many existing works on MaxWeight-type algorithms, so the proof techniques themselves for the stability analysis and the heavy-traffic analysis are also novel contributions.

Journal ArticleDOI
Jia Zhao1, Kun Yang2, Xiaohui Wei1, Yan Ding1, Liang Hu1, Gaochao Xu1 
TL;DR: Simulation results show that compared with the existing works, the proposed approach has reduced the failure number of task deployment events obviously, improved the throughput, and optimized the external services performance of cloud data centers.
Abstract: Aiming at the current problems that most physical hosts in the cloud data center are so overloaded that it makes the whole cloud data center’ load imbalanced and that existing load balancing approaches have relatively high complexity, this paper has focused on the selection problem of physical hosts for deploying requested tasks and proposed a novel heuristic approach called Load Balancing based on Bayes and Clustering (LB-BC). Most previous works, generally, utilize a series of algorithms through optimizing the candidate target hosts within an algorithm cycle and then picking out the optimal target hosts to achieve the immediate load balancing effect. However, the immediate effect doesn’t guarantee high execution efficiency for the next task although it has abilities in achieving high resource utilization. Based on this argument, LB-BC introduces the concept of achieving the overall load balancing in a long-term process in contrast to the immediate load balancing approaches in the current literature. LB-BC makes a limited constraint about all physical hosts aiming to achieve a task deployment approach with global search capability in terms of the performance function of computing resource. The Bayes theorem is combined with the clustering process to obtain the optimal clustering set of physical hosts finally. Simulation results show that compared with the existing works, the proposed approach has reduced the failure number of task deployment events obviously, improved the throughput, and optimized the external services performance of cloud data centers.

Journal ArticleDOI
TL;DR: Object of this work is to introduce and evaluate the proposed scheduling and load balancing algorithm by considering the capabilities of each virtual machine (VM), the task length of each requested job, and the interdependency of multiple tasks.
Abstract: Cloud computing uses the concepts of scheduling and load balancing to migrate tasks to underutilized VMs for effectively sharing the resources. The scheduling of the nonpreemptive tasks in the cloud computing environment is an irrecoverable restraint and hence it has to be assigned to the most appropriate VMs at the initial placement itself. Practically, the arrived jobs consist of multiple interdependent tasks and they may execute the independent tasks in multiple VMs or in the same VM's multiple cores. Also, the jobs arrive during the run time of the server in varying random intervals under various load conditions. The participating heterogeneous resources are managed by allocating the tasks to appropriate resources by static or dynamic scheduling to make the cloud computing more efficient and thus it improves the user satisfaction. Objective of this work is to introduce and evaluate the proposed scheduling and load balancing algorithm by considering the capabilities of each virtual machine (VM), the task length of each requested job, and the interdependency of multiple tasks. Performance of the proposed algorithm is studied by comparing with the existing methods.

Journal ArticleDOI
TL;DR: Proposals employing parallel platforms, including field-programmable gate array, GPU, general multi-processors, and ternary content addressable memory, to accelerate the matching process are introduced and thoroughly discussed and guidelines for efficient deployment are provided.
Abstract: Deep packet inspection (DPI) is widely used in content-aware network applications such as network intrusion detection systems, traffic billing, load balancing, and government surveillance. Pattern matching is a core and critical step in DPI, which checks the payload of each packet for known signatures (patterns) in order to identify packets with certain characteristics (e.g., malicious packets that carry viruses or worms). Regular expression is the major tool for signature description due to its powerful and flexible expressive ability. However, this flexibility also brings great challenges for efficient implementation in practice. Despite of hundreds to thousands of empirical proposals, wire-speed matching for large scale regular expressions still remains a big challenge. The gap between the matching throughput and the link speed is widening with the ever-increasing network link speed and pattern scale. This survey begins with a full-scale application background of DPI and technical background of regular expression matching in order to provide a global view and essential knowledge for readers. We then analyze the challenges in regular expression matching originated from the state explosion of finite state automaton used for regular expression matching. The nature of state explosion is analyzed in details, and the state-of-the-art solutions are grouped into categories of methods to relieve state expansion and methods to avoid state explosion, suggestions are also provided for building compact and efficient automata in different scenarios. Furthermore, proposals employing parallel platforms, including field-programmable gate array, GPU, general multi-processors, and ternary content addressable memory, to accelerate the matching process are introduced and thoroughly discussed. We also provide guidelines for efficient deployment for each of these platforms.

Journal ArticleDOI
TL;DR: A survey of congestion control solutions for multipath transport protocols and discusses the multipath congestion control design in order to address the need for some desirable properties including TCP-friendliness, load balancing, stability, and Pareto optimality.
Abstract: High-quality services over wired and wireless networks require high bitrate network content delivery. Multipath transport protocols utilize multiple paths to support increased throughput for network applications. Ubiquitous mobile devices with multiple interfaces such as WiFi and 4G/5G cellular, and data centers supporting big data analysis and cloud-computing, motivate adoption of multipath transmission in current and future network architectures. Congestion control mechanisms play a fundamental role in these multipath transport protocols. Diverse approaches have been proposed in the literature, differing in terms of their goals, principles, performance, and mostly in how various issues are addressed in their design. This paper presents a survey of congestion control solutions for multipath transport protocols and discusses the multipath congestion control design in order to address the need for some desirable properties including TCP-friendliness, load balancing, stability, and Pareto optimality. Existing window-based and rate-based multipath congestion control algorithms are investigated and categorized based on their theoretical algorithm design. Next, this paper discusses the congestion control mechanisms used in diverse multipath transport protocols in the context of different network scenarios. Finally, the trends in the future multipath congestion control design are presented.

Proceedings ArticleDOI
02 Nov 2016
TL;DR: EC-Cache is a load-balanced, low latency cluster cache that uses online erasure coding to overcome the limitations of selective replication, and improves load balancing and reduces the median and tail read latencies by more than 2×, while using the same amount of memory.
Abstract: Data-intensive clusters and object stores are increasingly relying on in-memory object caching to meet the I/O performance demands. These systems routinely face the challenges of popularity skew, background load imbalance, and server failures, which result in severe load imbalance across servers and degraded I/O performance. Selective replication is a commonly used technique to tackle these challenges, where the number of cached replicas of an object is proportional to its popularity. In this paper, we explore an alternative approach using erasure coding.EC-Cache is a load-balanced, low latency cluster cache that uses online erasure coding to overcome the limitations of selective replication. EC-Cache employs erasure coding by: (i) splitting and erasure coding individual objects during writes, and (ii) late binding, wherein obtaining any k out of (k + r) splits of an object are sufficient, during reads. As compared to selective replication, EC-Cache improves load balancing by more than 3× and reduces the median and tail read latencies by more than 2×, while using the same amount of memory. EC-Cache does so using 10% additional bandwidth and a small increase in the amount of stored metadata. The benefits offered by EC-Cache are further amplified in the presence of background network load imbalance and server failures.

Proceedings ArticleDOI
14 Mar 2016
TL;DR: This work designs a system for incremental deployment of hybrid SDN networks consisting of both legacy forwarding devices and programmable SDN switches, and designs the system on a production SDN controller to answer the following questions: which legacy devices to upgrade to SDN, and how legacy and SDN devices can interoperate in a hybrid environment.
Abstract: Introducing SDN into an existing network causes both deployment and operational issues. A systematic incremental deployment methodology as well as a hybrid operation model is needed. We present such a system for incremental deployment of hybrid SDN networks consisting of both legacy forwarding devices (i.e., traditional IP routers) and programmable SDN switches. We design the system on a production SDN controller to answer the following questions: which legacy devices to upgrade to SDN, and how legacy and SDN devices can interoperate in a hybrid environment to satisfy a variety of traffic engineering (TE) goals such as load balancing and fast failure recovery. Evaluation on real ISP and enterprise topologies shows that with only 20% devices upgraded to SDN, our system reduces the maximum link usage by an average of 32% compared with pure-legacy networks (shortest path routing), while only requiring an average of 41% of flow table capacity compared with pure-SDN networks.

Journal ArticleDOI
TL;DR: The communication cost minimization problem for BDSP is formulated into a mixed-integer linear programming (MILP) problem and proved to be NP-hard, and a computation-efficient solution based on MILP is proposed.
Abstract: With the explosion of big data, processing large numbers of continuous data streams, i.e., big data stream processing (BDSP), has become a crucial requirement for many scientific and industrial applications in recent years. By offering a pool of computation, communication and storage resources, public clouds, like Amazon's EC2, are undoubtedly the most efficient platforms to meet the ever-growing needs of BDSP. Public cloud service providers usually operate a number of geo-distributed datacenters across the globe. Different datacenter pairs are with different inter-datacenter network costs charged by Internet Service Providers (ISPs). While, inter-datacenter traffic in BDSP constitutes a large portion of a cloud provider's traffic demand over the Internet and incurs substantial communication cost, which may even become the dominant operational expenditure factor. As the datacenter resources are provided in a virtualized way, the virtual machines (VMs) for stream processing tasks can be freely deployed onto any datacenters, provided that the Service Level Agreement (SLA, e.g., quality-of-information) is obeyed. This raises the opportunity, but also a challenge, to explore the inter-datacenter network cost diversities to optimize both VM placement and load balancing towards network cost minimization with guaranteed SLA. In this paper, we first propose a general modeling framework that describes all representative inter-task relationship semantics in BDSP. Based on our novel framework, we then formulate the communication cost minimization problem for BDSP into a mixed-integer linear programming (MILP) problem and prove it to be NP-hard. We then propose a computation-efficient solution based on MILP. The high efficiency of our proposal is validated by extensive simulation based studies.

Journal ArticleDOI
TL;DR: This work proposes a real-time intrusion detection system for ultra-high-speed big data environment using Hadoop implementation that has overall higher accuracy than existing IDSs with the capability to work in real time in ultra- high-speedbig data environment.
Abstract: In recent years, the number of people using the Internet and network services is increasing day by day. On a daily basis, a large amount of data is generated over the Internet from zeta byte to petabytes with a very high speed. On the other hand, we see more security threats on the network, the Internet, websites, and the enterprise network. Therefore, detecting intrusion in such ultra-high-speed environment in real time is a challenging task. Many intrusion detection systems (IDSs) are proposed for various types of network attacks using machine learning approaches. Most of them are unable to detect recent unknown attacks, whereas the others do not provide a real-time solution to overcome the above-mentioned challenges. Therefore, to address these problems, we propose a real-time intrusion detection system for ultra-high-speed big data environment using Hadoop implementation. The proposed system includes four-layered IDS architecture, which consists of the capturing layer, filtration and load balancing layer, processing or Hadoop layer, and the decision-making layer. Furthermore, feature selection scheme is proposed that selects nine parameters for classification using (FSR) and (BER), as well as from the analysis of DARPA datasets. In addition, five major machine learning approaches are used to evaluate the proposed system including J48, REPTree, random forest tree, conjunctive rule, support vector machine, and Naive Bayes classifiers. Results show that among all these classifiers, REPTree and J48 are the best classifiers in terms of accuracy as well as efficiency. The proposed system architecture is evaluated with respect to accuracy in terms of true positive (TP) and false positive (FP), with respect to efficiency in terms of processing time and by comparing results with traditional techniques. It has more than 99 % TP and less than 0.001 % FP on REPTree and J48. The system has overall higher accuracy than existing IDSs with the capability to work in real time in ultra-high-speed big data environment.

Journal ArticleDOI
01 Mar 2016
TL;DR: FiDoop, a parallel frequent itemsets mining algorithm called FiDoop using the MapReduce programming model, which incorporates the frequent items ultrametric tree, rather than conventional FP trees, to achieve compressed storage and avoid building conditional pattern bases.
Abstract: Existing parallel mining algorithms for frequent itemsets lack a mechanism that enables automatic parallelization, load balancing, data distribution, and fault tolerance on large clusters. As a solution to this problem, we design a parallel frequent itemsets mining algorithm called FiDoop using the MapReduce programming model. To achieve compressed storage and avoid building conditional pattern bases, FiDoop incorporates the frequent items ultrametric tree, rather than conventional FP trees. In FiDoop, three MapReduce jobs are implemented to complete the mining task. In the crucial third MapReduce job, the mappers independently decompose itemsets, the reducers perform combination operations by constructing small ultrametric trees, and the actual mining of these trees separately. We implement FiDoop on our in-house Hadoop cluster. We show that FiDoop on the cluster is sensitive to data distribution and dimensions, because itemsets with different lengths have different decomposition and construction costs. To improve FiDoop’s performance, we develop a workload balance metric to measure load balance across the cluster’s computing nodes. We develop FiDoop-HD, an extension of FiDoop, to speed up the mining performance for high-dimensional data analysis. Extensive experiments using real-world celestial spectral data demonstrate that our proposed solution is efficient and scalable.

Journal ArticleDOI
TL;DR: A distinguishing feature of psc is described: patch-based load balancing using space-filling curves which is shown to lead to major efficiency gains over unbalanced methods and a previously used simpler balancing method.

Journal ArticleDOI
TL;DR: The proposed architecture introduced a new scheduling policy for load balancing in Fog Computing environment, which complete real tasks within deadline, increase throughput and network utilization, maintaining data consistency with less complexity to meet the present day demand of end users.
Abstract: Cloud computing is the new era technology, which is entirely dependent on the internet to maintain large applications, where data is shared over one platform to provide better services to clients belonging to a different organization. It ensures maximum utilization of computational resources by making availability of data, software and infrastructure with lower cost in a secure, reliable and flexible manner. Though cloud computing offers many advantages, but it suffers from certain limitation too, that during load balancing of data in cloud data centers the internet faces problems of network congestion, less bandwidth utilization, fault tolerance and security etc. To get rid out of this issue new computing model called Fog Computing is introduced which easily transfer sensitive data without delaying to distributed devices. Fog is similar to the cloud only difference lies in the fact that it is located more close to end users to process and give response to the client in less time. Secondly, it is beneficial to the real time streaming applications, sensor networks, Internet of things which need high speed and reliable internet connectivity. Our proposed architecture introduced a new scheduling policy for load balancing in Fog Computing environment, which complete real tasks within deadline, increase throughput and network utilization, maintaining data consistency with less complexity to meet the present day demand of end users.

Journal Article
TL;DR: The main aim of employing this project is to get a good scalability and a long lifetime for the required network and for the lower data gathering recess.
Abstract: In this project, a framework which consists of three layers is introduced for the mobile data gathering in the wireless sensor networks. The three layers of the framework are given as the sensor layer, clustered head layer, mobile collector layer (also called as SenCar). This framework is made use of with a LBC-DDU, which is called as the load balanced clustering and the dual data uploading. The main aim of employing this project is to get a good scalability and a long lifetime for the required network and for the lower data gathering recess.

Proceedings Article
16 Mar 2016
TL;DR: SwitchKV is a new key-value store system design that combines high-performance cache nodes with resource-constrained backend nodes to provide load balancing in the face of unpredictable workload skew.
Abstract: SwitchKV is a new key-value store system design that combines high-performance cache nodes with resource-constrained backend nodes to provide load balancing in the face of unpredictable workload skew. The cache nodes absorb the hottest queries so that no individual backend node is over-burdened. Compared with previous designs, SwitchKV exploits SDN techniques and deeply optimized switch hardware to enable efficient content-based routing. Programmable network switches keep track of cached keys and route requests to the appropriate nodes at line speed, based on keys encoded in packet headers. A new hybrid caching strategy keeps cache and switch forwarding rules updated with low overhead and ensures that system load is always well-balanced under rapidly changing workloads. Our evaluation results demonstrate that SwitchKV can achieve up to 5× throughput and 3× latency improvements over traditional system designs.

Journal ArticleDOI
01 Jan 2016
TL;DR: AutoElastic is presented, a PaaS-level elasticity model for HPC in the cloud that consists of providing elasticity for high performance applications without user intervention or source code modification and low intrusiveness when reconfigurations do not occur.
Abstract: Elasticity is undoubtedly one of the most striking characteristics of cloud computing. Especially in the area of high performance computing (HPC), elasticity can be used to execute irregular and CPU-intensive applications. However, the on- the-fly increase/decrease in resources is more widespread in Web systems, which have their own IaaS-level load balancer. Considering the HPC area, current approaches usually focus on batch jobs or assumptions such as previous knowledge of application phases, source code rewriting or the stop-reconfigure-and-go approach for elasticity. In this context, this article presents AutoElastic, a PaaS-level elasticity model for HPC in the cloud. Its differential approach consists of providing elasticity for high performance applications without user intervention or source code modification. The scientific contributions of AutoElastic are twofold: (i) an Aging-based approach to resource allocation and deallocation actions to avoid unnecessary virtual machine (VM) reconfigurations (thrashing) and (ii) asynchronism in creating and terminating VMs in such a way that the application does not need to wait for completing these procedures. The prototype evaluation using OpenNebula middleware showed performance gains of up to 26 percent in the execution time of an application with the AutoElastic manager. Moreover, we obtained low intrusiveness for AutoElastic when reconfigurations do not occur.

Proceedings ArticleDOI
22 May 2016
TL;DR: This work proposes a solution that considers at the same time three critical objectives for the optimal placement of controllers, and solves the system by using Bargaining Game in order to find a fair trade off between these objectives.
Abstract: In this paper we address the problem of Software Defined Networking (SDN) controller placement in large networks. Indeed, to solve the scalability issue raised by the centralized architecture of SDN, multi-controllers deployment (or distributed controllers system) is envisioned. However, the number and the location of controllers in large networks remain an issue. In this context, several works have been proposed to find the optimal placement of SDN controllers. Most of them consider latency among SDN controllers and switches as the main metric. In this work, we go beyond the state of art by proposing a solution that considers at the same time three critical objectives for the optimal placement of controllers: (i) the latency and communication overhead between switches and controllers; (ii)the latency and communication overhead between controllers; (iii) the guarantee of load balancing between controllers. We then solve the system by using Bargaining Game in order to find a fair trade off between these objectives. Simulation results clearly demonstrate the effectiveness of the proposed solution in finding the optimal placement of controllers that enforces this trade-off.

Journal ArticleDOI
TL;DR: This paper proposes a generalized Q-learning framework for the CCN functions and shows how the framework fits to a general SF control loop, and applies this framework to two functions on mobility robustness optimization (MRO) and mobility load balancing (MLB).
Abstract: Self-organizing networks (SON) aim at simplifying network management (NM) and optimizing network capital and operational expenditure through automation. Most SON functions (SFs) are rule-based control structures, which evaluate metrics and decide actions based on a set of rules. These rigid structures are, however, very complex to design since rules must be derived for each SF in each possible scenario. In practice, rules only support generic behavior, which cannot respond to the specific scenarios in each network or cell. Moreover, SON coordination becomes very complicated with such varied control structures. In this paper, we propose to advance SON toward cognitive cellular networks (CCN) by adding cognition that enables the SFs to independently learn the required optimal configurations. We propose a generalized Q-learning framework for the CCN functions and show how the framework fits to a general SF control loop. We then apply this framework to two functions on mobility robustness optimization (MRO) and mobility load balancing (MLB). Our results show that the MRO function learns to optimize handover performance while the MLB function learns to distribute instantaneous load among cells.