Proceedings ArticleDOI
Scaling Genetic Algorithms Using MapReduce
Abhishek Verma,Xavier Llorà,David E. Goldberg,Roy H. Campbell +3 more
- pp 13-18
Reads0
Chats0
TLDR
This paper describes the algorithm design and implementation of GAs on Hadoop, an open source implementation of MapReduce, and demonstrates the convergence and scalability up to 10^5 variable problems.Abstract:
Genetic algorithms(GAs) are increasingly being applied to large scale problems. The traditional MPI-based parallel GAs require detailed knowledge about machine architecture. On the other hand, MapReduce is a powerful abstraction proposed by Google for making scalable and fault tolerant applications. In this paper, we show how genetic algorithms can be modeled into the MapReduce model. We describe the algorithm design and implementation of GAs on Hadoop, an open source implementation of MapReduce. Our experiments demonstrate the convergence and scalability up to 10^5 variable problems. Adding more resources would enable us to solve even larger problems without any changes in the algorithms and implementation since we do not introduce any performance bottlenecks.read more
Citations
More filters
Journal ArticleDOI
Bio-inspired computation: Where we stand and what's next
Javier Del Ser,Javier Del Ser,Eneko Osaba,Daniel Molina,Xin-She Yang,Sancho Salcedo-Sanz,David Camacho,Swagatam Das,Ponnuthurai Nagaratnam Suganthan,Carlos A. Coello Coello,Francisco Herrera +10 more
TL;DR: The main purpose of this paper is to outline the state of the art and to identify open challenges concerning the most relevant areas within bio-inspired optimization, thereby highlighting the need for reaching a consensus and joining forces towards achieving valuable insights into the understanding of this family of optimization techniques.
Journal ArticleDOI
Artificial Intelligence and Big Data
TL;DR: AI Innovation in Industry is a new department for IEEE Intelligent Systems, and this paper examines some of the basic concerns and uses of AI for big data.
Journal ArticleDOI
Distributed evolutionary algorithms and their models
TL;DR: A comprehensive survey of the state-of-the-art distributed evolutionary algorithms and models, which have been classified into two groups according to their task division mechanism, and insights into the models are presented and discussed.
Journal Article
Cloud computing:architecture and key technologies
TL;DR: The up-to-date key technologies and research progresses of the three layers within the cloud framework are reviewed intensively and extensively and both QoS guarantee and security/privacy protection are discussed in depth.
Journal ArticleDOI
Genetic algorithms in wireless networking: techniques, applications, and issues
TL;DR: This paper is the first paper, to the best of the knowledge, which focuses on Genetic algorithms application in wireless networks and provides both an exposition of common GA models and configuration and a broad-ranging survey of GA techniques in Wireless networks.
References
More filters
Book
Genetic algorithms in search, optimization, and machine learning
TL;DR: In this article, the authors present the computer techniques, mathematical tools, and research results that will enable both students and practitioners to apply genetic algorithms to problems in many fields, including computer programming and mathematics.
Journal ArticleDOI
MapReduce: simplified data processing on large clusters
Jeffrey Dean,Sanjay Ghemawat +1 more
TL;DR: This paper presents the implementation of MapReduce, a programming model and an associated implementation for processing and generating large data sets that runs on a large cluster of commodity machines and is highly scalable.
Journal ArticleDOI
MapReduce: simplified data processing on large clusters
Jeffrey Dean,Sanjay Ghemawat +1 more
TL;DR: This presentation explains how the underlying runtime system automatically parallelizes the computation across large-scale clusters of machines, handles machine failures, and schedules inter-machine communication to make efficient use of the network and disks.
Journal ArticleDOI
The Google file system
TL;DR: This paper presents file system interface extensions designed to support distributed applications, discusses many aspects of the design, and reports measurements from both micro-benchmarks and real world use.