Journal ArticleDOI
DDQP: A Double Deep Q-Learning Approach to Online Fault-Tolerant SFC Placement
TLDR
In this article, a double deep Q-networks based online SFC placement scheme DDQP is proposed to deal with large continuous network state space, which offers constant generated state updates from active instances to standby instances to guarantee seamless redirection after failures.Abstract:
Since Network Function Virtualization (NFV) decouples network functions (NFs) from the underlying dedicated hardware and realizes them in the form of software called Virtual Network Functions (VNFs), they are enabled to run in any resource-sufficient virtual machines. A service function chain (SFC) is composed of a sequential set of VNFs. As VNFs are vulnerable to various faults such as software failures, we consider how to deploy both active and standby SFC instances. Given the complexity and unpredictability of the network state, we propose a double deep Q-networks based online SFC placement scheme DDQP. Specifically, DDQP uses deep neural networks to deal with large continuous network state space. In the case of stateful VNFs, we offer constant generated state updates from active instances to standby instances to guarantee seamless redirection after failures. With the goal of balancing the waste of resources and ensuring service reliability, we introduce five progressive schemes of resource reservations to meet different customer needs. Our experimental results demonstrate that DDQP responds rapidly to arriving requests and reaches near-optimal performance. Specifically, DDQP outweighs the state-of-the-art method by 16.30% and 38.51% higher acceptance ratio under different schemes with 82x speedup on average. In order to enhance the integrity of the SFC state transition, we further proposed DDQP+, which extends DDQP by adding the delayed placement mechanism. Compared with DDQP, the design of the DDQP+ algorithm is more reasonable and comprehensive. The experiment results also show that DDQP+ achieved further improvement in multiple performance indicators.read more
Citations
More filters
Journal ArticleDOI
Deep Reinforcement Learning for Resource Management on Network Slicing: A Survey
TL;DR: This paper identifies the relevant phases for resource management in network slicing and analyzes approaches using reinforcement learning (RL) and DRL algorithms for realizing each phase autonomously.
Journal ArticleDOI
Network Function Virtualization and Service Function Chaining Frameworks: A Comprehensive Review of Requirements, Objectives, Implementations, and Open Research Challenges
TL;DR: This work reviews the state-of-the-art NFV and SFC implementation frameworks and presents a taxonomy of the current proposals, which comprises three major categories based on the primary objectives of each of the surveyed frameworks: resource allocation and service orchestration, performance tuning, and resilience and fault recovery.
Journal ArticleDOI
SARM: Service function chain active reconfiguration mechanism based on load and demand prediction
TL;DR: An SFC active reconfiguration mechanism (SARM) based on computational load and resource demand is proposed and it is demonstrated that the SARM can effectively predict the nodes' load and the resource demand of SFCs.
Journal ArticleDOI
A reinforcement learning-based approach for availability-aware service function chain placement in large-scale networks
TL;DR: In this article , the authors proposed a solution for SFC placement based on reinforcement learning (RL) taking into account SFC availability, operational costs, and energy consumption, and compared two greedy algorithms in a variety of simulated scenarios.
References
More filters
Proceedings Article
ImageNet Classification with Deep Convolutional Neural Networks
TL;DR: The state-of-the-art performance of CNNs was achieved by Deep Convolutional Neural Networks (DCNNs) as discussed by the authors, which consists of five convolutional layers, some of which are followed by max-pooling layers, and three fully-connected layers with a final 1000-way softmax.
Journal ArticleDOI
Human-level control through deep reinforcement learning
Volodymyr Mnih,Koray Kavukcuoglu,David Silver,Andrei Rusu,Joel Veness,Marc G. Bellemare,Alex Graves,Martin Riedmiller,Andreas K. Fidjeland,Georg Ostrovski,Stig Petersen,Charles Beattie,Amir Sadik,Ioannis Antonoglou,Helen King,Dharshan Kumaran,Daan Wierstra,Shane Legg,Demis Hassabis +18 more
TL;DR: This work bridges the divide between high-dimensional sensory inputs and actions, resulting in the first artificial agent that is capable of learning to excel at a diverse array of challenging tasks.
Proceedings Article
Rectified Linear Units Improve Restricted Boltzmann Machines
Vinod Nair,Geoffrey E. Hinton +1 more
TL;DR: Restricted Boltzmann machines were developed using binary stochastic hidden units that learn features that are better for object recognition on the NORB dataset and face verification on the Labeled Faces in the Wild dataset.
Journal ArticleDOI
Mastering the game of Go with deep neural networks and tree search
David Silver,Aja Huang,Chris J. Maddison,Arthur Guez,Laurent Sifre,George van den Driessche,Julian Schrittwieser,Ioannis Antonoglou,Veda Panneershelvam,Marc Lanctot,Sander Dieleman,Dominik Grewe,John Nham,Nal Kalchbrenner,Ilya Sutskever,Timothy P. Lillicrap,Madeleine Leach,Koray Kavukcuoglu,Thore Graepel,Demis Hassabis +19 more
TL;DR: Using this search algorithm, the program AlphaGo achieved a 99.8% winning rate against other Go programs, and defeated the human European Go champion by 5 games to 0.5, the first time that a computer program has defeated a human professional player in the full-sized game of Go.
Proceedings Article
PyTorch: An Imperative Style, High-Performance Deep Learning Library
Adam Paszke,Sam Gross,Francisco Massa,Adam Lerer,James Bradbury,Gregory Chanan,Trevor Killeen,Zeming Lin,Natalia Gimelshein,Luca Antiga,Alban Desmaison,Andreas Kopf,Edward Z. Yang,Zachary DeVito,Martin Raison,Alykhan Tejani,Sasank Chilamkurthy,Benoit Steiner,Lu Fang,Junjie Bai,Soumith Chintala +20 more
TL;DR: This paper details the principles that drove the implementation of PyTorch and how they are reflected in its architecture, and explains how the careful and pragmatic implementation of the key components of its runtime enables them to work together to achieve compelling performance.