scispace - formally typeset
Search or ask a question
Proceedings ArticleDOI

SeScR: SDN-Enabled Spectral Clustering-Based Optimized Routing Using Deep Learning in VANET Environment

24 Nov 2020-pp 1-9
TL;DR: In this article, a spectral clustering technique along with the deep deterministic policy gradient (DDPG) algorithm using hybrid SDN architecture is proposed to enhance cluster stability and route selection method.
Abstract: In recent years, integration of clustering architecture with software-defined networking (SDN) has emerged as is the crucial enabler for next-generation intelligent transportation services (ITS). This paper proposes a spectral clustering technique along with the deep deterministic policy gradient (DDPG) algorithm using hybrid SDN architecture, called SeScR to enhance cluster stability and route selection method. The spectral clustering is used to overcome the arbitrary node distribution of vehicular ad-hoc networks (VANETs) and provide a flexible clustering using eigenvalues of graph laplacian. Moreover, the DDPG algorithm addresses the continuous address space of VANETs and provides an actor-critic architecture for optimal routing decisions. The experimental results demonstrate that the proposed scheme improves path selection and load balancing with better performance in terms of low average transmission delay up to 15%, throughput up to 18-22%, and low computation overhead 10% compared to the existing state-of-the-art protocols used in this research.
Citations
More filters
Journal ArticleDOI
TL;DR: In this paper, the authors present a survey of machine learning techniques for routing optimization in SDN based on three core categories (i.e., supervised learning, unsupervised learning, and reinforcement learning).
Abstract: In conventional networks, there was a tight bond between the control plane and the data plane. The introduction of Software-Defined Networking (SDN) separated these planes, and provided additional features and tools to solve some of the problems of traditional network (i.e., latency, consistency, efficiency). SDN is a flexible networking paradigm that boosts network control, programmability and automation. It proffers many benefits in many areas, including routing. More specifically, for efficiently organizing, managing and optimizing routing in networks, some intelligence is required, and SDN offers the possibility to easily integrate it. To this purpose, many researchers implemented different machine learning (ML) techniques to enhance SDN routing applications. This article surveys the use of ML techniques for routing optimization in SDN based on three core categories (i.e. supervised learning, unsupervised learning, and reinforcement learning). The main contributions of this survey are threefold. Firstly, it presents detailed summary tables related to these studies and their comparison is also discussed, including a summary of the best works according to our analysis. Secondly, it summarizes the main findings, best works and missing aspects, and it includes a quick guideline to choose the best ML technique in this field (based on available resources and objectives). Finally, it provides specific future research directions divided into six sections to conclude the survey. Our conclusion is that there is a huge trend to use intelligence-based routing in programmable networks, particularly during the last three years, but a lot of effort is still required to achieve comprehensive comparisons and synergies of approaches, meaningful evaluations based on open datasets and topologies, and detailed practical implementations (following recent standards) that could be adopted by industry. In summary, future efforts should be focused on reproducible research rather than on new isolated ideas. Otherwise, most of these applications will be barely implemented in practice.

31 citations

Journal ArticleDOI
TL;DR: A topology-aware resilient routing strategy based on adaptive TARRAQ-learning to accurately capture topology changes with low overhead and make routing decisions in a distributed and autonomous way is proposed.
Abstract: Flying ad hoc networks (FANETs) play a crucial role in numerous military and civil applications since it shortens mission duration and enhances coverage significantly compared with a single unmanned aerial vehicle (UAV). Whereas, designing an energy-efficient FANETs routing protocol with a high packet delivery rate (PDR) and low delay is challenging owing to the dynamic topology changes. In this article, we propose a topology-aware resilient routing strategy based on adaptive $Q$ -learning (TARRAQ) to accurately capture topology changes with low overhead and make routing decisions in a distributed and autonomous way. First, we analyze the dynamic behavior of UAVs nodes via the queuing theory, and then the closed-form solutions of neighbors’ change rate (NCR) and neighbors’ change interarrival time (NCIT) distribution are derived. Based on the real-time NCR and NCIT, a resilient sensing interval (SI) is determined by defining the expected sensing delay of network events. Besides, we also present an adaptive $Q$ -learning approach that enables UAVs to make distributed, autonomous, and adaptive routing decisions, where the above SI ensures that the action space can be updated in time with low cost. The simulation results verify the accuracy of the topology dynamic analysis model, and also prove that our TARRAQ outperforms the $Q$ -learning-based topology-aware routing (QTAR), mobility prediction-based virtual routing (MPVR), and greedy perimeter stateless routing based on energy-efficient hello (EE-Hello) in terms of 25.23%, 20.24%, and 13.73% lower overhead, 9.41%, 14.77%, and 16.70% higher PDR, and 5.12%, 15.65%, and 11.31% lower energy consumption, respectively.

11 citations

Journal ArticleDOI
TL;DR: In this paper , a topology-aware resilient routing strategy based on adaptive Q-learning (TARRAQ) is proposed to accurately capture topology changes with low overhead and make routing decisions in a distributed and autonomous way.
Abstract: Flying ad hoc networks (FANETs) play a crucial role in numerous military and civil applications since it shortens mission duration and enhances coverage significantly compared with a single unmanned aerial vehicle (UAV). Whereas, designing an energy-efficient FANET routing protocol with a high packet delivery rate (PDR) and low delay is challenging owing to the dynamic topology changes. In this article, we propose a topology-aware resilient routing strategy based on adaptive Q-learning (TARRAQ) to accurately capture topology changes with low overhead and make routing decisions in a distributed and autonomous way. First, we analyze the dynamic behavior of UAV nodes via the queuing theory, and then the closed-form solutions of neighbors' change rate (NCR) and neighbors' change interarrival time (NCIT) distribution are derived. Based on the real-time NCR and NCIT, a resilient sensing interval (SI) is determined by defining the expected sensing delay of network events. Besides, we also present an adaptive Q-learning approach that enables UAVs to make distributed, autonomous, and adaptive routing decisions, where the above SI ensures that the action space can be updated in time at a low cost. The simulation results verify the accuracy of the topology dynamic analysis model and also prove that our TARRAQ outperforms the Q-learning-based topology-aware routing (QTAR), mobility prediction-based virtual routing (MPVR), and greedy perimeter stateless routing based on energy-efficient hello (EE-Hello) in terms of 25.23%, 20.24%, and 13.73% lower overhead, 9.41%, 14.77%, and 16.70% higher PDR, and 5.12%, 15.65%, and 11.31% lower energy consumption, respectively.

8 citations

Journal ArticleDOI
TL;DR: In this paper , the authors review reinforcement learning and its characteristics and study how to use this technique for creating routing protocols in VANETs and propose a categorization of RL-based routing schemes in these networks.
Abstract: Today, the use of safety solutions in Intelligent Transportation Systems (ITS) is a serious challenge because of novel progress in wireless technologies and the high number of road accidents. Vehicular ad hoc network (VANET) is a momentous element in this system because they can improve safety and efficiency in ITS. In this network, vehicles act as moving nodes and work with other nodes within their communication range. Due to high-dynamic vehicles and their different speeds in this network, links between vehicles are valid for a short time interval. Therefore, routing is a challenging work in these networks. Recently, reinforcement learning (RL) plays a significant role in developing routing algorithms for VANET. In this paper, we review reinforcement learning and its characteristics and study how to use this technique for creating routing protocols in VANETs. We propose a categorization of RL-based routing schemes in these networks. This paper helps researchers to understand how to design RL-based routing algorithms in VANET and improve the existing methods by understanding the challenges and opportunities in this area.

2 citations

Journal ArticleDOI
TL;DR: In this paper , a comprehensive categorisation of RL-based routing protocols for both network types, having in mind their simultaneous use and the inclusion with other technologies is performed, based on different factors that influence the reward function in RL and the consequences they have on network performance.
Abstract: Vehicular and flying ad hoc networks (VANETs and FANETs) are becoming increasingly important with the development of smart cities and intelligent transportation systems (ITSs). The high mobility of nodes in these networks leads to frequent link breaks, which complicates the discovery of optimal route from source to destination and degrades network performance. One way to overcome this problem is to use machine learning (ML) in the routing process, and the most promising among different ML types is reinforcement learning (RL). Although there are several surveys on RL-based routing protocols for VANETs and FANETs, an important issue of integrating RL with well-established modern technologies, such as software-defined networking (SDN) or blockchain, has not been adequately addressed, especially when used in complex ITSs. In this paper, we focus on performing a comprehensive categorisation of RL-based routing protocols for both network types, having in mind their simultaneous use and the inclusion with other technologies. A detailed comparative analysis of protocols is carried out based on different factors that influence the reward function in RL and the consequences they have on network performance. Also, the key advantages and limitations of RL-based routing are discussed in detail.
References
More filters
Book ChapterDOI
01 Jan 2017
TL;DR: An Angle-based Clustering Algorithm (ACA) is proposed, which exploits the angular position and the direction of the vehicles to select the most stable vehicles that can act as cluster heads for a long period of time and significantly outperforms other clustering protocols in terms of cluster stability.
Abstract: A vehicular ad hoc network (VANET) is a mobile network in which vehicles acting as moving nodes communicate with each other through an ad hoc wireless network. VANETs have become the core component of Intelligent Transportation Systems (ITS) which aim to improve the road safety and efficiency. Only if the communication scheme used in a VANET is stable can these aims be achieved. Frequent changes in network topology and breaks in communication raise challenging issues in the design of communication protocols for such networks. Currently, clustering algorithms are being used as the control schemes to reduce changes in VANET topologies. However, the design of a clustering algorithm becomes a difficult task in VANETs when there are many road segments and intersections. In this work, we propose an Angle-based Clustering Algorithm (ACA), which exploits the angular position and the direction of the vehicles to select the most stable vehicles that can act as cluster heads for a long period of time. The simulation results reveal that ACA significantly outperforms other clustering protocols in terms of cluster stability.

22 citations

Journal ArticleDOI
Xiang Bi1, Baishun Guo1, Lei Shi1, Yang Lu1, Lin Feng1, Zengwei Lyu1 
TL;DR: A new AP clustering algorithm for the whole clustering process is proposed that performs better than other algorithms in terms of the cluster stability, and it also effectively improves throughput and reduces packet loss rate of VANETs over the classical APROVE algorithm and the NMDP-APC algorithm.
Abstract: Clustering is an efficient method for improving the communication performance of Vehicular Ad hoc NETworks (VANETs) that adopt Vehicle to Vehicle (V2V) communications. However, how to maximize the cluster stability while accounting for the high mobility of vehicles remains a challenging problem. In this paper, we first reconstruct the similarity function of the Affinity Propagation (AP) clustering algorithm by introducing communication-related parameters, so the vehicles with low relative mobility and good communication performance can easily be selected as cluster heads. Then, by formally defining three scaling functions, a weighted mechanism is designed to quantitatively assess the effect on the cluster stability when a vehicle joins it. Base on them, from the perspective of global balance, a new AP clustering algorithm for the whole clustering process is proposed. To ensure the validity of simulations, we use the vehicular mobility data generated on the realistic map of Cologne, Germany, and perform a series of simulations for eleven metrics commonly adopted in similar works. The results show that our proposed algorithm performs better than other algorithms in terms of the cluster stability, and it also effectively improves throughput and reduces packet loss rate of VANETs over the classical APROVE algorithm and the NMDP-APC algorithm.

18 citations

Proceedings ArticleDOI
01 Oct 2019
TL;DR: The eFoV of a vehicle is characterised as the perception range using local sensors only, and the collective Field of View as the region learn from the network, which suggests that the sharing of sensory information should be controlled appropriately to avoid overloading the communication networks.
Abstract: With the introduction of Connected and Autonomous Vehicles (CAVs), it is possible to extend the limited horizon of vehicles on the road by collective perceptions, where vehicles periodically share their sensory information with others using Vehide-2-Vehicle (V2V) communications. This technique relies on a certain number of participants to have a measurable advantage. Nevertheless, the spread of CAVs will take a considerable period of time, it is critical to understand the benefits and limits of V2V based collective perceptions in different market stages. In this work, we characterise the effective Field of View (eFoV) of a vehicle as the perception range using local sensors only, and the collective Field of View (cFoV) as the region learn from the network. Applying analytic and simulation studies in highway scenarios, we show that the eFoV drops quickly with the increase in traffic density due to blockage effects of surrounding vehicles, and it is insufficient to overcome this problem by increasing the sensing range of local sensors. On the other hand, vehicles can gain around 16 folds more information about the road environment by leveraging collective perceptions with only 10% CAV penetration rate. When the penetration rate reaches to around 30%, collective perceptions can provide 95% coverage over the road environments. Our analyses also show that apart from the benefits, employing collective perceptions could result in heavy broadcast redundancy, hence wasting the already scarce network resources. This observation suggests that the sharing of sensory information should be controlled appropriately to avoid overloading the communication networks.

12 citations

Proceedings ArticleDOI
07 Jun 2020
TL;DR: Simulation results show that the collaborative computing approach can adapt to different service environments and outperform the greedy offloading approach.
Abstract: Mobile edge computing (MEC) has been recognized as a promising technology to support various emerging services in vehicular networks. With MEC, vehicle users can offload their computation-intensive applications (e.g., intelligent path planning and safety applications) to edge computing servers located at roadside units. In this paper, an efficient computing offloading and server collaboration approach is proposed to reduce computing service delay and improve service reliability for vehicle users. Task partition is adopted, whereby the computation load offloaded by a vehicle can be divided and distributed to multiple edge servers. By the proposed approach, the computation delay can be reduced by parallel computing, and the failure in computing results delivery can also be alleviated via cooperation among edges. The offloading and computing decision-making is formulated as a long-term planning problem, and a deep reinforcement learning technique, i.e., deep deterministic policy gradient, is adopted to achieve the optimal solution of the complex stochastic nonlinear integer optimization problem. Simulation results show that our collaborative computing approach can adapt to different service environments and outperform the greedy offloading approach.

11 citations

Proceedings ArticleDOI
01 Sep 2019
TL;DR: A new distributed mobility-based multi-hop clustering algorithm (DMMCA) in highway scenarios that uses the vehicle's moving direction, relative speed and relative position to construct theMulti-hop cluster is proposed.
Abstract: In vehicular ad hoc networks (VANETs), due to the high dynamics of the vehicles, clustering is an effective way to alleviate the problem of frequent communication interruption. In recent years, many pieces of research have pointed out that clustering can improve the stability of the cluster by using similar mobility patterns of the vehicles. In this paper, we propose a new distributed mobility-based multi-hop clustering algorithm (DMMCA) in highway scenarios. DMMCA uses the vehicle's moving direction, relative speed and relative position to construct the multi-hop cluster. In addition, in order to reduce the overhead of the clustering algorithm, we design a new hello packet rebroadcast mode. Extensive simulation experiments are performed using ns-3 to demonstrate our proposed algorithm. The results show that our proposed algorithm can improve the performance of the cluster in terms of average cluster head duration, average cluster member duration, average state changes. Moreover, the number of rebroadcast hello packets is also reduced.

9 citations