Adaptive Reinforcement Routing in Software Defined Vehicular Networks
15 Jun 2020-pp 2118-2123
TL;DR: The outcomes exhibit that the proposed adaptive self-learning clustering algorithm with reinforcement routing in SDVN known as RL-SDVN improved cluster stability and life-time of a cluster member vehicle with better performance in terms of low average transmission delay, and high throughput compared to the existing routing protocols used in this research.
Abstract: The integration of learning architecture with SDN-based VANETs (SDVN) is beneficial for utilizing computing power by decoupling network management services from data transfer services. However, fast safety messages dissemination in a highly dynamic vehicular environment is a challenging and complex dilemma due to bi-directional traffic and the directional movement of vehicles. It is also challenging to get an effective solution against bottleneck situations and a reliable and fault-tolerant SDN network using clustering. So considering the features of adaptive learning, in this paper, we propose adaptive self-learning clustering algorithm with reinforcement routing in SDVN known as RL-SDVN. An Expectation-Maximization model is used to predict a vehicle's movement and further Q-learning model is used to route data packets, so that vehicles in the same cluster coordinate with each other to find optimum routes. We evaluate our experimental results by comparing our approach with the clustering and self-learning based schemes proposed in the past. The outcomes exhibit that the proposed scheme improved cluster stability and life-time of a cluster member vehicle with better performance in terms of low average transmission delay, and high throughput compared to the existing routing protocols used in this research.
Citations
More filters
[...]
TL;DR: In this article , the authors present a survey of the existing P2P approaches in the context of VANETs including a comprehensive survey of various problems related to data sharing in VANet using P2Pe techniques, along with their proposed solutions from the perspective of data access, information lookup, routing, and message management.
Abstract: Vehicular ad hoc networks (VANETs) provide an effective technology for vehicles communicating while on the road. Vehicles are organized in an ad hoc networks offering support for both safety-critical messaging, multi-hop entertainment applications, and connection to internet services through Road Side Units (RSUs). Sharing data between connected vehicles can help commuting people exchange travel information, and allows content delivery and entertainment applications using Peer-to-Peer (P2P) techniques. Despite the widely documented benefits of P2P techniques, there are a number of challenges faced when deployed in such a highly dynamic environments as VANETs. Such difficulties include the broadcast storm problem, network partition, and the temporal network fragmentation to name few. These issues are principally due to the high mobility of vehicles and thus to the constantly changing topology of the network. In this paper, we present a survey of the existing P2P approaches in the context of VANETs including a comprehensive survey of the various problems related to data sharing in VANET using P2P techniques, along with their proposed solutions from the perspective of data access, information lookup, routing, and message management. In this study, we provide a classification of the studied approaches in terms of network overlays, information structures, and network coding. To differentiate between the discussed approaches, we include a comparative study based on the crucial parameters and identify and discuss the open research challenges in P2P techniques. To the best of our knowledge, this survey constitutes the first attempt at a comprehensive analysis of P2P approaches in VANETs, which describes more appropriately the state-of-art of this specific topic of research.
16 citations
[...]
TL;DR: In this paper, a comparative study of RL-based routing protocols for VANETs is presented, by considering their working procedure, advantages, disadvantages, and applications, and open issues and research challenges are discussed.
Abstract: Vehicular-ad hoc networks (VANETs) hold great importance because of their potentials in road safety improvement, traffic monitoring, and in-vehicle infotainment services. Due to high mobility, sparse connectivity, road-side obstacles, and shortage of roadside units, the links between the vehicles are subject to frequent disconnections; consequently, routing is crucial. Recently, to achieve more efficient routing, reinforcement learning (RL)-based routing algorithms have been investigated. RL represents a class of artificial intelligence that implements a learning procedure based on previous experiences and provides a better solution for future operations. RL algorithms are more favorable than other optimization techniques owing to their modest usage of memory and computational resources. Because a VANET deals with passenger safety, any kind of flaw is intolerable in VANET routing. Fortunately, RL-based algorithms have the potentials to optimize the different quality-of-service parameters of VANET routing such as bandwidth, end-to-end delay, throughput, control overhead, and packet delivery ratio. However, to the best of the authors’ knowledge, surveys on RL-based routing protocols for VANETs have not been conducted. To fulfill this gap in the literature and to provide future research directions, it is necessary to aggregate the scattered works on this topic. This study presents a comparative investigation of RL-based routing protocols, by considering their working procedure, advantages, disadvantages, and applications. They are qualitatively compared in terms of key features, characteristics, optimization criteria, performance evaluation techniques, and implemented RL techniques. Lastly, open issues and research challenges are discussed to make RL-based VANET routing protocols more efficient in the future.
5 citations
[...]
TL;DR: In this article , the authors proposed MetaLearn, a technique akin to global search, which employs a parameterized approach to remove future rewards uncertainty as well as vehicular state exploration to optimize the multilevel network structure.
Abstract: Routing protocols in vehicular ad-hoc networks (VANETs) are typically challenged by high vehicular mobility and changing network topology. It becomes more apparent as the inherently dispersed nature of VANETs affects the Quality-of-Service (QoS), which makes it challenging to find a routing algorithm that maximizes the network throughput. Integrating Reinforcement Learning (RL) with Meta-Heuristic (MH) techniques allow for solving constrained, high dimensional problems such as routing optimization. Motivated by this fact, we introduce MetaLearn, a technique akin to global search, which employs a parameterized approach to remove future rewards uncertainty as well as vehicular state exploration to optimize the multilevel network structure. The proposed technique searches for the optimum solution that may be sped up by balancing global exploration using Grey Wolf Optimization (GWO) and exploitation through Temporal Difference Learning (particularly Q(λ)). MetaLearn approach enables cluster heads to learn how to adjust route request forwarding according to QoS parameters. The input received by a vehicle from previous evaluations is used to learn and adapt the subsequent actions accordingly. Furthermore, a customized reward function is developed to select the cluster head and identify stable clusters through GWO. An in-depth experimental demonstration of the proposed protocol addresses applicability and solution challenges for hybrid MH-RL algorithms in VANETs.
4 citations
[...]
TL;DR: A vehicle orientation based QoS routing in vehicular ad-hoc networks (VANETs) is proposed, called OBQR, that exploits vehicle orientational information instead of magnitude information and significantly improves path selection and load balancing with betterQoS routing performance.
Abstract: The source-based routing using quality of service (QoS) metrics results in alternate path discovery considering link-availability time, link costs, and path delays to overcome the barrier of information blocking on a selected path by choosing alternative routes. However, multipath selection, as well as the cost of selecting a path, make the route selection a challenging task. This paper proposes a vehicle orientation based QoS routing in vehicular ad-hoc networks (VANETs), called OBQR that exploits vehicle orientational information instead of magnitude information. The cosine similarity concept with a preliminary scalarization model converts the multi-constraint objectives into a single constraint objective to find a set of possible paths to the destination. We evaluate the performance of our approach and compare it with existing state-of-the-art schemes based on QoS routing and clustering. Experimental results demonstrate that the proposed scheme significantly improves path selection and load balancing with better QoS routing performance.
2 citations
Cites methods from "Adaptive Reinforcement Routing in S..."
[...]
References
More filters
[...]
TL;DR: Simulation results show that the proposed protocol CPB outperforms the existing schemes in terms of information coverage, average message delay and packet delivery ratio.
Abstract: VANETs (Vehicular Ad hoc Networks) have attracted tremendous attentions due to their high applicability and commercial value. However, the frequent topology changes caused by the fast mobility of nodes create many challenges to the efficient data delivery in vehicular environment. With the aim to guarantee the stable and reliable communication between nodes, in this paper, we propose a novel data dissemination scheme based on Clustering and Probabilistic Broadcasting (CPB). A clustering algorithm is first presented according to the driving directions of vehicles, by which vehicles could exchange their data in a clustered way with sufficient connection duration. In the constructed clustering structure, a probabilistic forwarding is presented to disseminate data among vehicles. Each cluster member forwards the received packet to its cluster head with a calculated probability which is associated with the number of times the same packet is received during one interval. When receiving the sent packet, the elected cluster header continues to disseminate it toward the transmission direction. Simulation results show that our proposed protocol CPB outperforms the existing schemes in terms of information coverage, average message delay and packet delivery ratio.
115 citations
"Adaptive Reinforcement Routing in S..." refers background or methods in this paper
[...]
[...]
[...]
[...]
[...]
[...]
TL;DR: A reliable self-adaptive routing algorithm (RSAR) based on this heuristic service algorithm is proposed and, by combining the reliability parameter and adjusting the heuristic function, RSAR achieves good performance with VANET.
Abstract: As a special MANET (mobile ad hoc network), VANET (vehicular ad-hoc network) has two important properties: the network topology changes frequently, and communication links are unreliable. Both properties are caused by vehicle mobility. To predict the reliability of links between vehicles effectively and design a reliable routing service protocol to meet various QoS application requirements, in this paper, details of the motion characteristics of vehicles and the reasons that cause links to go down are analyzed. Then a link duration model based on time duration is proposed. Link reliability is evaluated and used as a key parameter to design a new routing protocol. Quick changes in topology make it a huge challenge to find and maintain the end-to-end optimal path, but the heuristic Q-Learning algorithm can dynamically adjust the routing path through interaction with the surrounding environment. This paper proposes a reliable self-adaptive routing algorithm (RSAR) based on this heuristic service algorithm. By combining the reliability parameter and adjusting the heuristic function, RSAR achieves good performance with VANET. With the NS-2 simulator, RSAR performance is proved. The results show that RSAR is very useful for many VANET applications.
98 citations
[...]
TL;DR: A hierarchical SDN-based vehicular architecture that aims to have improved performance in the situation of loss of connection with the central SDN controller and outperforms traditional routing protocols in the scenario where there is no coordination from the centralSDN controller.
Abstract: With the recent advances in the telecommunications and auto industries, we have witnessed growing interest in ITS, of which VANETs are an essential component. SDN can bring advantages to ITS through its ability to provide flexibility and programmability to networks through a logically centralized controller entity that has a comprehensive view of the network. However, as the SDN paradigm initially had fixed networks in mind, adapting it to work on VANETs requires some changes to address particular characteristics of this kind of scenario, such as the high mobility of its nodes. There has been initial work on bringing SDN concepts to vehicular networks to expand its abilities to provide applications and services through the increased flexibility, but most of these studies do not directly tackle the issue of loss of connectivity with said controller entity. In this article, we propose a hierarchical SDN-based vehicular architecture that aims to have improved performance in the situation of loss of connection with the central SDN controller. Simulation results show that our proposal outperforms traditional routing protocols in the scenario where there is no coordination from the central SDN controller.
84 citations
"Adaptive Reinforcement Routing in S..." refers background in this paper
[...]
[...]
TL;DR: A new dynamic mobility-based and stability-based clustering scheme is introduced for urban city scenario that applies vehicle's moving direction, relative position and link lifetime estimation and shows a better stability performance.
Abstract: Vehicle clustering is an efficient approach to improve the scalability of networking protocols in vehicular ad-hoc networks (VANETs). However, some characteristics, like highly dynamic topology and intermittent connections, may affect the performance of the clustering. Establishing and maintaining stable clusters is becoming one of big challenging issues in VANETs. Recent years' researches prove that mobility metric based clustering schemes show better performance in improving cluster stability. Mobility metrics, including moving direction, vehicle density, relative velocity and relative distance, etc., are more suitable for VANETs instead of the received radio strength (RSS) and identifier number metrics, which are applied for MANETs clustering. In this paper, a new dynamic mobility-based and stability-based clustering scheme is introduced for urban city scenario. The proposed scheme applies vehicle's moving direction, relative position and link lifetime estimation. We compared the performance of our scheme with Lowest-ID and the most recent and the most cited clustering algorithm VMaSC in terms of cluster head duration, cluster member duration, number of clusters, cluster head change rate and number of state changes. The extensive simulation results showed that our proposed scheme shows a better stability performance.
84 citations
Related Papers (5)
[...]
[...]
[...]