Directional routing and scheduling for green vehicular delay tolerant networks
TL;DR: A directional routing and scheduling scheme (DRSS) for green vehicle DTNs is presented by using Nash Q-learning approach that can optimize the energy efficiency with the considerations of congestion, buffer and delay.
Abstract: The vehicle delay tolerant networks (DTNs) make opportunistic communications by utilizing the mobility of vehicles, where the node makes delay-tolerant based "carry and forward" mechanism to deliver the packets. The routing schemes for vehicle networks are challenging for varied network environment. Most of the existing DTN routing including routing for vehicular DTNs mainly focus on metrics such as delay, hop count and bandwidth, etc. A new focus in green communications is with the goal of saving energy by optimizing network performance and ultimately protecting the natural climate. The energy---efficient communication schemes designed for vehicular networks are imminent because of the pollution, energy consumption and heat dissipation. In this paper, we present a directional routing and scheduling scheme (DRSS) for green vehicle DTNs by using Nash Q-learning approach that can optimize the energy efficiency with the considerations of congestion, buffer and delay. Our scheme solves the routing and scheduling problem as a learning process by geographic routing and flow control toward the optimal direction. To speed up the learning process, our scheme uses a hybrid method with forwarding and replication according to traffic pattern. The DRSS algorithm explores the possible strategies, and then exploits the knowledge obtained to adapt its strategy and achieve the desired overall objective when considering the stochastic non-cooperative game in on-line multi-commodity routing situations. The simulation results of a vehicular DTN with predetermined mobility model show DRSS achieves good energy efficiency with learning ability, which can guarantee the delivery ratio within the delay bound.
Citations
More filters
01 Feb 2020
TL;DR: A survey on various ML techniques applied to communication, networking, and security parts in vehicular networks and envision the ways of enabling AI toward a future 6G vehicular network, including the evolution of intelligent radio (IR), network intelligentization, and self-learning with proactive exploration.
Abstract: As a powerful tool, the vehicular network has been built to connect human communication and transportation around the world for many years to come. However, with the rapid growth of vehicles, the vehicular network becomes heterogeneous, dynamic, and large scaled, which makes it difficult to meet the strict requirements, such as ultralow latency, high reliability, high security, and massive connections of the next-generation (6G) network. Recently, machine learning (ML) has emerged as a powerful artificial intelligence (AI) technique to make both the vehicle and wireless communication highly efficient and adaptable. Naturally, employing ML into vehicular communication and network becomes a hot topic and is being widely studied in both academia and industry, paving the way for the future intelligentization in 6G vehicular networks. In this article, we provide a survey on various ML techniques applied to communication, networking, and security parts in vehicular networks and envision the ways of enabling AI toward a future 6G vehicular network, including the evolution of intelligent radio (IR), network intelligentization, and self-learning with proactive exploration.
414 citations
Cites background from "Directional routing and scheduling ..."
...In the related works, the typical learning algorithms, such as Q-learning, collaborative reinforcement learning (CRL), and combined Q-learning and transfer learning [104]–[106], are employed....
[...]
TL;DR: An algorithm called fuzzy based unequal clustering is proposed in this paper to enhance the execution of the current algorithms and is compared with two algorithms, one with an equivalent clustering algorithm called LEACH and another with an unequal clusters algorithm called EAUCF.
Abstract: The primary challenges in outlining and arranging the operations of wireless sensor networks are to enhance energy utilization and the system lifetime. Clustering is a powerful approach to arranging a system into an associated order, load adjusting and enhancing the system lifetime. In a cluster based network, cluster head closer to the sink depletes its energy quickly resulting in hot spot problems. To conquer this issue, numerous algorithms on unequal clustering are contemplated. The drawback in these algorithms is that the nodes which join with the specific cluster head bring overburden for the cluster head. So, we propose an algorithm called fuzzy based unequal clustering in this paper to enhance the execution of the current algorithms. The proposed work is assessed by utilizing simulation. The proposed algorithm is compared with two algorithms, one with an equivalent clustering algorithm called LEACH and another with an unequal clustering algorithm called EAUCF. The simulation results using MATLAB demonstrate that the proposed algorithm provides better performance compared to the other two algorithms.
191 citations
Cites background from "Directional routing and scheduling ..."
...Other areas in the wireless networks includes MAC protocols in WSN and underwater sensor networks [20], routing and scheduling for green vehicular delay tolerant networks [21], service configuration and traffic distribution in composite radio environments [22] and trust management for internet of things [23]....
[...]
TL;DR: The experimental results shows that the proposed EE-LEACH yields better performance than the existing energy-balanced routing protocol (EBRP) and LEACH Protocol in terms of better packet delivery ratio, lesser end-to-end delay and energy consumption.
Abstract: A wireless sensor network (WSN) consists of a huge number of sensor nodes that are inadequate in energy, storage and processing power. One of the major tasks of the sensor nodes is the collection of data and forwarding the gathered data to the base station (BS). Hence, the network lifetime becomes the major criteria for effective design of the data gathering schemes in WSN. In this paper, an energy-efficient LEACH (EE-LEACH) Protocol for data gathering is introduced. It offers an energy-efficient routing in WSN based on the effective data ensemble and optimal clustering. In this system, a cluster head is elected for each clusters to minimize the energy dissipation of the sensor nodes and to optimize the resource utilization. The energy-efficient routing can be obtained by nodes which have the maximum residual energy. Hence, the highest residual energy nodes are selected to forward the data to BS. It helps to provide better packet delivery ratio with lesser energy utilization. The experimental results shows that the proposed EE-LEACH yields better performance than the existing energy-balanced routing protocol (EBRP) and LEACH Protocol in terms of better packet delivery ratio, lesser end-to-end delay and energy consumption. It is obviously proves that the proposed EE-LEACH can improve the network lifetime.
187 citations
Cites methods from "Directional routing and scheduling ..."
...A hybrid method with forwarding and replication was presented to speed up the learning process according to the traffic pattern [28]....
[...]
TL;DR: Simulation results demonstrate the feasibility of the proposed learning approach at enabling agents to learn how to flock in a leader-follower topology, while operating in a nonstationary stochastic environment.
Abstract: In the past two decades, unmanned aerial vehicles (UAVs) have demonstrated their efficacy in supporting both military and civilian applications, where tasks can be dull, dirty, dangerous, or simply too costly with conventional methods. Many of the applications contain tasks that can be executed in parallel, hence the natural progression is to deploy multiple UAVs working together as a force multiplier. However, to do so requires autonomous coordination among the UAVs, similar to swarming behaviors seen in animals and insects. This paper looks at flocking with small fixed-wing UAVs in the context of a model-free reinforcement learning problem. In particular, Peng’s $Q(\lambda )$ with a variable learning rate is employed by the followers to learn a control policy that facilitates flocking in a leader-follower topology. The problem is structured as a Markov decision process, where the agents are modeled as small fixed-wing UAVs that experience stochasticity due to disturbances such as winds and control noises, as well as weight and balance issues. Learned policies are compared to ones solved using stochastic optimal control (i.e., dynamic programming) by evaluating the average cost incurred during flight according to a cost function. Simulation results demonstrate the feasibility of the proposed learning approach at enabling agents to learn how to flock in a leader-follower topology, while operating in a nonstationary stochastic environment.
148 citations
Cites methods from "Directional routing and scheduling ..."
...RL has been applied to a large number of problems such as communications [36]–[39], vehicular networks [40], and...
[...]
TL;DR: A novel threshold credit-based incentive mechanism (TCBI) is proposed based on the modified model of population dynamics to efficiently resist the node compromise attacks, stimulate the cooperation among intermediate nodes, maximize vehicular nodes' interest, and realize the fairness of possessing the same opportunity of transmitting packets for credits.
Abstract: Cloud-assisted vehicular delay tolerant networks (DTNs) have been utilized in wide-ranging applications where a continuous end-to-end connection is unavailable, the message transmission is fulfilled by the cooperation among vehicular nodes and follows a store-carry-and-forward manner, and the complex computational work can be delegated to the disengaged vehicles in the parking lots which constitute the potential vehicular cloud. Nevertheless, the existing incentive schemes as well as the packet forwarding protocols cannot well model continuous vehicle collaboration, resist vehicle compromise attacks and collusion attacks, leaving the privacy preservation issues untouched. In this paper, a novel threshold credit-based incentive mechanism (TCBI) is proposed based on the modified model of population dynamics to efficiently resist the node compromise attacks, stimulate the cooperation among intermediate nodes, maximize vehicular nodes’ interest, and realize the fairness of possessing the same opportunity of transmitting packets for credits. Then, a TCBI-based privacy-preserving packet forwarding protocol is proposed to solve the open problem of resisting layer-adding attack by outsourcing the privacy-preserving aggregated transmission evidence generation for multiple resource-constrained vehicles to the cloud side from performing any one-way trapdoor function only once. The vehicle privacy is well protected from both the cloud and transportation manager. Finally, formal security proof and the extensive simulation show the effectiveness of our proposed TCBI in resisting the sophisticated attacks and the efficiency in terms of high reliability, high delivery ratio, and low average delay in cloud-assisted vehicular DTNs.
133 citations
Cites background from "Directional routing and scheduling ..."
...resource-constrained vehicle tries to cooperate in the energyconsuming task of message delivery, achieving the optimal reward by sacrificing the minimum cost [5]–[7]....
[...]
References
More filters
Book•
01 Jan 1988TL;DR: This book provides a clear and simple account of the key ideas and algorithms of reinforcement learning, which ranges from the history of the field's intellectual foundations to the most recent developments and applications.
Abstract: Reinforcement learning, one of the most active research areas in artificial intelligence, is a computational approach to learning whereby an agent tries to maximize the total amount of reward it receives when interacting with a complex, uncertain environment. In Reinforcement Learning, Richard Sutton and Andrew Barto provide a clear and simple account of the key ideas and algorithms of reinforcement learning. Their discussion ranges from the history of the field's intellectual foundations to the most recent developments and applications. The only necessary mathematical background is familiarity with elementary concepts of probability. The book is divided into three parts. Part I defines the reinforcement learning problem in terms of Markov decision processes. Part II provides basic solution methods: dynamic programming, Monte Carlo methods, and temporal-difference learning. Part III presents a unified view of the solution methods and incorporates artificial neural networks, eligibility traces, and planning; the two final chapters present case studies and consider the future of reinforcement learning.
37,989 citations
"Directional routing and scheduling ..." refers background in this paper
...According to [31], the Nash equilibrium NashQi is the agent i’s payoff in state s for the selected equilibrium, which is with equilibrium strategy p 1 p m for the stage game (Q1(s) Qm(s))computed from Formula (16)....
[...]
01 Jan 2000
TL;DR: This work introduces Epidemic Routing, where random pair-wise exchanges of messages among mobile hosts ensure eventual message delivery and achieves eventual delivery of 100% of messages with reasonable aggregate resource consumption in a number of interesting scenarios.
Abstract: Mobile ad hoc routing protocols allow nodes with wireless adaptors to communicate with one another without any pre-existing network infrastructure. Existing ad hoc routing protocols, while robust to rapidly changing network topology, assume the presence of a connected path from source to destination. Given power limitations, the advent of short-range wireless networks, and the wide physical conditions over which ad hoc networks must be deployed, in some scenarios it is likely that this assumption is invalid. In this work, we develop techniques to deliver messages in the case where there is never a connected path from source to destination or when a network partition exists at the time a message is originated. To this end, we introduce Epidemic Routing, where random pair-wise exchanges of messages among mobile hosts ensure eventual message delivery. The goals of Epidemic Routing are to: i) maximize message delivery rate, ii) minimize message latency, and iii) minimize the total resources consumed in message delivery. Through an implementation in the Monarch simulator, we show that Epidemic Routing achieves eventual delivery of 100% of messages with reasonable aggregate resource consumption in a number of interesting scenarios.
4,355 citations
"Directional routing and scheduling ..." refers background or result in this paper
...protocol [11] is a naive flooding, but it wastes resources and degrade the network performance....
[...]
...We compare our DRSS with epidemic [11] and modified Dijkstra algorithm [18]....
[...]
25 Aug 2003
TL;DR: This work proposes a network architecture and application interface structured around optionally-reliable asynchronous message forwarding, with limited expectations of end-to-end connectivity and node resources.
Abstract: The highly successful architecture and protocols of today's Internet may operate poorly in environments characterized by very long delay paths and frequent network partitions. These problems are exacerbated by end nodes with limited power or memory resources. Often deployed in mobile and extreme environments lacking continuous connectivity, many such networks have their own specialized protocols, and do not utilize IP. To achieve interoperability between them, we propose a network architecture and application interface structured around optionally-reliable asynchronous message forwarding, with limited expectations of end-to-end connectivity and node resources. The architecture operates as an overlay above the transport layers of the networks it interconnects, and provides key services such as in-network data storage and retransmission, interoperable naming, authenticated forwarding and a coarse-grained class of service.
3,511 citations
TL;DR: The per-session throughput for applications with loose delay constraints, such that the topology changes over the time-scale of packet delivery, can be increased dramatically under this assumption, and a form of multiuser diversity via packet relaying is exploited.
Abstract: The capacity of ad hoc wireless networks is constrained by the mutual interference of concurrent transmissions between nodes. We study a model of an ad hoc network where n nodes communicate in random source-destination pairs. These nodes are assumed to be mobile. We examine the per-session throughput for applications with loose delay constraints, such that the topology changes over the time-scale of packet delivery. Under this assumption, the per-user throughput can increase dramatically when nodes are mobile rather than fixed. This improvement can be achieved by exploiting a form of multiuser diversity via packet relaying.
2,736 citations
"Directional routing and scheduling ..." refers background in this paper
...[2] suggest that delay-tolerant applications can take advantage of node mobility to significantly optimize the network capacity....
[...]
...tolerant routing mechanism [2]....
[...]
22 Aug 2005
TL;DR: A new routing scheme, called Spray and Wait, that "sprays" a number of copies into the network, and then "waits" till one of these nodes meets the destination, which outperforms all existing schemes with respect to both average message delivery delay and number of transmissions per message delivered.
Abstract: Intermittently connected mobile networks are sparse wireless networks where most of the time there does not exist a complete path from the source to the destination. These networks fall into the general category of Delay Tolerant Networks. There are many real networks that follow this paradigm, for example, wildlife tracking sensor networks, military networks, inter-planetary networks, etc. In this context, conventional routing schemes would fail.To deal with such networks researchers have suggested to use flooding-based routing schemes. While flooding-based schemes have a high probability of delivery, they waste a lot of energy and suffer from severe contention, which can significantly degrade their performance. Furthermore, proposed efforts to significantly reduce the overhead of flooding-based schemes have often be plagued by large delays. With this in mind, we introduce a new routing scheme, called Spray and Wait, that "sprays" a number of copies into the network, and then "waits" till one of these nodes meets the destination.Using theory and simulations we show that Spray and Wait outperforms all existing schemes with respect to both average message delivery delay and number of transmissions per message delivered; its overall performance is close to the optimal scheme. Furthermore, it is highly scalable retaining good performance under a large range of scenarios, unlike other schemes. Finally, it is simple to implement and to optimize in order to achieve given performance goals in practice.
2,712 citations