Reinforcement learning, one of the most active research areas in artificial intelligence, is a computational approach to learning whereby an agent tries to maximize the total amount of reward it receives when interacting with a complex, uncertain environment. In Reinforcement Learning, Richard Sutton and Andrew Barto provide a clear and simple account of the key ideas and algorithms of reinforcement learning. Their discussion ranges from the history of the field's intellectual foundations to the most recent developments and applications. The only necessary mathematical background is familiarity with elementary concepts of probability. The book is divided into three parts. Part I defines the reinforcement learning problem in terms of Markov decision processes. Part II provides basic solution methods: dynamic programming, Monte Carlo methods, and temporal-difference learning. Part III presents a unified view of the solution methods and incorporates artificial neural networks, eligibility traces, and planning; the two final chapters present case studies and consider the future of reinforcement learning.

/pdf/reinforcement-learning-an-introduction-rzxgej9p17.pdf

Reinforcement Learning: An Introduction

Power Aware Routing in Mobile Ad Hoc Networks

With the rapid deployment of new wireless devices and applications, the last decade has witnessed a growing demand for wireless radio spectrum. However, the fixed spectrum assignment policy becomes a bottleneck for more efficient spectrum utilization, under which a great portion of the licensed spectrum is severely under-utilized. The inefficient usage of the limited spectrum resources urges the spectrum regulatory bodies to review their policy and start to seek for innovative communication technology that can exploit the wireless spectrum in a more intelligent and flexible way. The concept of cognitive radio is proposed to address the issue of spectrum efficiency and has been receiving an increasing attention in recent years, since it equips wireless users the capability to optimally adapt their operating parameters according to the interactions with the surrounding radio environment. There have been many significant developments in the past few years on cognitive radios. This paper surveys recent advances in research related to cognitive radios. The fundamentals of cognitive radio technology, architecture of a cognitive radio network and its applications are first introduced. The existing works in spectrum sensing are reviewed, and important issues in dynamic spectrum allocation and sharing are investigated in detail.

/pdf/advances-in-cognitive-radio-networks-a-survey-3gp2nvo27m.pdf

Advances in cognitive radio networks: A survey

This article provides a detailed discussion of wireless resource and channel allocation schemes. The authors provide a survey of a large number of published papers in the area of fixed, dynamic, and hybrid allocation schemes and compare their trade-offs in terms of complexity and performance. We also investigate these channel allocation schemes based on other factors such as distributed/centralized control and adaptability to traffic conditions. Moreover, we provide a detailed discussion on reuse partitioning schemes, the effect of handoffs, and prioritization schemes. Finally, we discuss other important issues in resource allocation such as overlay cells, frequency planning, and power control.

Channel assignment schemes for cellular mobile telecommunication systems: a comprehensive survey

A location reporting paging communication system comprising space satellites, ground stations and a remote receiving unit adapted to resolve a global position from signals transmitted from a communication transmitter. The subscriber in possession of the remote receiving unit updates the paging network with global positioning information. A caller paging a subscriber in possession of the remote receiving unit may request the global location of the remote receiving unit. The paging network could divulge or block such information from a caller depending on the requirements of the subscriber.

Location reporting satellite paging system with optional blocking of location reporting

Two novel channel-assignment strategies are proposed: the locally optimized dynamic assignment (LODA) strategy and the borrowing with directional channel-locking (BDCL) strategy. Their performance is compared with the fixed-assignment (FA) strategy (currently used on certain systems) and the borrowing with channel ordering (BCO) strategy (the strategy that has given the lowest blocking probability in previous research). Computer simulations on a 49-cell network for both uniform and nonuniform traffic showed that the average call-blocking probability of the BDCL strategy is always the lowest. The LODA performance is comparable with that of BCO under nonuniform traffic conditions but is inferior under uniform traffic conditions. >

Comparisons of channel-assignment strategies in cellular mobile telephone systems

A Markov chain analysis for spectrum access in licensed bands for cognitive radios is presented and forced termination probability, blocking probability and traffic throughput are derived. In addition, a channel reservation scheme for cognitive radio spectrum handoff is proposed. This scheme allows the tradeoff between forced termination and blocking according to QoS requirements. Numerical results show that the proposed scheme can greatly reduce forced termination probability at a slight increase in blocking probability

/pdf/analysis-of-cognitive-radio-spectrum-access-with-optimal-3bvhhzrst6.pdf

Analysis of Cognitive Radio Spectrum Access with Optimal Channel Reservation

Two new channel assignment strategies are proposed. They are the locally optimized dynamic assignment (LODA) strategy and the borrowing with directional channel locking (BDCL) strategy. Their performance is compared with the fixed assignment (FA) strategy (currently used on certain systems) and the borrowing with channel ordering (BCO) strategy (the stability that has given the lowest blocking probability in previous research). Computer simulations on a 49-cell network for both uniform and nonuniform traffic show that the average call-blocking probability of the BDCL strategy is always the lowest. The LODA performance is comparable with BCO under nonuniform traffic conditions but is inferior under uniform traffic conditions. >

Comparisons of channel assignment strategies in cellular mobile telephone systems

An optimal routing and data aggregation scheme for wireless sensor networks is proposed in this paper. The objective is to maximize the network lifetime by jointly optimizing data aggregation and routing. We adopt a model to integrate data aggregation with the underlying routing scheme and present a smoothing approximation function for the optimization problem. The necessary and sufficient conditions for achieving the optimality are derived and a distributed gradient algorithm is designed accordingly. We show that the proposed scheme can significantly reduce the data traffic and improve the network lifetime. The distributed algorithm can converge to the optimal value efficiently under all network configurations.

/pdf/optimal-routing-and-data-aggregation-for-maximizing-lifetime-3fze2efrbz.pdf

Optimal routing and data aggregation for maximizing lifetime of wireless sensor networks

An algorithm for allocating nominal channels according to traffic distribution is designed. The algorithm attempts to minimize the average blocking probability as nominal channels are allocated one at a time. Simulation results show that the system's traffic-carrying capacity can be increased by about 10% by the use of this algorithm, and that the gain is additional to the improvement obtained from the channel-borrowing strategies. If the effect of shadow blocking is considered in the assignment of channels, only a very small increase in the traffic capacity is observed. >

Tak-Shing Peter Yum

Papers

Comparisons of channel-assignment strategies in cellular mobile telephone systems

Analysis of Cognitive Radio Spectrum Access with Optimal Channel Reservation

Comparisons of channel assignment strategies in cellular mobile telephone systems

Optimal routing and data aggregation for maximizing lifetime of wireless sensor networks

The nonuniform compact pattern allocation algorithm cellular mobile systems