We solve the adaptive call admission control (CAC) problem in multimedia networks via reinforcement learning (RL). The problem requires that network revenue be maximized while simultaneously meeting quality of service (QoS) constraints that forbid entry into certain states and use of certain actions. We show that RL provides a solution to this constrained semi-Markov decision problem and is able to earn significantly higher revenues than alternative heuristics. Unlike other model-based algorithms, RL does not require the explicit state transition models to solve the decision problems. This feature is very important if one considers large integrated service networks supporting a number of different service types, where the number of states is so large that model-based optimization algorithms are infeasible. Both packet-level and call-level QoS constraints are addressed, and both conservative and aggressive approaches to the QoS constraints are considered. Results are demonstrated on a single link and extended to routing on a multilink network.

Adaptive call admission control under quality of service constraints: a reinforcement learning solution

In ATM networks, a virtual path (VP) concept has been proposed to simplify traffic control and resource management. As a consequence, cell routing becomes more flexible and significant call setup processing can be reduced when resources are reserved on VP's. In this paper, we study the routing problem in homogeneous VP-based ATM networks in which VP's are used for traffic segregation such that all VC's in a VP have the same traffic characteristics and QOS requirement. Four adaptive routing algorithms based on the least loaded routing (LLR) concept are designed and evaluated. Our simulation results show that network blocking probability can be significantly reduced by LLR routing.

LLR routing in homogeneous VP-based ATM networks

In QoS routing, paths for flows are selected based on knowledge of resource availability at network nodes and the QoS requirements of flows. Several QoS routing schemes have been proposed that differ in the way they gather information about the network state and select paths based on this information. We broadly categorize these schemes into best path routing and proportional routing. The best path routing schemes gather global network state information and always select the best path for an incoming I-low,based on this global view. It has been shown that best path routing schemes require frequent exchange of network state, imposing both communication overhead on the network and processing overheads on the core routers. On the other hand, proportional routing schemes proportion incoming flows among a set of candidate paths. We have shown that it is possible to compute near-optimal proportions using only locally collected information. Furthermore, a few good candidate paths can be selected using infrequently exchanged global information and thus with minimal communication overhead. We describe these schemes in detail and demonstrate that proportional routing schemes can achieve higher throughput with lower overhead than best path routing schemes.

/pdf/a-localized-adaptive-proportioning-approach-to-qos-routing-21narbxmyv.pdf

A localized adaptive proportioning approach to QoS routing

https://www.netlab.tkk.fi/~esa/pub/files/hyytia-peva-2011-author.pdf

M/M/1-PS queue and size-aware task assignment

This paper proposes a hierarchical approach to the routing issue in ATM-based broadband multi-service networks. We show how distributed route selection algorithms embedded within network nodes may be managed by a TMN system, by virtue of a management service called load balancing, to increase their adaptivity to network wide conditions and therefore their effectiveness. The paper elaborates on the issues involved, presents specific algorithms and a TMN-compliant management architecture. The proposed management service influences the routing decisions by conveying network-wide information and contributes to the network load balancing by directing the routing algorithms to route traffic to the least congested network areas. Results regarding the effect of the proposed management system on the network operation are also presented.

A Management System for Load Balancing through Adaptive Routing in Multi-Service ATM Networks

The nature of the traffic-routing problem is described, and early studies of state-dependent routing are noted. A state-dependent scheme seeks to route each call so as to minimize the risk of blocking future calls, and thus responds to the current state of the network on the basis of certain assumptions about future traffic demands. State-dependent routing is considered as a Markov decision process. How the relative costs can be determined for the case of direct routing is shown. >

K.R. Krishnan

Papers

Markov decision algorithms for dynamic routing (telephone networks)