Software-Defined Networking (SDN) adapts logically-centralized control by decoupling control plane from data plane and provides the efficient use of network resources. However, due to the limitation of traditional routing strategies relying on manual configuration, SDN may suffer from link congestion and inefficient bandwidth allocation among flows, which could degrade network performance significantly. In this paper, we propose EARS, an intelligence-driven experiential network architecture for automatic routing. EARS adapts deep reinforcement learning (DRL) to simulate the human methods of learning experiential knowledge, employs the closed-loop network control mechanism incorporating with network monitoring technologies to realize the interaction with network environment. The proposed EARS can learn to make better control decision from its own experience by interacting with network environment and optimize the network intelligently by adjusting services and resources offered based on network requirements and environmental conditions. Under the network architecture, we design the network utility function with throughput and delay awareness, differentiate flows based on their size characteristics, and design a DDPG-based automatic routing algorithm as DRL decision brain to find the near-optimal paths for mice and elephant flows. To validate the network architecture, we implement it on a real network environment. Extensive simulation results show that EARS significantly improve the network throughput and reduces the average packet delay in comparison with baseline schemes (e.g. OSPF, ECMP).

EARS: Intelligence-driven experiential network architecture for automatic routing in software-defined networking

Due to the rapid development of network communication technology and the significant increase in network terminal equipment, the application of new network architecture software-defined networking (SDN) combined with reinforcement learning in network traffic scheduling has become an important focus of research. Because of network traffic transmission variability and complexity, the traditional reinforcement-learning algorithms in SDN face problems such as slow convergence rates and unbalanced loads. The problems seriously affect network performance, resulting in network link congestion and the low efficiency of inter-stream bandwidth allocation. This paper proposes an automatic load-balancing architecture based on reinforcement learning (ALBRL) in SDN. In this architecture, we design a load-balancing optimization model in high-load traffic scenarios and adapt the improved Deep Deterministic Policy Gradient (DDPG) algorithm to find a near-optimal path between network hosts. The proposed ALBRL uses the sampling method of updating the experience pool with the SumTree structure to improve the random extraction strategy of the empirical-playback mechanism in DDPG. It extracts a more meaningful experience for network updating with greater probability, which can effectively improve the convergence rate. The experiment results show that the proposed ALBRL has a faster training speed than existing reinforcement-learning algorithms and significantly improves network throughput.

/pdf/albrl-automatic-load-balancing-architecture-based-on-8em8up9z.pdf

ALBRL: Automatic Load-Balancing Architecture Based on Reinforcement Learning in Software-Defined Networking

Optical transport networks (OTNs) are widely used in backbone- and metro-area transmission networks to increase network transmission capacity. In the OTN, it is particularly crucial to rationally allocate routes and maximize network capacities. By employing deep reinforcement learning (DRL)- and software-defined networking (SDN)-based solutions, the capacity of optical networks can be effectively increased. However, because most DRL-based routing optimization methods have low sample usage and difficulty in coping with sudden network connectivity changes, converging in software-defined OTN scenarios is challenging. Additionally, the generalization ability of these methods is weak. This paper proposes an ensembles- and message-passing neural-network-based Deep Q-Network (EMDQN) method for optical network routing optimization to address this problem. To effectively explore the environment and improve agent performance, the multiple EMDQN agents select actions based on the highest upper-confidence bounds. Furthermore, the EMDQN agent captures the network’s spatial feature information using a message passing neural network (MPNN)-based DRL policy network, which enables the DRL agent to have generalization capability. The experimental results show that the EMDQN algorithm proposed in this paper performs better in terms of convergence. EMDQN effectively improves the throughput rate and link utilization of optical networks and has better generalization capabilities.

https://www.mdpi.com/1424-8220/22/21/8139/pdf?version=1666618411

A Routing Optimization Method for Software-Defined Optical Transport Networks Based on Ensembles and Reinforcement Learning

Software defined network (SDN) is a new paradigm in network engineering, where control plane and data plane are separated. Data plane is carried out on each node, while a control plane is centrally located. In conventional networks, the planes are implemented in the firmware of the router. In this paper, we implemented multimedia services on SDN using exterior and interior routing protocols, such as open shortest path first (OSPF), border gateway protocol (BGP) and routing information protocol (RIP), measured the quality of service (QoS), and analyze the measurement results. The test results showed that the greater the background traffic provided, the smaller the throughput. Traffic on the network will be congested, so the available bandwidth is also increasingly used up so that the number of bits sent every second decrease. Mean opinion score (MOS) test results are 4 showing good categories based on ITU-T standards.

https://www.beei.org/index.php/EEI/article/download/2389/2028

Comparison of routing protocol performance on multimedia services on software defined network

Due to vehicle mobility, the topology of Vehicle Ad-hoc Networks (VANETs) may change dynamically. High mobility, limited bandwidth, and dynamic network topology pose challenges for communication in the Internet of Vehicles (IoVs). Literature works have attempted to promote efficient (e.g., lower end-to-end latency) message forwarding. However, due to the uncertain direction of message forwarding and vehicle mobility, they suffer from unreachable destinations and unstable connections. This paper explores the efficient method of message forwarding to alleviate network congestion in IoVs. We propose a Spatio-Temporal domain Autonomous Load Balancing (STALB) routing protocol. Specifically, STALB is a trajectory-based method for controlling the direction of message forwarding. STALB can significantly reduce the end-to-end latency and overload ratio, since it considers the local status of network relay devices (i.e., buffer score, congestion status) from the spatio-temporal domain. Then, we present a path reconstruction mechanism, which ensures that messages are forwarded to destinations within limited Time-To-live (TTL). Extensive simulation results show that STALB significantly outperforms other baseline methods (BSaW, TDOR, and TBHGR) regarding overhead ratio, average delivery latency, and average buffer time. Especially, the delivery rate of STALB can reach 99.9% under the sparse network scenario (4,500 messages), at least 0.7% higher than other baseline methods. Similarly, the average delivery delay of STALB is at least 84.31% lower than that of other baseline methods under the dense network scenario (18,000 messages).

STALB: A Spatio-Temporal Domain Autonomous Load Balancing Routing Protocol

Software Defined Network (SDN) for large-scale IP provider network is an open issue and different solutions were proposed. However, the hybrid IP networks in which both distributed and centralized approach provide centralization of SDN and reliability of distributed networks. The common approach in which SDN controls the prioritized traffic and OSPF (Open Shortest Path First) guarantees the operation of traffic. In this research, we propose the SDN segregation, which maintain central management over dispersed routing control. A given topology is split in some fields with OpenFlow enabled switches as in between nodes. OSPF enabled router triggered updates to other routers in other field via SDN switches. The centralized controller defines how two OSPF routers observe each other. There will be a tradeoff between central control of SDN and fault tolerance capability of OSPF. As we increase SDN nodes control will increase and fault tolerance capacity of overall network decreases. The novelty of research work for balanced topology segregation also offers the models for network management. To show the enhancement provided by hybrid network over routing protocol deployment we have deployed separate test beds for routing protocol and proposed hybrid network.

Performance analysis of OSPF and hybrid networks

To fulfill the extensive increase in the capacity and quality of multiple users, 5G next generation cellular network system is expected to be in the form of heterogeneous radio access networks (Het-RAN) [1]. It is expected to encompass small cells, known as macro, pico or femto cells. For such dense deployment, network operators have preferably deduced the mechanism of cloud networks called C-RAN due to their highly optimized operational efficiency and cost effectiveness. A centralized operating system which can easily be integrated with existing 4G/LTE networks is main aim of mobile phone operators. This approach reduces remarkably front haul overheads in C-RANs [2]. To expand the architecture of C-RAN, large number of antennas at access end majorly contributes to increase the network capacity either by improving the spectrum bandwidth or by increasing the number of active users at a specific time. Future 5G wireless and mobile communication operating systems require high transmission data rates and large channel bandwidth. Therefore, they can adopt millimeter-wave (MMW) band (30–300 GHz) as main operating frequencies to fulfill the requirements. In this paper, we focused on 60 GHz MMW frequency band. The potential benefits are, it is license free, large bandwidth ∼2 GHz, high oxygen attenuation that supports high frequency reuse and large number of components integration on small area (Massive MIMO). Moreover, we designed, simulated and optimized novel antennas suitable for implementing in the wireless devices of CRAN networks. In addition to compact, low cost, light weight, and ease of integration, the proposed antennas exhibit wide bandwidth above 4 GHz with an average gain values more than 12.5 dBi.

Ali Ahmad

Papers

Performance analysis of OSPF and hybrid networks

Art of antenna designing for 5G (mmWave) next generation networks