The end-to-end quality of service (QoS) and quality of experience (QoE) guarantee is quite important for network optimization. The current 5G and conceived 6G network in the future with ultra high density, bandwidth, mobility and large scale brings urgent requirement of high efficient end-to-end optimization methods. The conventional network optimization methods without learning and intelligent decision ability are hard to handle the high complexity and dynamic scenarios of 6G. Recently, machine learning based QoS and QoE aware network optimization algorithms emerge as a hot research area and attract much attention, which is widely acknowledged as the potential solution for end-to-end optimization in 6G. However, there are still many critical issues of employing machine learning in networks, especially in 6G. In this paper, we give a comprehensive survey on the recent machine learning based network optimization methods to guarantee the end-to-end QoS and QoE. To easy to follow, we introduce the investigated works following the end-to-end transmission flow from network access, routing to network congestion control and adaptive steaming control. Then we discuss some open issues and potential future research directions.

Survey on Machine Learning for Intelligent End-to-End Communication Toward 6G: From Network Access, Routing to Traffic Control and Streaming Adaption

The growing network density and unprecedented increase in network traffic, caused by the massively expanding number of connected devices and online services, require intelligent network operations. Machine Learning (ML) has been applied in this regard in different types of networks and networking technologies to meet the requirements of future communicating devices and services. In this article, we provide a detailed account of current research on the application of ML in communication networks and shed light on future research challenges. Research on the application of ML in communication networks is described in: i) the three layers, i.e., physical, access, and network layers; and ii) novel computing and networking concepts such as Multi-access Edge Computing (MEC), Software Defined Networking (SDN), Network Functions Virtualization (NFV), and a brief overview of ML-based network security. Important future research challenges are identified and presented to help stir further research in key areas in this direction.

/pdf/machine-learning-meets-communication-networks-current-trends-2iwfl5x5zg.pdf

Machine Learning Meets Communication Networks: Current Trends and Future Challenges

In conventional networks, there was a tight bond between the control plane and the data plane. The introduction of Software-Defined Networking (SDN) separated these planes, and provided additional features and tools to solve some of the problems of traditional network (i.e., latency, consistency, efficiency). SDN is a flexible networking paradigm that boosts network control, programmability and automation. It proffers many benefits in many areas, including routing. More specifically, for efficiently organizing, managing and optimizing routing in networks, some intelligence is required, and SDN offers the possibility to easily integrate it. To this purpose, many researchers implemented different machine learning (ML) techniques to enhance SDN routing applications. This article surveys the use of ML techniques for routing optimization in SDN based on three core categories (i.e. supervised learning, unsupervised learning, and reinforcement learning). The main contributions of this survey are threefold. Firstly, it presents detailed summary tables related to these studies and their comparison is also discussed, including a summary of the best works according to our analysis. Secondly, it summarizes the main findings, best works and missing aspects, and it includes a quick guideline to choose the best ML technique in this field (based on available resources and objectives). Finally, it provides specific future research directions divided into six sections to conclude the survey. Our conclusion is that there is a huge trend to use intelligence-based routing in programmable networks, particularly during the last three years, but a lot of effort is still required to achieve comprehensive comparisons and synergies of approaches, meaningful evaluations based on open datasets and topologies, and detailed practical implementations (following recent standards) that could be adopted by industry. In summary, future efforts should be focused on reproducible research rather than on new isolated ideas. Otherwise, most of these applications will be barely implemented in practice.

/pdf/a-survey-on-machine-learning-techniques-for-routing-1cfrotoops.pdf

A Survey on Machine Learning Techniques for Routing Optimization in SDN

Wireless- and 5G-enabled industrial automation is expected to include a plethora of different applications with a wide variety of requirements. In this article, evaluations are undertaken for the deployment of 5G in realistic industrial production environments with realistic deployment settings. Both deployments using commercial 5G systems and a 5G prototype system including pre-commercial and standard compliant URLLC functionality have been investigated. Systematic latency and reliability measurements were performed, over the air and in live networks, for different packet sizes, different devices, and networks with different capabilities (at different sites) to characterize the expected performance. The results indicate that today’s 5G latency performance significantly depends on packet size, transmission direction (uplink or downlink), and network configuration as well as on the end device’s design and capabilities. Our over-the-air measurements also empirically show that 5G technology and future networks have the capability of providing one-way latency of around 1 ms in both uplink and downlink for the various packet sizes tested. It is concluded that the requirements for very low latencies can be achieved with high reliability guarantees, as required in some of the most stringent industrial IoT applications.

https://www.mdpi.com/2079-9292/11/3/412/pdf?version=1643441596

Performance of 5G Trials for Industrial Automation

Network softwarization has recently been enabled via the software-defined networking (SDN) paradigm, which separates the data plane from control plane allowing for a flexible and centralized control of networks. This separation facilitates implementation of machine learning techniques for network management and optimization. In this work, a machine learning-based multipath routing (MLMR) framework is proposed for software-defined networks with quality-of-service (QoS) constraints and flow rules space constraints. The QoS-aware multipath routing problem in SDN is modeled as multicommodity network flow problem with side constraints, that is known to be NP-hard. The proposed framework utilizes network status estimates, and their corresponding routing configurations available at the network central controller to learn a mapping function between them. Once the mapping function is learned, it is applied on live-inputs of network status and routing requests to predict a multipath routing solutions in real-time. Performance evaluations of the MLMR framework on real traces of network traffic verify its accuracy and resilience to noise in training data. Furthermore, the MLMR framework demonstrates more than 98.99% improvement in computational efficiency.

Machine Learning-Based Multipath Routing for Software Defined Networks

A 5G campus network is a 5G network for the users affiliated with the campus organization, e.g., an industrial campus, covering a prescribed geographical area. A 5G campus network can operate as a so-called 5G non-standalone (NSA) network (which requires 4G Long-Term Evolution (LTE) spectrum access) or as a 5G standalone (SA) network (without 4G LTE spectrum access). 5G campus networks are envisioned to enable new use cases, which require cyclic delay-sensitive industrial communication, such as robot control. We design a rigorous testbed for measuring the one-way packet delays between a 5G end device via a radio access network (RAN) to a packet core with sub-microsecond precision as well as for measuring the packet core delay with nanosecond precision. With our testbed design, we conduct detailed measurements of the one-way download (downstream, i.e., core to end device) as well as one-way upload (upstream, i.e., end device to core) packet delays and losses for both 5G SA and 5G NSA hardware and network operation. We also measure the corresponding 5G SA and 5G NSA packet core processing delays for download and upload. We find that typically 95% of the SA download packet delays are in the range from 4–10 ms, indicating a fairly wide spread of the packet delays. Also, existing packet core implementations regularly incur packet processing latencies up to 0.4 ms, with outliers above one millisecond. Our measurement results inform the further development and refinement of 5G SA and 5G NSA campus networks for industrial use cases. We make the measurement data traces publicly available as the IEEE DataPort 5G Campus Networks: Measurement Traces dataset (DOI 10.21227/xe3c-e968).

/pdf/5g-campus-networks-a-first-measurement-study-51v05sz56d.pdf

5G Campus Networks: A First Measurement Study

Flow routing can achieve fine-grained network performance optimizations by routing distinct packet traffic flows over different network paths. While the centralized control of Software-Defined Networking (SDN) provides a control framework for implementing centralized network optimizations, e.g., optimized flow routing, the implementation of flow routing that is adaptive to varying traffic loads requires complex models. The goal of this study is to pursue a model-free approach that is based on reinforcement learning. We design and evaluate QR-SDN, a classical tabular reinforcement learning approach that directly represents the routing paths of individual flows in its state-action space. Due to the direct representation of flow routes in the QR-SDN state-action space, QR-SDN is the first reinforcement learning SDN routing approach to enable multiple routing paths between a given source (ingress) switch–destination (egress) switch pair while preserving the flow integrity. That is, in QR-SDN, packets of a given flow take the same routing path, while different flows with the same source-destination switch pair may take different routes (in contrast, the recent DRL-TE approach splits a given flow on a per-packet basis incurring high complexity and out-of-order packets). We implemented QR-SDN in a Software-Defined Network (SDN) emulation testbed. Our evaluations demonstrate that the flow-preserving multi-path routing of QR-SDN achieves substantially lower flow latencies than prior routing approaches that determine only a single source-destination route. A limitation of QR-SDN is that the state-action space grows exponentially with the number of network nodes. Addressing the scalability of direct flow routing, e.g., through routing only high-rate flows, is an important direction for future research. The QR-SDN code is made publicly available to support this future research.

/pdf/qr-sdn-towards-reinforcement-learning-states-actions-and-kczj6ssudt.pdf

QR-SDN: Towards Reinforcement Learning States, Actions, and Rewards for Direct Flow Routing in Software-Defined Networks

In this chapter, we extend the ComNetsEmu environment with a Reinforcement Learning (RL)-based routing algorithm inside the SDN controller. The RL agent tries to minimize the average flow latency by evaluating different routing configurations. In this way, multiple paths of a network are used for communication instead of relying only on one shortest path like state-of-the-art protocols. This is only possible due to the centralized control in SDN, since the controller can gather network statistics such as latency and globally modify the routing. First, we explain how RL can be applied to routing. Afterwards, we demonstrate its usage in a basic scenario.

Machine learning for routing

Mobile robots are an essential part of the vision of flexible production in a smart factory. To monitor and connect the robots, reliable and low latency communication is necessary. In this work, we conduct packet-based active measurements to evaluate the performance of a state-of-the-art 5G standalone system in a production environment. The focus is on whether 5G connections can meet the requirements specified by 3G PP in terms of delay and reliability. The results indicate that without cross-traffic, the requirement of a delay of less than 10 ms for 99.9 % of the packets can be met for the remote control and fleet management of mobile robots. However, as soon as cross-traffic is injected, especially in the uplink, the upper percentiles of the delay increase significantly, thus failing to hold the reliability requirements.

Private 5G Solutions for Mobile Industrial Robots: A Feasibility Study

Simultaneous Localization and Mapping (SLAM), among other critical functions of mobile robots, such as navigation, are computationally expensive. When deployed at the robot, those functions demand high energy consumption and result in shorter operation time. Offloading SLAM to an Edge Cloud (EC) can significantly reduce the robot’s computing demand and resources, subsequently reducing energy consumption. We offload intelligence of mobile robot control functionality, i.e., navigation, localization, and control to an EC. The EC processes sensor data and sends the robot the directional velocities. Meanwhile, a 5G wireless connection ensures the necessary low latencies and high throughputs. We demonstrate the feasibility of offloading SLAM and navigation in an EC based on a use case in automotive production. Additionally, we developed a digital twin of the robot and visualized its current sensor data.

Peter Sossalla

Papers

5G Campus Networks: A First Measurement Study

QR-SDN: Towards Reinforcement Learning States, Actions, and Rewards for Direct Flow Routing in Software-Defined Networks

Machine learning for routing

Private 5G Solutions for Mobile Industrial Robots: A Feasibility Study

Offloading Robot Control with 5G