Nowadays, many disruptive Internet-of-Things (IoT) applications emerge, such as augmented/virtual reality online games, autonomous driving, and smart everything, which are massive in number, data intensive, computation intensive, and delay sensitive. Due to the mismatch between the fifth generation (5G) and the requirements of such massive IoT-enabled applications, there is a need for technological advancements and evolutions for wireless communications and networking toward the sixth-generation (6G) networks. 6G is expected to deliver extended 5G capabilities at a very high level, such as Tbps data rate, sub-ms latency, cm-level localization, and so on, which will play a significant role in supporting massive IoT devices to operate seamlessly with highly diverse service requirements. Motivated by the aforementioned facts, in this article, we present a comprehensive survey on 6G-enabled massive IoT. First, we present the drivers and requirements by summarizing the emerging IoT-enabled applications and the corresponding requirements, along with the limitations of 5G. Second, visions of 6G are provided in terms of core technical requirements, use cases, and trends. Third, a new network architecture provided by 6G to enable massive IoT is introduced, i.e., space–air–ground–underwater/sea networks enhanced by edge computing. Fourth, some breakthrough technologies, such as machine learning and blockchain, in 6G are introduced, where the motivations, applications, and open issues of these technologies for massive IoT are summarized. Finally, a use case of fully autonomous driving is presented to show 6G supports massive IoT.

Enabling Massive IoT Toward 6G: A Comprehensive Survey

Multi-access edge computing (MEC) has recently been proposed to aid mobile end devices in providing compute- and data-intensive services with low latency. Growing service demands by the end devices may overwhelm MEC installations, while cost constraints limit the increases of the installed MEC computing and data storage capacities. At the same time, the ever increasing computation capabilities and storage capacities of mobile end devices are valuable resources that can be utilized to enhance the MEC. This article comprehensively surveys the topic area of device-enhanced MEC, i.e., mechanisms that jointly utilize the resources of the community of end devices and the installed MEC to provide services to end devices. We classify the device-enhanced MEC mechanisms into mechanisms for computation offloading and mechanisms for caching. We further subclassify the offloading and caching mechanisms according to the targeted performance goals, which include throughput maximization, latency minimization, energy conservation, utility maximization, and enhanced security. We identify the main limitations of the existing device-enhanced MEC mechanisms and outline future research directions.

/pdf/device-enhanced-mec-multi-access-edge-computing-mec-aided-by-b0wbx0l663.pdf

Device-Enhanced MEC: Multi-Access Edge Computing (MEC) Aided by End Device Computation and Caching: A Survey

The Internet of Things (IoT) extends the Internet connectivity into billions of IoT devices around the world, where the IoT devices collect and share information to reflect status of the physical world. The Autonomous Control System (ACS), on the other hand, performs control functions on the physical systems without external intervention over an extended period of time. The integration of IoT and ACS results in a new concept - autonomous IoT (AIoT). The sensors collect information on the system status, based on which the intelligent agents in the IoT devices as well as the Edge/Fog/Cloud servers make control decisions for the actuators to react. In order to achieve autonomy, a promising method is for the intelligent agents to leverage the techniques in the field of artificial intelligence, especially reinforcement learning (RL) and deep reinforcement learning (DRL) for decision making. In this paper, we first provide a tutorial of DRL, and then propose a general model for the applications of RL/DRL in AIoT. Next, a comprehensive survey of the state-of-art research on DRL for AIoT is presented, where the existing works are classified and summarized under the umbrella of the proposed general DRL model. Finally, the challenges and open issues for future research are identified.

Deep Reinforcement Learning for Autonomous Internet of Things: Model, Applications and Challenges

Cooperative communication technology has become a research hotspot in wireless sensor networks (WSNs) in recent years, and will become one of the key technologies for improving spectrum utilization in wireless communication systems in the future. It leverages cooperation among multiple relay nodes in the wireless network to realize path transmission sharing, thereby improving the system throughput. In this paper, we model the process of cooperative communications with relay selection in WSNs as a Markov decision process and propose DQ-RSS, a deep-reinforcement-learning-based relay selection scheme, in WSNs. In DQ-RSS, a deep-Q-network (DQN) is trained according to the outage probability and mutual information, and the optimal relay is selected from a plurality of relay nodes without the need for a network model or prior data. More specifically, we use DQN to process high-dimensional state spaces and accelerate the learning rate. We compare DQ-RSS with the Q-learning-based relay selection scheme and evaluate the network performance on the basis of three aspects: outage probability, system capacity, and energy consumption. Simulation results indicate that DQ-RSS can achieve better performance on these elements and save the convergence time compared with existing schemes.

Cooperative Communications With Relay Selection Based on Deep Reinforcement Learning in Wireless Sensor Networks

The incumbent Internet of Things suffers from poor scalability and elasticity exhibiting in communication, computing, caching and control (4Cs) problems. The recent advances in deep reinforcement learning (DRL) algorithms can potentially address the above problems of IoT systems. In this context, this paper provides a comprehensive survey that overviews DRL algorithms and discusses DRL-enabled IoT applications. In particular, we first briefly review the state-of-the-art DRL algorithms and present a comprehensive analysis on their advantages and challenges. We then discuss on applying DRL algorithms to a wide variety of IoT applications including smart grid, intelligent transportation systems, industrial IoT applications, mobile crowdsensing, and blockchain-empowered IoT. Meanwhile, the discussion of each IoT application domain is accompanied by an in-depth summary and comparison of DRL algorithms. Moreover, we highlight emerging challenges and outline future research directions in driving the further success of DRL in IoT applications.

Deep Reinforcement Learning for Internet of Things: A Comprehensive Survey

Internet of Underwater Things (IoUT) consists of numerous sensor nodes distributed in an underwater area for sensing, collecting, processing information, and sending related messages to the data processing center. However, the characteristics of the underwater environment will bring strict limitations on communication coverage and power scarcity to IoUT networks. Applying cooperative communications to IoUT networks can expand the communication range and alleviate power shortages. In this article, we investigate the cooperative communication problem in a power-limited cooperative IoUT system and propose a reinforcement learning-based underwater relay selection strategy. Specifically, we first determine the optimal transmit powers of the source node and the selected underwater relay to maximize the end-to-end signal-to-noise ratio of the system. Then, we formulate the underwater cooperative relaying process as a Markov process and apply reinforcement learning to obtain an effective underwater relay selection strategy. The simulation results show that the performance of the proposed scheme outperforms that of the equal transmit power settings under the same conditions. In addition, the proposed deep Q-network-based underwater relay selection strategy improves the communication efficiency compared with the Q-learning-based strategy, and the number of iterations needed for convergence can be effectively reduced.

Optimal Cooperative Relaying and Power Control for IoUT Networks With Reinforcement Learning

Due to the lack of resources in the low spectrum, long term evolution (LTE) in unlicensed spectrum (LTE-U) technology has been proposed to extend LTE to unlicensed spectrum. LTE-U undertakes the task of streaming data traffic for licensed spectrum, which can greatly enhance the capacity of the system. However, the introduction of LTE-U technology also gives rise to the problem of coexistence with Wi-Fi systems. In this paper, an LTE-U and Wi-Fi coexistence algorithm is proposed in multi-channel scenarios based on Q-learning. By taking the idea of alternately transferring data in LTE-U and Wi-Fi, the algorithm takes into account both the fairness and the performance of the system and optimizes the duty cycle. The simulation results show that the proposed algorithm can effectively improve the throughput of the system in the premise of ensuring fairness.

LTE-U and Wi-Fi Coexistence Algorithm Based on Q-Learning in Multi-Channel

The Active Queue Management (AQM) algorithm is one of most important research fields in network congestion control. To adjust the maximum dropping probability (maxp) according to the network situation the maxp calculation based on the RED algorithm is improved using the Q-learning algorithm, and a new algorithm, known as QRED (Q-learning RED), is proposed. The self-adaptive adjustment for the maxp is achieved using the QRED algorithm and the queue length stability in a dynamic network environment is realized. In addition, the QRED algorithm not only avoids the sensitivity of the RED algorithm parameters, but also adapts the packet loss rate according to the specific network service type. Results based on the NS2 simulation show that the QRED algorithm has better stability in complex network environments, and hence, are superior to the RED active queue management algorithm.

/pdf/qred-a-q-learning-based-active-queue-management-scheme-3t0hvti23o.pdf

QRED: A Q-Learning-based Active Queue Management Scheme

Next-generation (5G) vehicular networks will support various network applications, leading to specific requirements and challenges for wireless access technologies. This trend has motivated the development of the long-term evolution-vehicle (LTE-V) network, a 5G cellular-based vehicular technology. Due to the limited bandwidth for vehicular communications, it is important to efficiently utilize slim spectrum resources in vehicular networks. In this paper, we introduce a cloud radio access network (C-RAN)-based vehicular network architecture, named C-VRAN, which facilitates efficient management and centralized processing of vehicular networks. Furthermore, we propose a discrete cosine transform (DCT)-based data compression scheme for C-VRAN to enhance the effective data rate of the fronthaul network. This scheme first uses DCT to perform time-frequency conversion of LTE-V I/Q data and then utilizes the Lloyd-Max algorithm to quantify data in the frequency domain before finally selecting an appropriate coding scheme to achieve better performance. Simulation results show that the proposed scheme can achieve 3 times compression ratio within 1% error vector amplitude distortion, and it also has strong independence and versatility, allowing it to be used as a standalone module for the current LTE-V system.

Yuhan Su

Papers

Cooperative Communications With Relay Selection Based on Deep Reinforcement Learning in Wireless Sensor Networks

Optimal Cooperative Relaying and Power Control for IoUT Networks With Reinforcement Learning

LTE-U and Wi-Fi Coexistence Algorithm Based on Q-Learning in Multi-Channel

QRED: A Q-Learning-based Active Queue Management Scheme

A Novel DCT-Based Compression Scheme for 5G Vehicular Networks