Machine Learning is the study of methods for programming computers to learn. Computers are applied to a wide range of tasks, and for most of these it is relatively easy for programmers to design and implement the necessary software. However, there are many tasks for which this is difficult or impossible. These can be divided into four general categories. First, there are problems for which there exist no human experts. For example, in modern automated manufacturing facilities, there is a need to predict machine failures before they occur by analyzing sensor readings. Because the machines are new, there are no human experts who can be interviewed by a programmer to provide the knowledge necessary to build a computer system. A machine learning system can study recorded data and subsequent machine failures and learn prediction rules. Second, there are problems where human experts exist, but where they are unable to explain their expertise. This is the case in many perceptual tasks, such as speech recognition, hand-writing recognition, and natural language understanding. Virtually all humans exhibit expert-level abilities on these tasks, but none of them can describe the detailed steps that they follow as they perform them. Fortunately, humans can provide machines with examples of the inputs and correct outputs for these tasks, so machine learning algorithms can learn to map the inputs to the outputs. Third, there are problems where phenomena are changing rapidly. In finance, for example, people would like to predict the future behavior of the stock market, of consumer purchases, or of exchange rates. These behaviors change frequently, so that even if a programmer could construct a good predictive computer program, it would need to be rewritten frequently. A learning program can relieve the programmer of this burden by constantly modifying and tuning a set of learned prediction rules. Fourth, there are applications that need to be customized for each computer user separately. Consider, for example, a program to filter unwanted electronic mail messages. Different users will need different filters. It is unreasonable to expect each user to program his or her own rules, and it is infeasible to provide every user with a software engineer to keep the rules up-to-date. A machine learning system can learn which mail messages the user rejects and maintain the filtering rules automatically. Machine learning addresses many of the same research questions as the fields of statistics, data mining, and psychology, but with differences of emphasis. Statistics focuses on understanding the phenomena that have generated the data, often with the goal of testing different hypotheses about those phenomena. Data mining seeks to find patterns in the data that are understandable by people. Psychological studies of human learning aspire to understand the mechanisms underlying the various learning behaviors exhibited by people (concept learning, skill acquisition, strategy change, etc.).

Machine learning

Statistics for Spatial Data.

This paper presents a comprehensive literature review on applications of deep reinforcement learning (DRL) in communications and networking. Modern networks, e.g., Internet of Things (IoT) and unmanned aerial vehicle (UAV) networks, become more decentralized and autonomous. In such networks, network entities need to make decisions locally to maximize the network performance under uncertainty of network environment. Reinforcement learning has been efficiently used to enable the network entities to obtain the optimal policy including, e.g., decisions or actions, given their states when the state and action spaces are small. However, in complex and large-scale networks, the state and action spaces are usually large, and the reinforcement learning may not be able to find the optimal policy in reasonable time. Therefore, DRL, a combination of reinforcement learning with deep learning, has been developed to overcome the shortcomings. In this survey, we first give a tutorial of DRL from fundamental concepts to advanced models. Then, we review DRL approaches proposed to address emerging issues in communications and networking. The issues include dynamic network access, data rate control, wireless caching, data offloading, network security, and connectivity preservation which are all important to next generation networks, such as 5G and beyond. Furthermore, we present applications of DRL for traffic routing, resource sharing, and data collection. Finally, we highlight important challenges, open issues, and future research directions of applying DRL.

Applications of Deep Reinforcement Learning in Communications and Networking: A Survey

In this chapter, you will see how the simplex method simplifies when it is applied to a class of optimization problems that are known as “network flow models.” You will also see that if a network flow model has “integer-valued data,” the simplex method finds an optimal solution that is integer-valued.

Flows in Networks

In this chapter we deal with the following nonlinear fractional programming problem: 

$$P:\mathop{{\max }}\limits_{{x \in s}} q(x) = (f(x) + \alpha )/((x) + \beta )$$

where f, g: R n → R, α, β ∈ R, S ⊆ R n . To simplify things, and without restricting the generality of the problem, it is usually assumed that, g(x) + β + 0 for all x ∈ S,S is non-empty and that the objective function has a finite optimal value.

Nonlinear Fractional Programming

In this paper, a distributed power and spectrum allocation scheme is proposed for Device-to-Device communication on unlicensed bands (D2D-U) enabled networks. To make full use of the spectrum resources on the unlicensed bands while guaranteeing the fairness among D2D-U links and the harmonious coexistence with WiFi networks, an online trained Neural network (NN) is first utilized on each D2D-U pair to determine the price to use the unlicensed channels according to the channel state and traffic loads. Then, a non-convex optimization problem can be formulated and solved on each D2D-U link to determine the optimal spectrum and power allocation scheme which can maximize the its transmission data rate. Numerical simulation results are demonstrated to verify the performance of the proposed method which enables each D2D-U link to maximize its own data-rate individually under the constraint of the fair coexistence with other D2D-U devices and WiFi networks.

Distributed Spectrum and Power Allocation for D2D-U Networks

Vehicular ad hoc networks (VANETs) provide a distributed approach for connecting vehicles on the move. VANETs have been attracting interest for their potential roles in Internet of things (IoT) applications including intelligent transportation systems, autonomous driving, and so on. In VANETs, a multi-hop communication protocol is required when the transmission range is not enough to cover all intended receivers. Many applications including collision warning systems and other value-added applications require inter-vehicle communications. However, due to the various vehicle densities, vehicle movement, limited bandwidth of wireless communications, it is challenging to provide a high packet reception ratio and low end-to-end delay. There have been many studies about the inter-vehicle communications. These studies cover all communication layers and various types of applications. In this paper, we survey the recent works on the inter-vehicle communications for emerging IoT applications, with emphasis on network layer protocols, and explain the corresponding challenges and research directions.

Connected vehicles and Internet of things

A Sequential Classifiers Combination Method to Reduce False Negative for Intrusion Detection System

Recent advancements in deep reinforcement learning (DRL) have led to its application in multi-agent scenarios to solve complex real-world problems, such as network resource allocation and sharing, network routing, and traffic signal controls. Multi-agent DRL (MADRL) enables multiple agents to interact with each other and with their operating environment, and learn without the need for external critics (or teachers), thereby solving complex problems. Significant performance enhancements brought about by the use of MADRL have been reported in multi-agent domains; for instance, it has been shown to provide higher quality of service (QoS) in network resource allocation and sharing. This paper presents a survey of MADRL models that have been proposed for various kinds of multi-agent domains, in a taxonomic approach that highlights various aspects of MADRL models and applications, including objectives, characteristics, challenges, applications, and performance measures. Furthermore, we present open issues and future directions of MADRL.

Applications of Multi-Agent Deep Reinforcement Learning: Models and Algorithms

We propose a vehicle-to-roadside communication protocol based on a distributed clustering algorithm where a coalitional game approach is used to stimulate the vehicles to join a cluster, and a fuzzy logic algorithm is employed to generate stable clusters by taking into account multiple metrics of vehicle velocity, moving pattern, and signal qualities between vehicles A reinforcement learning algorithm with game theory-based reward discount is employed to guide each vehicle to select the route which can maximize the whole network performance We conduct extensive computer simulations to show the performance advantage of the protocol over other approaches

Celimuge Wu

Papers

Distributed Spectrum and Power Allocation for D2D-U Networks

Connected vehicles and Internet of things

A Sequential Classifiers Combination Method to Reduce False Negative for Intrusion Detection System

Applications of Multi-Agent Deep Reinforcement Learning: Models and Algorithms

V2R Communication Protocol Based on Game Theory Inspired Clustering