Machine Learning is the study of methods for programming computers to learn. Computers are applied to a wide range of tasks, and for most of these it is relatively easy for programmers to design and implement the necessary software. However, there are many tasks for which this is difficult or impossible. These can be divided into four general categories. First, there are problems for which there exist no human experts. For example, in modern automated manufacturing facilities, there is a need to predict machine failures before they occur by analyzing sensor readings. Because the machines are new, there are no human experts who can be interviewed by a programmer to provide the knowledge necessary to build a computer system. A machine learning system can study recorded data and subsequent machine failures and learn prediction rules. Second, there are problems where human experts exist, but where they are unable to explain their expertise. This is the case in many perceptual tasks, such as speech recognition, hand-writing recognition, and natural language understanding. Virtually all humans exhibit expert-level abilities on these tasks, but none of them can describe the detailed steps that they follow as they perform them. Fortunately, humans can provide machines with examples of the inputs and correct outputs for these tasks, so machine learning algorithms can learn to map the inputs to the outputs. Third, there are problems where phenomena are changing rapidly. In finance, for example, people would like to predict the future behavior of the stock market, of consumer purchases, or of exchange rates. These behaviors change frequently, so that even if a programmer could construct a good predictive computer program, it would need to be rewritten frequently. A learning program can relieve the programmer of this burden by constantly modifying and tuning a set of learned prediction rules. Fourth, there are applications that need to be customized for each computer user separately. Consider, for example, a program to filter unwanted electronic mail messages. Different users will need different filters. It is unreasonable to expect each user to program his or her own rules, and it is infeasible to provide every user with a software engineer to keep the rules up-to-date. A machine learning system can learn which mail messages the user rejects and maintain the filtering rules automatically. Machine learning addresses many of the same research questions as the fields of statistics, data mining, and psychology, but with differences of emphasis. Statistics focuses on understanding the phenomena that have generated the data, often with the goal of testing different hypotheses about those phenomena. Data mining seeks to find patterns in the data that are understandable by people. Psychological studies of human learning aspire to understand the mechanisms underlying the various learning behaviors exhibited by people (concept learning, skill acquisition, strategy change, etc.).

Machine learning

Statistics for Spatial Data.

This paper presents a comprehensive literature review on applications of deep reinforcement learning (DRL) in communications and networking. Modern networks, e.g., Internet of Things (IoT) and unmanned aerial vehicle (UAV) networks, become more decentralized and autonomous. In such networks, network entities need to make decisions locally to maximize the network performance under uncertainty of network environment. Reinforcement learning has been efficiently used to enable the network entities to obtain the optimal policy including, e.g., decisions or actions, given their states when the state and action spaces are small. However, in complex and large-scale networks, the state and action spaces are usually large, and the reinforcement learning may not be able to find the optimal policy in reasonable time. Therefore, DRL, a combination of reinforcement learning with deep learning, has been developed to overcome the shortcomings. In this survey, we first give a tutorial of DRL from fundamental concepts to advanced models. Then, we review DRL approaches proposed to address emerging issues in communications and networking. The issues include dynamic network access, data rate control, wireless caching, data offloading, network security, and connectivity preservation which are all important to next generation networks, such as 5G and beyond. Furthermore, we present applications of DRL for traffic routing, resource sharing, and data collection. Finally, we highlight important challenges, open issues, and future research directions of applying DRL.

Applications of Deep Reinforcement Learning in Communications and Networking: A Survey

In this chapter, you will see how the simplex method simplifies when it is applied to a class of optimization problems that are known as “network flow models.” You will also see that if a network flow model has “integer-valued data,” the simplex method finds an optimal solution that is integer-valued.

Flows in Networks

In this chapter we deal with the following nonlinear fractional programming problem: 

$$P:\mathop{{\max }}\limits_{{x \in s}} q(x) = (f(x) + \alpha )/((x) + \beta )$$

where f, g: R n → R, α, β ∈ R, S ⊆ R n . To simplify things, and without restricting the generality of the problem, it is usually assumed that, g(x) + β + 0 for all x ∈ S,S is non-empty and that the objective function has a finite optimal value.

Nonlinear Fractional Programming

Although TCP was originally designed to provide the reliable data transfer over the Internet, packet losses detected in TCP are considered as an indication of network congestion due to the high quality of data transmission provided by recent transmission technologies and media access control technologies. However, packet losses can be caused by transmission interruptions such as handoffs in the mobile networks and protection switching in the transport networks. These packet losses are bursty because the transmission interruptions continue for tens of miliseconds through several seconds. In this paper, we describe the experimental analysis of TCP behaviors by inserting errors such that all packets are lost during transmission interruptions. We have tested various TCP versions including those in Linux, that in Windows and that in Mac OS. This paper suggests (1) that the tested TCPs in Linux follow the similar procedure and retransmit lost packets quickly, (2) that TCP in Windows also behaves well but the increase of congestion window seems to be limited, and (3) that TCP in Mac OS has shown some problems in retransmitting contiguously lost packets. Keywords-TCP; Transmission Interruption; Bursty Packet Losses; Retransmission; SACK Based Loss Recovery.

Experimental Analysis of TCP Behaviors against Bursty Packet Losses Caused by Transmission Interruption

We propose a QoS supporting ad hoc network scheme which employs a hybrid approach utilizing both TDMA and 80211 DCF and present performance evaluation results of the scheme In the scheme, TDMA period provides contention free transmissions for QoS flows, and DCF period is used to provide contention-based access for best effort or low priority flows We evaluate the proposed scheme for various numbers of TCP flows and different CBR data rates with QualNet simulator Simulation results show that the protocol is able to provide an efficient solution for QoS control in ad hoc networks by combing TDMA and 80211 DCF

Performance evaluation of time slot based QoS aware ad hoc network scheme for CBR and TCP flows

Improving Traffic Load Distribution for Tor Network (情報ネットワーク)

In Vehicular Ad-Hoc Network (VANET), as a result of frequent changes of network topology caused by vehicle's movement, the general purpose ad hoc routing protocols such as AODV and DSR cannot work efficiently. This paper proposed a VANET routing protocol QLAODV which fits for unicast application in high mobility scenario. QLAODV is a distributed reinforcement learning routing protocol, which uses Q-Learning algorithm to infer network state information and uses unicast control packets checking the availability of paths in a real time manner in order to allow Q-Learning to work efficiently in highly dynamic network environment. In this paper, we show the performance analysis of QLAODV by simulation with NS2 in different mobility models, and give the simulation results confirming that QLAODV outperforms original AODV significantly in highly dynamic networks.

An enhanced reinforcement routing protocol for inter-vehicular unicast application

Edge computing can reduce service latency through task offloading. Since computational resources on the edge of the network are scarce, selecting a node with rich computational capability is the key for getting a high-quality service. In this paper, we propose a virtual edge scheme where a node can offload its tasks to a virtual edge node that consists of multiple vehicles in vicinity. The relative vehicle velocity and computational capability are considered in the virtual edge selection. We compare our proposed scheme with several baseline schemes and show the superiority of the scheme.

Celimuge Wu

Papers

Experimental Analysis of TCP Behaviors against Bursty Packet Losses Caused by Transmission Interruption

Performance evaluation of time slot based QoS aware ad hoc network scheme for CBR and TCP flows

Improving Traffic Load Distribution for Tor Network (情報ネットワーク)

An enhanced reinforcement routing protocol for inter-vehicular unicast application

Virtual Edge: Collaborative Computation Offloading in VANETs