Functions of One Complex Variable

Functions of one complex variable II

Future wireless networks have a substantial potential in terms of supporting a broad range of complex compelling applications both in military and civilian fields, where the users are able to enjoy high-rate, low-latency, low-cost and reliable information services. Achieving this ambitious goal requires new radio techniques for adaptive learning and intelligent decision making because of the complex heterogeneous nature of the network structures and wireless services. Machine learning (ML) algorithms have great success in supporting big data analytics, efficient parameter estimation and interactive decision making. Hence, in this article, we review the thirty-year history of ML by elaborating on supervised learning, unsupervised learning, reinforcement learning and deep learning. Furthermore, we investigate their employment in the compelling applications of wireless networks, including heterogeneous networks (HetNets), cognitive radios (CR), Internet of Things (IoT), machine to machine networks (M2M), and so on. This article aims for assisting the readers in clarifying the motivation and methodology of the various ML algorithms, so as to invoke them for hitherto unexplored services as well as scenarios of future wireless networks.

/pdf/thirty-years-of-machine-learning-the-road-to-pareto-optimal-51ltzgc7ke.pdf

Thirty Years of Machine Learning: The Road to Pareto-Optimal Wireless Networks

To overcome the difficulties of charging the wireless sensors in the wild with conventional energy supply, more and more researchers have focused on the sensor networks with renewable generations. Considering the uncertainty of the renewable generations, an effective energy management strategy is necessary for the sensors. In this paper, we propose a novel energy management algorithm based on the reinforcement learning. By utilizing deep deterministic policy gradient (DDPG), the proposed algorithm is applicable for the continuous states and realizes the continuous energy management. We also propose a state normalization algorithm to help the neural network initialize and learn. With only one day’s real solar data and the simulative channel data for training, the proposed algorithm shows excellent performance in the validation with about 800 days length of real solar data. Compared with the state-of-the-art algorithms, the proposed algorithm achieves better performance in terms of long-term average net bit rate.

Deep Deterministic Policy Gradient (DDPG)-Based Energy Harvesting Wireless Communications

In this paper, we investigate the resource allocation problem for unmanned aerial vehicle (UAV)-assisted networks, where a UAV acting as an energy source provides radio frequency energy for multiple energy harvesting-powered device-to-device (D2D) pairs with much information to be transmitted. The goal is to maximize the average throughput within a time horizon while satisfying the energy causality constraint under a generalized harvest-transmit-store model, which results in a non-convex problem. By introducing the Lagrangian relaxation method, we analytically show that the behavior of all D2D pairs at each time slot is exclusive: harvesting energy or transmitting information signals. The formulated non-convex optimization problem is thus transformed into a mixed integer nonlinear programming (MINIP). We then design an efficient resource allocation algorithm to solve this MINIP, where D.C. (difference of two convex functions) programming and golden section method are combined to achieve a suboptimal solution. Furthermore, we provide an idea to reduce the computational complexity for facilitating the application in practice. Simulations are conducted to validate the effectiveness of the proposed algorithm and evaluate the system throughput performance.

Resource Allocation for Energy Harvesting-Powered D2D Communication Underlaying UAV-Assisted Networks

Energy harvesting point-to-point communications are considered. The transmitter harvests energy from the environment and stores it in a finite battery. It is assumed that the transmitter has always data to transmit and the harvested energy is used exclusively for data transmission. As in practical scenarios prior knowledge about the energy harvesting process might not be available, we assume that at each time instant only information about the current state of the transmitter is available, i.e., harvested energy, battery level and channel coefficient. We model the scenario as a Markov decision process and we implement reinforcement learning at the transmitter to find a power allocation policy that aims at maximizing the throughput. To overcome the limitations of traditional reinforcement learning algorithms, we apply the concept of function approximation and we propose a set of binary functions to approximate the expected throughput given the state of the transmitter. Numerical results show that the performance of the proposed approach, which requires only causal knowledge of the energy harvesting process and channel coefficients, has only a small degradation compared to the optimum case which requires perfect non-causal knowledge. Additionally, the proposed approach outperforms naive policies that assume only causal knowledge at the transmitter.

Reinforcement learning for energy harvesting point-to-point communications

Energy harvesting (EH) two-hop communications are considered. The transmitter and the relay harvest energy from the environment and use it exclusively for transmitting data. A data arrival process is assumed at the transmitter. At the relay, a finite data buffer is used to store the received data. We consider a realistic scenario in which the EH nodes have only local causal knowledge, i.e., at any time instant, each EH node only knows the current value of its EH process, channel state, and data arrival process. Our goal is to find a power allocation policy to maximize the throughput at the receiver. We show that because the EH nodes have local causal knowledge, the two-hop communication problem can be separated into two point-to-point problems. Consequently, independent power allocation problems are solved at each EH node. To find the power allocation policy, reinforcement learning with linear function approximation is applied. Moreover, to perform function approximation two feature functions which consider the data arrival process are introduced. Numerical results show that the proposed approach has only a small degradation as compared to the offline optimum case. Furthermore, we show that with the use of the proposed feature functions a better performance is achieved compared to standard approximation techniques.

Reinforcement Learning for Energy Harvesting Decode-and-Forward Two-Hop Communications

In this paper, we propose a novel algorithm to maximize the sum-rate in interference-limited scenarios where each user decodes its own message with the presence of unknown interferences and noise. The problem of adapting the transmit and receive filters of the users to maximize the sum-rate with a transmit power constraint is nonconvex. Our novel approach is to formulate the sum-rate maximization problem as an equivalent multiconvex optimization problem by adding two sets of auxiliary variables. An iterative algorithm, which alternatingly adjusts the system variables and the auxiliary variables is proposed to solve the multiconvex optimization problem and we show that the algorithm converges to a stationary point. The proposed algorithm is applied to a downlink cellular scenario consisting of several cells each of which contains a base station serving several mobile stations. We examine the two cases, with or without several half-duplex amplify-and-forward relays assisting the transmission. A sum power constraint at the base stations and at the relays are assumed. The applicability of our approach to the individual power constraints case is also shown. Finally, we show that the proposed multiconvex formulation of the sum-rate maximization problem is applicable to many other wireless systems in which the estimated data symbols are multiaffine functions of the system variables.

Maximizing the Sum Rate in Cellular Networks Using Multiconvex Optimization

This paper focuses on signal space interference alignment for the uplink and downlink transmissions in a cellular relay network with multiple cells where a single base station serves multiple mobile stations in each cell and several amplifyand-forward relays are deployed. We show that the interference alignment problems in the uplink and downlink transmissions are a pair of formally dual problems. Exploiting the reciprocity of the channels, a two-step procedure first nullifying the intercell interferences following the idea of relay-aided interference alignment and then designing a zero-forcing filter for each base station to suppress the intra-cell interferences is proposed to obtain the dual interference alignment solutions. Furthermore, the dual solutions also achieve the same sum rate in both the uplink and the reciprocal downlink transmissions with a sum transmit power constraint even if the power consumed by the relays and the relay retransmitted noises are considered.

Uplink-Downlink Duality of Interference Alignment in Cellular Relay Networks

In this paper, we investigate the feasibility conditions for relay-aided interference alignment in a class of partially connected networks. The considered scenario consists of several single-antenna communicating node pairs and multiple single-antenna amplify-and-forward relays. Some of the direct links between the source and the destination nodes may have zero gain. A two-hop transmission scheme is applied. The relays' scaling factors, along with the transmit and receive filters, are adapted to the channel to null the interference signals at every destination node. However, this would also null the desired useful signal at certain destinations if the number of relays is insufficient. To avoid this, the required number of relays is studied. We show that the required number of relays depends on the rank of the incidence matrix of a graph defined by the topology of the direct links.

Xiang Li

Papers

Reinforcement learning for energy harvesting point-to-point communications

Reinforcement Learning for Energy Harvesting Decode-and-Forward Two-Hop Communications

Maximizing the Sum Rate in Cellular Networks Using Multiconvex Optimization

Uplink-Downlink Duality of Interference Alignment in Cellular Relay Networks

Feasibility Conditions for Relay-Aided Interference Alignment in Partially Connected Networks