This review provides a pedagogic and self-contained introduction to master equations and to their representation by path integrals. Since the 1930s, master equations have served as a fundamental tool to understand the role of fluctuations in complex biological, chemical, and physical systems. Despite their simple appearance, analyses of master equations most often rely on low-noise approximations such as the Kramers-Moyal or the system size expansion, or require ad-hoc closure schemes for the derivation of low-order moment equations. We focus on numerical and analytical methods going beyond the low-noise limit and provide a unified framework for the study of master equations. After deriving the forward and backward master equations from the Chapman-Kolmogorov equation, we show how the two master equations can be cast into either of four linear partial differential equations (PDEs). Three of these PDEs are discussed in detail. The first PDE governs the time evolution of a generalized probability generating function whose basis depends on the stochastic process under consideration. Spectral methods, WKB approximations, and a variational approach have been proposed for the analysis of the PDE. The second PDE is novel and is obeyed by a distribution that is marginalized over an initial state. It proves useful for the computation of mean extinction times. The third PDE describes the time evolution of a 'generating functional', which generalizes the so-called Poisson representation. Subsequently, the solutions of the PDEs are expressed in terms of two path integrals: a 'forward' and a 'backward' path integral. Combined with inverse transformations, one obtains two distinct path integral representations of the conditional probability distribution solving the master equations. We exemplify both path integrals in analysing elementary chemical reactions. Moreover, we show how a well-known path integral representation of averaged observables can be recovered from them. Upon expanding the forward and the backward path integrals around stationary paths, we then discuss and extend a recent method for the computation of rare event probabilities. Besides, we also derive path integral representations for processes with continuous state spaces whose forward and backward master equations admit Kramers-Moyal expansions. A truncation of the backward expansion at the level of a diffusion approximation recovers a classic path integral representation of the (backward) Fokker-Planck equation. One can rewrite this path integral in terms of an Onsager-Machlup function and, for purely diffusive Brownian motion, it simplifies to the path integral of Wiener. To make this review accessible to a broad community, we have used the language of probability theory rather than quantum (field) theory and do not assume any knowledge of the latter. The probabilistic structures underpinning various technical concepts, such as coherent states, the Doi-shift, and normal-ordered observables, are thereby made explicit.

Master equations and the theory of stochastic path integrals

With the advent of the fifth generation of wireless standards and an increasing demand for higher throughput, methods to improve spectral efficiency of wireless systems have become very important. In the context of cognitive radio, a substantial increase in throughput is possible if the secondary user can make smart decisions regarding which channel to sense and when or how often to sense. Here, we propose an algorithm to not only select a channel for data transmission, but also to predict how long the channel will remain unoccupied so that the time spent on channel sensing can be minimized. Our algorithm learns in two stages—a reinforcement learning approach for channel selection and a Bayesian approach to determine the duration for which sensing can be skipped. Comparisons with other methods are provided through extensive simulations. We show that the number of sensing operations is minimized with negligible increase in primary user interference; this implies that less energy is spent by the secondary user in sensing, and also higher throughput is achieved by saving the time spent on sensing.

Spectrum Access In Cognitive Radio Using a Two-Stage Reinforcement Learning Approach

This review provides a pedagogic and self-contained introduction to master equations and to their representation by path integrals. We discuss analytical and numerical methods for the solution of master equations, keeping our focus on methods that are applicable even when stochastic fluctuations are strong. The reviewed methods include the generating function technique and the Poisson representation, as well as novel ways of mapping the forward and backward master equations onto linear partial differential equations (PDEs). Spectral methods, WKB approximations, and a variational approach have been proposed for the analysis of the PDE obeyed by the generating function. After outlining these methods, we solve the derived PDEs in terms of two path integrals. The path integrals provide distinct exact representations of the conditional probability distribution solving the master equations. We exemplify both path integrals in analysing elementary chemical reactions. Furthermore, we review a method for the approximation of rare event probabilities and derive path integral representations of Fokker-Planck equations. To make our review accessible to a broad community, we have used the language of probability theory rather than quantum (field) theory. The probabilistic structures underpinning various technical concepts, such as coherent states, the Doi-shift, and normal-ordered observables, are thereby made explicit.

Based on a coherent state representation of noise operator and an ensemble averaging procedure using Wigner canonical thermal distribution for harmonic oscillators, a generalized quantum Langevin equation has been recently developed [Phys. Rev. E 65, 021109 (2002); 66, 051106 (2002)] to derive the equations of motion for probability distribution functions in c-number phase-space. We extend the treatment to explore several systematic approximation schemes for the solutions of the Langevin equation for nonlinear potentials for a wide range of noise correlation, strength and temperature down to the vacuum limit. The method is exemplified by an analytic application to harmonic oscillator for arbitrary memory kernel and with the help of a numerical calculation of barrier crossing, in a cubic potential to demonstrate the quantum Kramers’ turnover and the quantum Arrhenius plot.

Solution of quantum Langevin equation: Approximations, theoretical and numerical aspects

IEEE 802.11ax-2019 will replace both IEEE 802.11n-2009 and IEEE 802.11ac-2013 as the next high-throughput Wireless Local Area Network (WLAN) amendment. In this paper, we review the expected future WLAN scenarios and use-cases that justify the push for a new PHY/MAC IEEE 802.11 amendment. After that, we overview a set of new technical features that may be included in the IEEE 802.11ax-2019 amendment and describe both their advantages and drawbacks. Finally, we discuss some of the network-level functionalities that are required to fully improve the user experience in next-generation WLANs and note their relation with other on-going IEEE 802.11 amendments.

/pdf/ieee-802-11ax-high-efficiency-wlans-57ejajfm5d.pdf

IEEE 802.11ax: High-Efficiency WLANs

Faced with the tremendous increase in the amount of data traffic and associated congestion, mobile network operators are moving towards Heterogeneous networks HetNets, in the process of expanding network capacity. Offloading data traffic onto Wi-Fi in order to avoid congestion in the backbone is an important step in the evolution of HetNets. On-the-spot and delayed offloading have been widely studied in the literature. This paper proposes an offloading algorithm which has low computational complexity. The proposed algorithm offloads data based on a balking function which is dependent on present network condition. Using extensive simulations, the authors demonstrate that the proposed algorithm achieves reduction in mean transmission delay without sacrificing much on the offloading efficiency. This technique is more efficient and applicable to real-time traffic, like live streaming video and audio, which has short and stringent delay requirements or deadlines.

Mobile Data Offloading for Streaming-Class Traffic with QoS Guarantee

Delayed mobile data offloading scheme for quality of service traffic: Design and analysis

We consider a multichannel Cognitive Radio Network (CRN), where secondary users sequentially sense channels for opportunistic spectrum access. In this scenario, the Channel Selection Algorithm (CSA) allows secondary users to find a vacant channel with the minimal number of channel switches. Most of the existing CSA literature assumes exponential ON-OFF time distribution for primary users (PU) channel occupancy pattern. This exponential assumption might be helpful to get performance bounds; but not useful to evaluate the performance of CSA under realistic conditions. An in-depth analysis of independent spectrum measurement traces reveals that wireless channels have typically heavy-tailed PU OFF times. In this paper, we propose an extension to the Predictive CSA framework and its generalization for heavy tailed PU OFF time distribution, which represents realistic scenarios. In particular, we calculate the probability of channel being idle for hyper-exponential OFF times to use in CSA. We implement our proposed CSA framework in a wireless test-bed and comprehensively evaluate its performance by recreating the realistic PU channel occupancy patterns. The proposed CSA shows significant reduction in channel switches and energy consumption as compared to Predictive CSA which always assumes exponential PU ON-OFF times.Through our work, we show the impact of the PU channel occupancy pattern on the performance of CSA in multichannel CRN.

T. G. Venkatesh

Papers

Mobile Data Offloading for Streaming-Class Traffic with QoS Guarantee

Delayed mobile data offloading scheme for quality of service traffic: Design and analysis

Channel Selection Algorithm for Cognitive Radio Networks with Heavy-Tailed Idle Times