This review provides a pedagogic and self-contained introduction to master equations and to their representation by path integrals. Since the 1930s, master equations have served as a fundamental tool to understand the role of fluctuations in complex biological, chemical, and physical systems. Despite their simple appearance, analyses of master equations most often rely on low-noise approximations such as the Kramers-Moyal or the system size expansion, or require ad-hoc closure schemes for the derivation of low-order moment equations. We focus on numerical and analytical methods going beyond the low-noise limit and provide a unified framework for the study of master equations. After deriving the forward and backward master equations from the Chapman-Kolmogorov equation, we show how the two master equations can be cast into either of four linear partial differential equations (PDEs). Three of these PDEs are discussed in detail. The first PDE governs the time evolution of a generalized probability generating function whose basis depends on the stochastic process under consideration. Spectral methods, WKB approximations, and a variational approach have been proposed for the analysis of the PDE. The second PDE is novel and is obeyed by a distribution that is marginalized over an initial state. It proves useful for the computation of mean extinction times. The third PDE describes the time evolution of a 'generating functional', which generalizes the so-called Poisson representation. Subsequently, the solutions of the PDEs are expressed in terms of two path integrals: a 'forward' and a 'backward' path integral. Combined with inverse transformations, one obtains two distinct path integral representations of the conditional probability distribution solving the master equations. We exemplify both path integrals in analysing elementary chemical reactions. Moreover, we show how a well-known path integral representation of averaged observables can be recovered from them. Upon expanding the forward and the backward path integrals around stationary paths, we then discuss and extend a recent method for the computation of rare event probabilities. Besides, we also derive path integral representations for processes with continuous state spaces whose forward and backward master equations admit Kramers-Moyal expansions. A truncation of the backward expansion at the level of a diffusion approximation recovers a classic path integral representation of the (backward) Fokker-Planck equation. One can rewrite this path integral in terms of an Onsager-Machlup function and, for purely diffusive Brownian motion, it simplifies to the path integral of Wiener. To make this review accessible to a broad community, we have used the language of probability theory rather than quantum (field) theory and do not assume any knowledge of the latter. The probabilistic structures underpinning various technical concepts, such as coherent states, the Doi-shift, and normal-ordered observables, are thereby made explicit.

Master equations and the theory of stochastic path integrals

With the advent of the fifth generation of wireless standards and an increasing demand for higher throughput, methods to improve spectral efficiency of wireless systems have become very important. In the context of cognitive radio, a substantial increase in throughput is possible if the secondary user can make smart decisions regarding which channel to sense and when or how often to sense. Here, we propose an algorithm to not only select a channel for data transmission, but also to predict how long the channel will remain unoccupied so that the time spent on channel sensing can be minimized. Our algorithm learns in two stages—a reinforcement learning approach for channel selection and a Bayesian approach to determine the duration for which sensing can be skipped. Comparisons with other methods are provided through extensive simulations. We show that the number of sensing operations is minimized with negligible increase in primary user interference; this implies that less energy is spent by the secondary user in sensing, and also higher throughput is achieved by saving the time spent on sensing.

Spectrum Access In Cognitive Radio Using a Two-Stage Reinforcement Learning Approach

This review provides a pedagogic and self-contained introduction to master equations and to their representation by path integrals. We discuss analytical and numerical methods for the solution of master equations, keeping our focus on methods that are applicable even when stochastic fluctuations are strong. The reviewed methods include the generating function technique and the Poisson representation, as well as novel ways of mapping the forward and backward master equations onto linear partial differential equations (PDEs). Spectral methods, WKB approximations, and a variational approach have been proposed for the analysis of the PDE obeyed by the generating function. After outlining these methods, we solve the derived PDEs in terms of two path integrals. The path integrals provide distinct exact representations of the conditional probability distribution solving the master equations. We exemplify both path integrals in analysing elementary chemical reactions. Furthermore, we review a method for the approximation of rare event probabilities and derive path integral representations of Fokker-Planck equations. To make our review accessible to a broad community, we have used the language of probability theory rather than quantum (field) theory. The probabilistic structures underpinning various technical concepts, such as coherent states, the Doi-shift, and normal-ordered observables, are thereby made explicit.

Based on a coherent state representation of noise operator and an ensemble averaging procedure using Wigner canonical thermal distribution for harmonic oscillators, a generalized quantum Langevin equation has been recently developed [Phys. Rev. E 65, 021109 (2002); 66, 051106 (2002)] to derive the equations of motion for probability distribution functions in c-number phase-space. We extend the treatment to explore several systematic approximation schemes for the solutions of the Langevin equation for nonlinear potentials for a wide range of noise correlation, strength and temperature down to the vacuum limit. The method is exemplified by an analytic application to harmonic oscillator for arbitrary memory kernel and with the help of a numerical calculation of barrier crossing, in a cubic potential to demonstrate the quantum Kramers’ turnover and the quantum Arrhenius plot.

Solution of quantum Langevin equation: Approximations, theoretical and numerical aspects

IEEE 802.11ax-2019 will replace both IEEE 802.11n-2009 and IEEE 802.11ac-2013 as the next high-throughput Wireless Local Area Network (WLAN) amendment. In this paper, we review the expected future WLAN scenarios and use-cases that justify the push for a new PHY/MAC IEEE 802.11 amendment. After that, we overview a set of new technical features that may be included in the IEEE 802.11ax-2019 amendment and describe both their advantages and drawbacks. Finally, we discuss some of the network-level functionalities that are required to fully improve the user experience in next-generation WLANs and note their relation with other on-going IEEE 802.11 amendments.

/pdf/ieee-802-11ax-high-efficiency-wlans-57ejajfm5d.pdf

IEEE 802.11ax: High-Efficiency WLANs

We consider a multichannel Cognitive Radio Network (CRN), where secondary users sequentially sense channels for opportunistic spectrum access. In this scenario, the Channel Selection Algorithm (CSA) allows secondary users to find a vacant channel with the minimal number of channel switches. Most of the existing CSA literature assumes exponential ON-OFF time distribution for primary user's (PU) channel occupancy pattern. This exponential assumption might be helpful to get performance bounds; but not useful to evaluate the performance of CSA under realistic conditions. An in-depth analysis of independent spectrum measurement traces reveals that wireless channels have typically heavy-tailed PU OFF times. In this paper, we propose an extension to the Predictive CSA framework and its generalization for heavy tailed PU OFF time distribution, which represents realistic scenarios. In particular, we calculate the probability of channel being idle for hyper-exponential OFF times to use in CSA. We implement our proposed CSA framework in a wireless test-bed and comprehensively evaluate its performance by recreating the realistic PU channel occupancy patterns. The proposed CSA shows significant reduction in channel switches and energy consumption as compared to Predictive CSA which always assumes exponential PU ON-OFF times. Through our work, we show the impact of the PU channel occupancy pattern on the performance of CSA in multichannel CRN.

Channel Selection Algorithm for Cognitive Radio Networks with Heavy-Tailed Idle Times

In this study, the authors present an analytical model for the contention-based access periods (CBAPs) and service periods (SPs) as specified in the medium access control (MAC) layer of IEEE 802.11ad standards. The analytical model for CBAP is based on a two-dimensional Markov chain which captures the hybrid nature of IEEE 802.11ad MAC. The Markov chain is used to obtain important metrics like MAC throughput and average frame service time. The SPs are modelled as a M/G/1 queueing system with vacations. Using this model, the delay experienced by packets is calculated. The accuracy of the analytical models is established by extensive simulation results. A brief insight into optimal allocation of SP and CBAP is also discussed.

Performance analysis of contention-based access periods and service periods of 802.11ad hybrid medium access control

Dynamic spectrum access (DSA) allows the spectrum to be utilized efficiently through opportunistic access. Cognitive radio (CR) is a technology used to enable DSA. Unlike a wireless local area network (WLAN) that operates in unlicensed band, cognitive WLAN allows CRs to identify spectrum holes in licensed bands and opportunistically use it. DSA makes the design of the medium access control (MAC) layer in the cognitive WLAN challenging. In this paper, we design and carry out performance analysis of IEEE 802.11 DCF based MAC protocol for multichannel cognitive WLAN. Our main contributions are as follows: 1) A Markov chain model for CRs and 2) another Markov chain to model the joint usage of the channel by the licensed users and CRs. These coupled Markov chain models are then solved for stationary probability distribution. Using the above models, we derive throughput and delay of the CRs and study their variation against the number of CRs, the number of channels, and spectrum availability. Theoretical results are validated using simulation and compared with results from existing literature.

Design and Performance Analysis of Multichannel MAC Protocol for Cognitive WLAN

Reuse distance is an important metric for analytical estimation of cache miss rate. To find the miss rate of a particular cache, the reuse distance profile has to be measured for that particular level and configuration of the cache. Significant amount of simulation time and overhead can be reduced if we can find the miss rate of higher level cache like L2 cache from the RD profile with respect to a lower level cache (i.e., cache that is closer to the processor) such as L1. The objective of this paper is to give an analytical method to find the miss rate of L2 cache for various configurations from the RD profile with respect to L1 cache. We consider all three types of cache inclusion policies namely (i) Strictly Inclusive, (ii) Mutually Exclusive and (iii) Non-Inclusive Non-Exclusive policy. We first prove some general results relating the RD profile of L1 cache to that of L2 cache. We use probabilistic analysis for our derivations. We validate our model against simulations, using the multi-core simulator Sniper with the PARSEC and the SPLASH benchmark suites.

Analytical Miss Rate Calculation of L2 Cache from the RD Profile of L1 Cache

We present an analytical model for the access during the Service Periods (SP) of the IEEE 802.11ad Hybrid Medium Access Control protocol. As a performance measure of this protocol, we derive the worst case and average delay faced by the SP packets. We show that as the arrival rate of the SP packets increases, the delay increases linearly till a point, beyond which it grows exponentially. Further, we extend the model to variable length of beacon interval, and random allocation of SPs to the nodes. We show how a network designer can do optimal allocation of the SP and the CBAP duration to achieve a tradeoff between SP delay and CBAP throughput. We further extend our analysis for the case of heterogeneous system. Our analytical results are compared with simulation and the results show a good match.

T. G. Venkatesh

Papers

Channel Selection Algorithm for Cognitive Radio Networks with Heavy-Tailed Idle Times

Performance analysis of contention-based access periods and service periods of 802.11ad hybrid medium access control

Design and Performance Analysis of Multichannel MAC Protocol for Cognitive WLAN

Analytical Miss Rate Calculation of L2 Cache from the RD Profile of L1 Cache

Performance Analysis of Service Periods (SP) of the IEEE 802.11ad Hybrid MAC Protocol