This review provides a pedagogic and self-contained introduction to master equations and to their representation by path integrals. Since the 1930s, master equations have served as a fundamental tool to understand the role of fluctuations in complex biological, chemical, and physical systems. Despite their simple appearance, analyses of master equations most often rely on low-noise approximations such as the Kramers-Moyal or the system size expansion, or require ad-hoc closure schemes for the derivation of low-order moment equations. We focus on numerical and analytical methods going beyond the low-noise limit and provide a unified framework for the study of master equations. After deriving the forward and backward master equations from the Chapman-Kolmogorov equation, we show how the two master equations can be cast into either of four linear partial differential equations (PDEs). Three of these PDEs are discussed in detail. The first PDE governs the time evolution of a generalized probability generating function whose basis depends on the stochastic process under consideration. Spectral methods, WKB approximations, and a variational approach have been proposed for the analysis of the PDE. The second PDE is novel and is obeyed by a distribution that is marginalized over an initial state. It proves useful for the computation of mean extinction times. The third PDE describes the time evolution of a 'generating functional', which generalizes the so-called Poisson representation. Subsequently, the solutions of the PDEs are expressed in terms of two path integrals: a 'forward' and a 'backward' path integral. Combined with inverse transformations, one obtains two distinct path integral representations of the conditional probability distribution solving the master equations. We exemplify both path integrals in analysing elementary chemical reactions. Moreover, we show how a well-known path integral representation of averaged observables can be recovered from them. Upon expanding the forward and the backward path integrals around stationary paths, we then discuss and extend a recent method for the computation of rare event probabilities. Besides, we also derive path integral representations for processes with continuous state spaces whose forward and backward master equations admit Kramers-Moyal expansions. A truncation of the backward expansion at the level of a diffusion approximation recovers a classic path integral representation of the (backward) Fokker-Planck equation. One can rewrite this path integral in terms of an Onsager-Machlup function and, for purely diffusive Brownian motion, it simplifies to the path integral of Wiener. To make this review accessible to a broad community, we have used the language of probability theory rather than quantum (field) theory and do not assume any knowledge of the latter. The probabilistic structures underpinning various technical concepts, such as coherent states, the Doi-shift, and normal-ordered observables, are thereby made explicit.

Master equations and the theory of stochastic path integrals

With the advent of the fifth generation of wireless standards and an increasing demand for higher throughput, methods to improve spectral efficiency of wireless systems have become very important. In the context of cognitive radio, a substantial increase in throughput is possible if the secondary user can make smart decisions regarding which channel to sense and when or how often to sense. Here, we propose an algorithm to not only select a channel for data transmission, but also to predict how long the channel will remain unoccupied so that the time spent on channel sensing can be minimized. Our algorithm learns in two stages—a reinforcement learning approach for channel selection and a Bayesian approach to determine the duration for which sensing can be skipped. Comparisons with other methods are provided through extensive simulations. We show that the number of sensing operations is minimized with negligible increase in primary user interference; this implies that less energy is spent by the secondary user in sensing, and also higher throughput is achieved by saving the time spent on sensing.

Spectrum Access In Cognitive Radio Using a Two-Stage Reinforcement Learning Approach

This review provides a pedagogic and self-contained introduction to master equations and to their representation by path integrals. We discuss analytical and numerical methods for the solution of master equations, keeping our focus on methods that are applicable even when stochastic fluctuations are strong. The reviewed methods include the generating function technique and the Poisson representation, as well as novel ways of mapping the forward and backward master equations onto linear partial differential equations (PDEs). Spectral methods, WKB approximations, and a variational approach have been proposed for the analysis of the PDE obeyed by the generating function. After outlining these methods, we solve the derived PDEs in terms of two path integrals. The path integrals provide distinct exact representations of the conditional probability distribution solving the master equations. We exemplify both path integrals in analysing elementary chemical reactions. Furthermore, we review a method for the approximation of rare event probabilities and derive path integral representations of Fokker-Planck equations. To make our review accessible to a broad community, we have used the language of probability theory rather than quantum (field) theory. The probabilistic structures underpinning various technical concepts, such as coherent states, the Doi-shift, and normal-ordered observables, are thereby made explicit.

Based on a coherent state representation of noise operator and an ensemble averaging procedure using Wigner canonical thermal distribution for harmonic oscillators, a generalized quantum Langevin equation has been recently developed [Phys. Rev. E 65, 021109 (2002); 66, 051106 (2002)] to derive the equations of motion for probability distribution functions in c-number phase-space. We extend the treatment to explore several systematic approximation schemes for the solutions of the Langevin equation for nonlinear potentials for a wide range of noise correlation, strength and temperature down to the vacuum limit. The method is exemplified by an analytic application to harmonic oscillator for arbitrary memory kernel and with the help of a numerical calculation of barrier crossing, in a cubic potential to demonstrate the quantum Kramers’ turnover and the quantum Arrhenius plot.

Solution of quantum Langevin equation: Approximations, theoretical and numerical aspects

IEEE 802.11ax-2019 will replace both IEEE 802.11n-2009 and IEEE 802.11ac-2013 as the next high-throughput Wireless Local Area Network (WLAN) amendment. In this paper, we review the expected future WLAN scenarios and use-cases that justify the push for a new PHY/MAC IEEE 802.11 amendment. After that, we overview a set of new technical features that may be included in the IEEE 802.11ax-2019 amendment and describe both their advantages and drawbacks. Finally, we discuss some of the network-level functionalities that are required to fully improve the user experience in next-generation WLANs and note their relation with other on-going IEEE 802.11 amendments.

/pdf/ieee-802-11ax-high-efficiency-wlans-57ejajfm5d.pdf

IEEE 802.11ax: High-Efficiency WLANs

In the domain of wireless networking, there have been several attempts to design, and analyze resource allocation problems using Game theoretic techniques (such as, random access game for contention control). In the current paper, a Medium Access Control (MAC) protocol based on Minority Game(MG) theory is proposed for channel supporting Multipacket Reception (MPR). Several performance metrics of the protocol (like, mean of Attendance, Volatility, System Throughput and Energy Expenditure) are studied through extensive simulation. The behaviour of the protocol is also analyzed from the perspective of Crowd-Anticrowd Theory, a popular tool from econophysics literature.

Wireless MAC Protocol based on Crowd-Anticrowd Theory

Software Defined Networking (SDN) has been preferred over traditional networking due to its dynamic nature in adapting the network structure This agile nature of SDN imparts non-stationarity in traffic In this work, we characterize the SDN traffic and study its behavior under dynamic conditions using Augmented Dickey Fuller (ADF) test Later, we model the SDN under non-stationary conditions using queueing model and solve for average queue length at both controller and switch using Pointwise Stationary Fluid Flow Approximation (PSFFA) The analytical results have been validated through simulations We develop congestion control algorithm based on (a) Proportional Integral Derivative (PID) control mechanism and (b) Dynamic Random Early Detection (DRED) control mechanism for SDN controller using the fluid flow model Finally we demonstrate their effectiveness in stabilizing the queue length at the switch and controller under non-stationary conditions In nut shell our work brings out the importance of the non-stationary behaviour of the traffic in the design and analysis of SDN and its control algorithms

Modeling & analysis of software defined networks under non-stationary conditions

Software Defined Networking (SDN) is a trending technology and its popularity can be attributed to its speed and agility in deploying new applications. Realistic modeling of SDN traffic is crucial in designing control algorithms and in provisioning network resources. The purpose of this paper is to model and characterize the SDN traffic. To this end, we have created an SDN environment using the mininet emulator and OpenDayLight (ODL) controller using which we have collected the SDN traffic data traces. The quantiles of inter-arrival times of SDN data traffic have been plotted against the quantiles of theoretical distributions. Then the Kolmogorov Smirnov (KS) test has been applied to model the SDN traffic and to capture its distribution characteristics effectively. From the KS test, it has been inferred that the inter arrival times of SDN traffic follows the Generalized Extreme Value (GEV) distribution. These findings indicate that traditional Poisson models for SDN traffic may not be suitable for evaluating the performance of Software Defined Networks.

Traffic Modeling & Characterization of Software Defined Networks

Ensuring safety on the roads can save thousands of people from dying in the road accidents. Many accidents occur due to over speeding of the vehicles and congestion on the roads. In order ensure the safety on the road as well as for infotainment applications IEEE has introduced the WAVE protocol. The 802.11p PHY layer and 1609.4 MAC layer are the bases for the WAVE protocol. The objective of the paper is to evaluate the performance of the WAVE protocol in ensuring the road safety under different traffic scenarios and to bring out the optimal network parameters for ensuring safety. In our model each vehicle broadcasts a beacon periodically containing its location and route. The Road Side Units (RSUs) within the range receives the beacon and replies back with information about the congestion in the future path. Based on that information, the vehicle can change / update the path to its destination. We have performed the simulations with help of two tools, SUMO for generating the traffic and creating the maps and OMNET ++ for simulating the network. We have computed the vehicle collision rate by varying different parameters of our model such as the transmission bit rate, packet interval, Transmission power, vehicle density and mobility. Inferences are drawn based on the simulation results.

Evaluation of Traffic-safety Features of WAVE Protocol for Vehicular Ad-hoc Networks

In Cognitive Radio Network (CRN) secondary users (SUs) opportunistically access the channel whenever it is not being used by primary users (PUs). The CRN Medium Access Control (MAC) protocol needs to sense the channel to find out the availability of spectrum. In this paper we propose a hybrid sensing mechanism for SUs to access the channel in CRN environment. In the proposed hybrid sensing mechanism the SU sense the channel periodically when it is used by PU while the SU sense the channel continuously whenever the channel is used by the SU. We investigate the role of this hybrid sensing mechanism wherein PUs have a preemptive priority over SUs. Later we study the performance of our MAC protocol with respect to the following metrics: Mean Waiting Time (MWT) and Queue Length Distribution (QLD). We study the variation of these metrics as a function of sensing period (τ), SU arrival rate (λ s ), SU service rate (μ s ), and PU traffic intensity rate (ρ PU ). From simulation results, we observe that there is a trade-off between MWT and sensing energy while considering periodic sensing. We finally suggest the type of sensing mechanism to be used for the channel access, based on the traffic intensity of PU.

T. G. Venkatesh

Papers

Wireless MAC Protocol based on Crowd-Anticrowd Theory

Modeling & analysis of software defined networks under non-stationary conditions

Traffic Modeling & Characterization of Software Defined Networks

Evaluation of Traffic-safety Features of WAVE Protocol for Vehicular Ad-hoc Networks

Preemptive priority mechanism with hybrid spectrum sensing for cognitive radio networks