Shalabh Bhatnagar

TL;DR: This problem of finding optimal energy management policies in the presence of energy harvesting sources to maximize network performance is considered in the discounted cost Markov decision process framework and two reinforcement learning algorithms are applied.

TL;DR: In this paper, a two timescale stochastic approximation scheme which uses coupled iterations is used for simulation-based parametric optimization as an alternative to traditional "infinitesimal perturbation analysis" schemes, it avoids the aggregation of data present in many other schemes.

TL;DR: This paper presents an algorithm based on stochastic optimization to tune the thresholds that are associated with a TLC algorithm for optimal performance, and proposes the following three novel TLC algorithms: a full-state Q- learning algorithm with state aggregation, a Q-learning algorithm with function approximation that involves an enhanced feature selection scheme, and a priority-based TLC scheme.

TL;DR: The crucial idea in the method is the concept of partial observability and how UAVs can retain relevant information about the environment structure to make better future navigation decisions and it has a high inference rate and reduces power wastage as it minimizes oscillatory motion of UAV.

TL;DR: An asymptotic convergence analysis of two time-scale stochastic approximation driven by “controlled” Markov noise is presented and a solution to the off-policy convergence problem for temporal-difference learning with linear function approximation is presented.