Journal ArticleDOI
Performance optimization of continuous-time Markov control processes based on performance potentials
Tang Hao,Xi Hongsheng,Yin Baoqun +2 more
TLDR
Average-cost optimization problems for a class of continuous-time Markov control processes with a compact action set with average-cost optimality equation derived and the existence of its solution established are studied.Abstract:
Average-cost optimization problems for a class of continuous-time Markov control processes with a compact action set have been studied The definition of a generalized average-cost Poisson equation, which can be viewed as an extension to the standard one is first given Markov performance potentials are defined as its unique solution Based on the formula of performance potentials, an average-cost optimality equation is derived and the existence of its solution is established Then, policy iteration and value iteration algorithms are proposed and their convergence discussed A numerical example for controlled closed queuing networks illustrates the application of the proposed value iteration algorithmread more
Citations
More filters
Journal ArticleDOI
50 Years of international journal of systems science: a review of the past and trends for the future
TL;DR: The fundamental characteristics of the documents were identified from the bibliometric indicators, and features of keywords were revealed over the half century, and the future research trends in the four clusters were predicted as evidenced in the historical data.
Journal ArticleDOI
Error bounds of optimization algorithms for semi-Markov decision processes
Tang Hao,Yin Baoqun,Xi Hongsheng +2 more
TL;DR: This work introduces an α-uniformized Markov chain (UMC) for a semi-Markov decision process (SMDP) via A α and a uniformized parameter, and derives the error bounds for a potential-based policy-iteration algorithm and a value-iterations algorithm, respectively, when there exist various calculation errors.
Journal ArticleDOI
Modeling and Optimization of M/G/1-Type Queueing Networks: An Efficient Sensitivity Analysis Approach
TL;DR: An online policy gradient optimization algorithm based on a single sample path is provided to avoid suffering from a “curse of dimensionality” and the asymptotic convergence properties of this algorithm are proved.
Journal ArticleDOI
Modeling and optimization control of a demand-driven, conveyor-serviced production station
TL;DR: Simulation results are presented to show that by the established model and proposed optimization methods the system can achieve an optimal or suboptimal look-ahead control policy once the capacities of both the buffer and the bank are designed appropriately.
Journal ArticleDOI
The optimal robust control policy for uncertain semi-Markov control processes
Tang Hao,Xi Hongsheng,Yin Baoqun +2 more
TL;DR: A potential-based policy iteration method is proposed in this work to search for the optimal robust control policy for SMCPs with uncertain infinitesimal transition rates that are represented as compact sets.
References
More filters
Book
Dynamic Programming and Optimal Control
TL;DR: The leading and most up-to-date textbook on the far-ranging algorithmic methododogy of Dynamic Programming, which can be used for optimal control, Markovian decision problems, planning and sequential decision making under uncertainty, and discrete/combinatorial optimization.
Journal ArticleDOI
The Relations Among Potentials, Perturbation Analysis,and Markov Decision Processes
TL;DR: This paper provides an introductory discussion for an important concept, the performance potentials of Markov processes, and its relations with perturbation analysis (PA), average-cost Markov decision processes, Poisson equations, α-potentials, the fundamental matrix, and the group inverse of the transition matrix.
ReportDOI
A Lyapunov Bound for Solutions of Poisson's Equation
TL;DR: In this article, a Lyapunov function criterion was developed to bound the solution g to Poisson's equation for a positive recurrent Harris chain with invariant measure pi, and then applied to obtain sufficient conditions that guarantee that the solution be an element of L sub p (pi) when p = 2.
Journal ArticleDOI
Technical Communique: A unified approach to Markov decision problems and performance sensitivity analysis
TL;DR: A simple approach is proposed that provides a unified formulation for the performance sensitivity analysis of Markov chains and Markov decision problems with both infinite horizon average-cost and discounted performance criteria.
Journal ArticleDOI
Single sample path-based optimization of Markov chains
TL;DR: A fast algorithm is proposed, which updates the policy whenever the system reaches a particular set of states and it is proved that the algorithm converges to the true optimal policy with probability one under some conditions.