We consider two-player turn-based games with zero-reachability and zero-safety objectives generated by extended vector addition systems with states. Although the problem of deciding the winner in such games is undecidable in general, we identify several decidable and even tractable subcases of this problem obtained by restricting the number of counters and/or the sets of target configurations.

Reachability Games on Extended Vector Addition Systems with States

We study the computational complexity of basic decision
problems for one-counter simple stochastic games (OC-SSGs),
under various objectives. OC-SSGs are 2-player turn-based
stochastic games played on transition graphs of classic
one-counter automata. We study primarily the termination
objective, where one player has to maximize the probability of
reaching counter value 0, while the other player wishes to
avoid this. Partly motivated by the goal of understanding
termination objectives, we also study certain ``limit'' and
``long run average'' reward objectives. We show that the
qualitative termination problem for OC-SSGs is both in NP and
coNP, and in P-time for 1-player OC-SSGs. Moreover, we show
that quantitative limit problems for OC-SSGs are both in NP and
coNP, and are in P-time for 1-player OC-MDPs. Both qualitative
limit problems and qualitative termination problems for OC-SSGs
are already at least as hard as Condon's quantitative decision
problem for finite-state SSGs.

/pdf/one-counter-stochastic-games-oou0dlzkr9.pdf

One-Counter Stochastic Games

We introduce Recursive Markov Decision Processes (RMDPs) and Recursive Simple Stochastic Games (RSSGs), which are classes of (finitely presented) countable-state MDPs and zero-sum turn-based (perfect information) stochastic games. They extend standard finite-state MDPs and stochastic games with a recursion feature. We study the decidability and computational complexity of these games under termination objectives for the two players: one player's goal is to maximize the probability of termination at a given exit, while the other player's goal is to minimize this probability. In the quantitative termination problems, given an RMDP (or RSSG) and probability p, we wish to decide whether the value of such a termination game is at least p (or at most p); in the qualitative termination problem we wish to decide whether the value is 1. The important 1-exit subclasses of these models, 1-RMDPs and 1-RSSGs, correspond in a precise sense to controlled and game versions of classic stochastic models, including multitype Branching Processes and Stochastic Context-Free Grammars, where the objective of the players is to maximize or minimize the probability of termination (extinction). We provide a number of upper and lower bounds for qualitative and quantitative termination problems for RMDPs and RSSGs. We show both problems are undecidable for multi-exit RMDPs, but are decidable for 1-RMDPs and 1-RSSGs. Specifically, the quantitative termination problem is decidable in PSPACE for both 1-RMDPs and 1-RSSGs, and is at least as hard as the square root sum problem, a well-known open problem in numerical computation. We show that the qualitative termination problem for 1-RMDPs (i.e., a controlled version of branching processes) can be solved in polynomial time both for maximizing and minimizing 1-RMDPs. The qualitative problem for 1-RSSGs is in NP ∩ coNP, and is at least as hard as the quantitative termination problem for Condon's finite-state simple stochastic games, whose complexity remains a well known open problem. Finally, we show that even for 1-RMDPs, more general (qualitative and quantitative) model-checking problems with respect to linear-time temporal properties are undecidable even for a fixed property.

/pdf/recursive-markov-decision-processes-and-recursive-stochastic-5b6jzgri66.pdf

Recursive Markov Decision Processes and Recursive Stochastic Games

https://www.research.ed.ac.uk/portal/files/14010472/qest08_qbd_paper.pdf

Quasi-Birth-Death Processes, Tree-Like QBDs, Probabilistic 1-Counter Automata, and Pushdown Systems

We study termination time and recurrence time in programs with unbounded recursion, which are either randomized or operate on some statistically quantified inputs. As the underlying formal model for such programs we use probabilistic pushdown automata (pPDA) which are equivalent to probabilistic recursive state machines. We obtain tail bounds for the distribution of termination time for pPDA. We also study the recurrence time for probabilistic recursive programs that are not supposed to terminate (such as system daemons, network servers, etc.). Typically, such programs react to certain requests generated by their environment, and hence operate in finite request-service cycles. We obtain bounds for the frequency of long request-service cycles.

WojtczakDominik

Papers

Quasi-Birth-Death Processes, Tree-Like QBDs, Probabilistic 1-Counter Automata, and Pushdown Systems