Communication complexity of stochastic games
13 May 2009-pp 411-417
TL;DR: It is proved that pure equilibria of single controller stochastic games and those of SER-SIT (Separable Reward - State Independent Transition) games correspond to those of bimatrix games that are constructed from these stochastics games.
Abstract: We derive upper and lower bounds on the communication complexity of determining the existence of pure strategy Nash equilibria for some classes of stochastic games. We prove that pure equilibria of single controller stochastic games and those of SER-SIT (Separable Reward - State Independent Transition) games correspond to those of bimatrix games that are constructed from these stochastic games. Hence we extend communication complexity upper bounds of bimatrix games to these stochastic games. For SER-SIT games, we prove an upper bound of O(n 2 ) which is tight and which coincides with that for bimatrix games. Here n is the number of actions of each player in each state. Note that this bound is independent of the size of the actual payoffs. For single-controller games, we obtain an upper bound of min (O(n2|S|), O(|S| n2 log M)) where S is the set of states and M is the largest entry across all payoff matrices. Further, we reduce bimatrix games to stochastic games and hence, the lower bound extends from bimatrix games to stochastic games as well. We also establish the following results while proving upper bounds for SER-SIT games. To prove that pure equilibria of SER-SIT games correspond to those of auxiliary bimatrix games, we show that every SER-SIT game that has a pure equilibrium has a state-independent pure equilibrium too. We also show that we cannot relax the constraints of separable rewards or state independent transitions. We provide counter examples when the game is SER (but not SIT) and SIT (but not SER).
Citations
More filters
Journal Article•
TL;DR: In this article, it was shown that the problem of finding optimal strategies for both players in a simple stochastic game reduces to the generalized linear complementarity problem with a P-matrix, a well-studied problem whose hardness would imply NP = co-NP.
Abstract: We show that the problem of finding optimal strategies for both players in a simple stochastic game reduces to the generalized linear complementarity problem (GLCP) with a P-matrix, a well-studied problem whose hardness would imply NP = co-NP. This makes the rich GLCP theory and numerous existing algorithms available for simple stochastic games. As a special case, we get a reduction from binary simple stochastic games to the P-matrix linear complementarity problem (LCP).
28 citations
Dissertation•
03 Sep 2013
5 citations
Cites background from "Communication complexity of stochas..."
...2 Stochastic Games A stochastic game is a repeated game where the state can change from stage to stage according to a transition that is dependent on the current state and the moves of both players [11]....
[...]
TL;DR: A sufficient condition is provided for discounted as well as certain classes of undiscounted stochastic games to have symmetric optimal/equilibrium strategies—namely, transitions are symmetric and the payoff matrices of one player are the transpose of those of the other.
Abstract: In this paper, we address various types of two-person stochastic games—both zero-sum and nonzero-sum, discounted and undiscounted. In particular, we address different aspects of stochastic games, namely: (1) When is a two-person stochastic game completely mixed? (2) Can we identify classes of undiscounted zero-sum stochastic games that have stationary optimal strategies? (3) When does a two-person stochastic game possess symmetric optimal/equilibrium strategies? Firstly, we provide some necessary and some sufficient conditions under which certain classes of discounted and undiscounted stochastic games are completely mixed. In particular, we show that, if a discounted zero-sum switching control stochastic game with symmetric payoff matrices has a completely mixed stationary optimal strategy, then the stochastic game is completely mixed if and only if the matrix games restricted to states are all completely mixed. Secondly, we identify certain classes of undiscounted zero-sum stochastic games that have stationary optima under specific conditions for individual payoff matrices and transition probabilities. Thirdly, we provide sufficient conditions for discounted as well as certain classes of undiscounted stochastic games to have symmetric optimal/equilibrium strategies—namely, transitions are symmetric and the payoff matrices of one player are the transpose of those of the other. We also provide a sufficient condition for the stochastic game to have a symmetric pure strategy equilibrium. We also provide examples to show the sharpness of our results.
3 citations
Cites background from "Communication complexity of stochas..."
...In Krishnamurthy et al. (2009), the authors discuss pure strategy equilibria and show that pure strategy equilibria of the SER-SIT game and of the bimatrix game (E, F) correspond. In general, equilibria of the SER-SIT game and of the bimatrix game (E, F) may not correspond. Parthasarathy et al. (1984) give an example where the SER-SIT game has more number of equilibrium points than the bimatrix game to which it has been reduced....
[...]
...In Krishnamurthy et al. (2009), the authors discuss pure strategy equilibria and show that pure strategy equilibria of the SER-SIT game and of the bimatrix game (E, F) correspond....
[...]
References
More filters
20 Apr 2009
TL;DR: This beginning graduate textbook describes both recent achievements and classical results of computational complexity theory and can be used as a reference for self-study for anyone interested in complexity.
Abstract: This beginning graduate textbook describes both recent achievements and classical results of computational complexity theory. Requiring essentially no background apart from mathematical maturity, the book can be used as a reference for self-study for anyone interested in complexity, including physicists, mathematicians, and other scientists, as well as a textbook for a variety of courses and seminars. More than 300 exercises are included with a selected hint set.
2,965 citations
[...]
TL;DR: In a stochastic game the play proceeds by steps from position to position, according to transition probabilities controlled jointly by the two players, and the expected total gain or loss is bounded by M, which depends on N 2 + N matrices.
Abstract: In a stochastic game the play proceeds by steps from position to position, according to transition probabilities controlled jointly by the two players. We shall assume a finite number, N , of positions, and finite numbers m K , n K of choices at each position; nevertheless, the game may not be bounded in length. If, when at position k , the players choose their i th and j th alternatives, respectively, then with probability s i j k > 0 the game stops, while with probability p i j k l the game moves to position l . Define s = min k , i , j s i j k . Since s is positive, the game ends with probability 1 after a finite number of steps, because, for any number t , the probability that it has not stopped after t steps is not more than (1 − s ) t .
Payments accumulate throughout the course of play: the first player takes a i j k from the second whenever the pair i , j is chosen at position k. If we define the bound M: M = max k , i , j | a i j k | , then we see that the expected total gain or loss is bounded by M + ( 1 − s ) M + ( 1 − s ) 2 M + … = M / s . (1)
The process therefore depends on N 2 + N matrices P K l = ( p i j k l | i = 1 , 2 , … , m K ; j = 1 , 2 , … , n K ) A K = ( a i j k | …
2,622 citations
"Communication complexity of stochas..." refers background in this paper
...As (f, g) is a pure strategy equilibrium of Γ1 and as discounted stochastic games have a unique value (Shapley, [19]), we have...
[...]
...[19] L. Shapley, 1953, “”Stochastic Games”, Proceedings of the National Academy of Sciences, Vol. 39, pp. 1095-1100....
[...]
TL;DR: In this paper, simple constructive proofs are given of solutions to the matric matric system Mz − ω = q; z ≧ 0; ω ≧ 1; zT = 0, for various kinds of data M, q, which embrace quadratic programming and the problem of finding equilibrium points of bimatrix games.
Abstract: Some simple constructive proofs are given of solutions to the matric system Mz − ω = q; z ≧ 0; ω ≧ 0; and zT ω = 0, for various kinds of data M, q, which embrace the quadratic programming problem and the problem of finding equilibrium points of bimatrix games. The general scheme is, assuming non-degeneracy, to generate an adjacent extreme point path leading to a solution. The scheme does not require that some functional be reduced.
966 citations
Additional excerpts
...[9] C. Lemke, 1965, “Bimatrix Equilibrium Points and Mathematical Programming”, Management Science, Vol. 11, pp. 681-689....
[...]