This paper is the first step in the proof of existence of equilibrium payoffs for two-player stochastic games with finite state and action sets. It reduces the existence problem to the class of so-called positive absorbing recursive games. The existence problem for this class is solved in a subsequent paper.

Two-player stochastic games I: A reduction

In this article we describe a major part of the theory of zero-sum discrete-time stochastic games. We overview all basic streams of research in this area, in the context of the existence of value and uniform value, algorithms, vector payoffs, incomplete information and imperfect state observation. Also some models related to continuous-time games, e.g., games with short-stage duration, semi-Markov games, are overviewed. Moreover, a number of applications of stochastic games are pointed out. The provided reference list reveals a tremendous progress made in the field of zero-sum stochastic games since the seminal work of Shapley (1953).

Zero-Sum Stochastic Games

We show that the problem of finding optimal strategies for both players in a simple stochastic game reduces to the generalized linear complementarity problem (GLCP) with a P-matrix, a well-studied problem whose hardness would imply NP = co-NP. This makes the rich GLCP theory and numerous existing algorithms available for simple stochastic games. As a special case, we get a reduction from binary simple stochastic games to the P-matrix linear complementarity problem (LCP).

Simple stochastic games and P-matrix generalized linear complementarity problems

Network Formation Games

We consider two-person zero-sum mean payoff undiscounted stochastic games and obtain sufficient conditions for the existence of a saddle point in uniformly optimal stationary strategies. Namely, these conditions enable us to bring the game, by applying potential transformations, to a canonical form in which locally optimal strategies are globally optimal, and hence the value for every initial position and the optimal strategies of both players can be obtained by playing the local game at each state. We show that these conditions hold for the class of additive transition (AT) games, that is, the special case when the transitions at each state can be decomposed into two parts, each controlled completely by one of the two players. An important special case of AT-games form the so-called BWR-games which are played by two players on a directed graph with positions of three types: Black, White and Random. We give an independent proof for the existence of a canonical form in such games, and use this result to derive the existence of a canonical form (and hence, of a saddle point in uniformly optimal stationary strategies) in a wide class of games, which includes stochastic games with perfect information (PI), switching controller (SC) games and additive rewards, additive transition (ARAT) games. Unlike the proof for AT-games, our proof for the BWR-case does not rely on the existence of a saddle point in stationary strategies. We also derive some algorithmic consequences from these our reductions to BWR-games, in terms of solving PI-, and ARAT-games in sub-exponential time.

On Canonical Forms for Zero-Sum Stochastic Mean Payoff Games

We model social storage systems as a strategic network formation game. We define the utility of each player in the network under two different frameworks, one where the cost to add and maintain links is considered in the utility function and the other where budget constraints are considered. In the context of social storage and social cloud computing, these utility functions are the first of its kind, and we use them to define and analyze the social storage network game. We, then, present the pairwise stability concept adapted for social storage where both addition and deletion of links require mutual consent, as compared to mutual consent just for link addition in the pairwise stability concept defined by Jackson and Wolinsky [35]. Mutual consent for link deletion is especially important in the social storage setting. For symmetric storage networks, we prove that there exists a unique neighborhood size, independent of the number of players, where no pair of users has any incentive to increase or decrease their neighborhood size. We call this neighborhood size as the stability point. We provide some necessary and sufficient conditions for pairwise stability of social storage networks. Further, given the number of players and other parameters, we discuss which pairwise stable networks would evolve. We also show that in a connected pairwise stable network there can be at most one player with neighborhood size one less than (or one more than) the stability point.

Unique Stability Point in Social Storage Networks.

We consider certain mixtures, Γ, of classes of stochastic games and provide sufficient conditions for these mixtures to possess the orderfield property. For 2-player zero-sum and non-zero sum stochastic games, we prove that if we mix a set of states S1 where the transitions are controlled by one player with a set of states S2 constituting a sub-game having the orderfield property (where S1 ∩ S2 = ∅), the resulting mixture Γ with states S = S1 ∪ S2 has the orderfield property if there are no transitions from S2 to S1. This is true for discounted as well as undiscounted games. This condition on the transitions is sufficient when S1 is perfect information or SC (Switching Control) or ARAT (Additive Reward Additive Transition). In the zero-sum case, S1 can be a mixture of SC and ARAT as well. On the other hand,when S1 is SER-SIT (Separable Reward — State Independent Transition), we provide a counter example to show that this condition is not sufficient for the mixture Γ to possess the orderfield property. In addition to the condition that there are no transitions from S2 to S1, if the sum of all transition probabilities from S1 to S2 is independent of the actions of the players, then Γ has the orderfield property even when S1 is SER-SIT. When S1 and S2 are both SERSIT, their mixture Γ has the orderfield property even if we allow transitions from S2 to S1. We also extend these results to some multi-player games namely, mixtures with one player control Polystochastic games. In all the above cases, we can inductively mix many such games and continue to retain the orderfield property.

Orderfield property of mixtures of stochastic games

We briefly survey some results on the orderfield property of 2-player and multi-player stochastic games and their mixtures. Some of these classes of stochastic games can be solved by formulating them as a Linear Complementarity Problem (LCP) or (Generalized) Vertical Linear Complementarity Problem (VLCP). We discuss some of these results and prove that certain new subclasses and mixtures of multi-player (or n-person) stochastic games can be solved via LCP formulations.

Solving subclasses of multi-player stochastic games via linear complementarity problem formulations—a survey and some new results

Social Clouds have been gaining importance because of their potential for efficient and stable resource sharing without any (monetary) cost implications . There is a need, however, to look at how a social structure or relationship evolves to build a Social Cloud (by identifying factors that affect the social structure) and how social structure impacts individual resource sharing behavior. This paper presents a pairwise resource (or pairwise service) sharing social network model to explore the interdependence between social structure and resource (service) availability for an individual user or player. The paper also investigates effects of social structure on individual resource availability. Further, the paper analyzes positive and negative externalities, and aims to characterize stable social clouds.

Externalities and stability in social cloud

Finding a Nash equilibrium in a bimatrix game is PPAD-hard (Chen and Deng, 2006 [5], Chen, Deng and Teng, 2009 [6]). The problem, even when restricted to win-lose bimatrix games, remains PPAD-hard (Abbott, Kane and Valiant, 2005 [1]). However, there do exist polynomial time tractable classes of win-lose bimatrix games - such as, very sparse games (Codenotti, Leoncini and Resta, 2006 [8]) and planar games (Addario-Berry, Olver and Vetta, 2007 [2]).

We extend the results in the latter work to K3,3 minor-free games and a subclass of K5 minor-free games. Both these classes strictly contain planar games. Further, we sharpen the upper bound to unambiguous logspace UL, a small complexity class contained well within polynomial time P. Apart from these classes of games, our results also extend to a class of games that contain both K3,3 and K5 as minors, thereby covering a large and non-trivial class of win-lose bimatrix games. For this class, we prove an upper bound of nondeterministic logspace NL, again a small complexity class in P. Our techniques are primarily graph theoretic and use structural characterizations of the considered minor-closed families.

Nagarajan Krishnamurthy

Papers

Unique Stability Point in Social Storage Networks.

Orderfield property of mixtures of stochastic games

Solving subclasses of multi-player stochastic games via linear complementarity problem formulations—a survey and some new results

Externalities and stability in social cloud

Some tractable win-lose games