Showing papers on "Stochastic game published in 1992"

A Primer in Game Theory

Abstract: PrefaceGame theory is the study of multiperson decision problems. Such problems arise frequently in economics. As is widely appreciated, for example, oligopolies present multiperson problems -- each firm must consider what the others will do. But many other applications of game theory arise in fields of economics other than industrial organization. At the micro level, models of trading processes (such as bargaining and auction models) involve game theory. At an intermediate level of aggregation, labor and financial economics include game-theoretic models of the behavior of a firm in its input markets (rather than its output market, as in an oligopoly). There also are multiperson problems within a firm: many workers may vie for one promotion; several divisions may compete for the corporation's investment capital. Finally, at a high level of aggregation, international economics includes models in which countries compete (or collude) in choosing tariffs and other trade policies, and macroeconomics includes models in which the monetary authority and wage or price setters interact strategically to determine the effects of monetary policy.This book is designed to introduce game theory to those who will later construct (or at least consume) game-theoretic models in applied fields within economics. The exposition emphasizes the economic applications of the theory at least as much as the pure theory itself, for three reasons. First, the applications help teach the theory; formal arguments about abstract games also appear but play a lesser role. Second, the applications illustrate the process of model building -- the process of translating an informal description of a multiperson decision situation into a formal, game-theoretic problem to be analyzed. Third, the variety of applications shows that similar issues arise in different areas of economics, and that the same game-theoretic tools can be applied in each setting. In order to emphasize the broad potential scope of the theory, conventional applications from industrial organization largely have been replaced by applications from labor, macro, and other applied fields in economics.We will discuss four classes of games: static games of complete information, dynamic games of complete information, static games of incomplete information, and dynamic games of incomplete information. (A game has incomplete information if one player does not know another player's payoff, such as in an auction when one bidder does not know how much another bidder is willing to pay for the good being sold.) Corresponding to these four classes of games will be four notions of equilibrium in games: Nash equilibrium, subgame-perfect Nash equilibrium, Bayesian Nash equilibrium, and perfect Bayesian equilibrium.Two (related) ways to organize one's thinking about these equilibrium concepts are as follows. First, one could construct sequences of equilibrium concepts of increasing strength, where stronger (i.e., more restrictive) concepts are attempts to eliminate implausible equilibria allowed by weaker notions of equilibrium. We will see, for example, that subgame-perfect Nash equilibrium is stronger than Nash equilibrium and that perfect Bayesian equilibrium in turn is stronger than subgame-perfect Nash equilibrium. Second, one could say that the equilibrium concept of interest is always perfect Bayesian equilibrium (or perhaps an even stronger equilibrium concept), but that it is equivalent to Nash equilibrium in static games of complete information, equivalent to subgame-perfection in dynamic games of complete (and perfect) information, and equivalent to Bayesian Nash equilibrium in static games of incomplete information.The book can be used in two ways. For first-year graduate students in economics, many of the applications will already be familiar, so the game theory can be covered in a half-semester course, leaving many of the applications to be studied outside of class. For undergraduates, a full-semester course can present the theory a bit more slowly, as well as cover virtually all the applications in class. The main mathematical prerequisite is single-variable calculus; the rudiments of probability and analysis are introduced as needed.I learned game theory from David Kreps, John Roberts, and Bob Wilson in graduate school, and from Adam Brandenburger, Drew Fudenberg, and Jean Tirole afterward. I owe the theoretical perspective in this book to them. The focus on applications and other aspects of the pedagogical style, however, are largely due to the students in the MIT Economics Department from 1985 to 1990, who inspired and rewarded the courses that led to this book. I am very grateful for the insights and encouragement all these friends have provided, as well as for the many helpful comments on the manuscript I received from Joe Farrell, Milt Harris, George Mailath, Matthew Rabin, Andy Weiss, and several anonymous reviewers. Finally, I am glad to acknowledge the advice and encouragement of Jack Repcheck of Princeton University Press and financial support from an Olin Fellowship in Economics at the National Bureau of Economic Research.

The complexity of stochastic games

Abstract: We consider the complexity of stochastic games—simple games of chance played by two players We show that the problem of deciding which player has the greatest chance of winning the game is in the class NP ⌢ co- NP

The principal-agent relationship with an informed principal. II : Common values

Abstract: In many circumstances, a principal may have relevant private information when she proposes a contract to an agent. We analyze such a principal-agent relationship as a noncooperative game. The principal proposes a contract, which is accepted or rejected by the agent (who, for most of our analysis, has no private information). The contract is executed if accepted; otherwise, the reservation allocation takes effect. This allocation may be determined by a pre-existing contract (which the principal, by her proposal, is attempting to renegotiate), or it may simply be the no-trade point. In this paper, we assume that the principal's information directly affects the agent's payoff. Before solving the game, we discuss Pareto efficiency with asymmetric information. We define an incentive-compatible allocation to be weakly interim efficient (WIE) if there exists no alternative incentive-compatible allocation that both parties prefer for all possible beliefs that the agent might have about the principal's private information (type). We show that any WIE allocation is interim-efficient (IE) for some beliefs. The Rothschild-Stiglitz-Wilson (RSW) allocation relative to the reservation allocation /.L is the allocation that maximizes the payoff of each type of principal within the class of incentive-compatible allocations that guarantee the agent at least the utility he gets from A' irrespective of his beliefs about the principal's type. The equilibrium set of the contract proposal game consists of the allocations that weakly Pareto dominate the RSW allocation. Thus, there is a unique equilibrium outcome if and only if the latter is IE (and the equilibrium outcome is the RSW allocation itself). After characterizing the equilibrium allocations, we study those that are renegotiation-proof, when either the principal or the agent leads the renegotiation. We then compare our contract proposal game, which is a signaling model, with its "screening" counterpart. We conclude by extending our results to the case in which the agent as well as the principal has private information under the assumption of quasi-linear preferences.

Repeated Games Played by Overlapping Generations of Players

Abstract: The present paper tries to explain cooperative behaviour in an organization run by a sequence of long- but finitely-lived agents. We show that the Folk Theorem holds for infinitely repeated games with overlapping generations of finitely-lived players; any mutually beneficial outcome can approximately be sustained if the player's life span and the overlapping periods are long enough. The result is stronger than the usual Folk Theorems in that it employs no assumption on the stage game, such as the full dimensionality of payoff set or multiplicity of equilibria.

Existence of Stationary Correlated Equilibria with Symmetric Information for Discounted Stochastic Games

Abstract: The following theorem is proved: Every nonzero-sum discounted stochastic game in countably generated measurable state space with compact metric action spaces admits a stationary correlated equilibrium point with symmetric public information whenever the immediate rewards and transition densities are measurable with respect to the state variable and continuous with respect to joint actions.

The Use of Information in Repeated Games with Imperfect Monitoring

Abstract: The present paper formalizes the idea that improved monitoring helps coordination in long term relationships. Specifically, the pure-strategy sequential equilibrium payoff set is shown to expand (in the sense of set inclusion) in repeated games with inperfect monitoring, when the quality of the signal improves in Blackwell's sense. Furthermore, the directions of the expansion are identified.

Strongly balanced cooperative games

Abstract: Kaneko/Wooders (1982) derived a list of necessary and sufficient conditions for a partitioning game to have a nonempty core regardless of the payoff functions of its effective coalitions. The main purpose of our paper is to provide a graph-theoretical characterization of this family of games whose associated hypergraphs we callstrongly balanced: we show that the strong balancedness condition is equivalent to thenormality of the hypergraph, which is a type ofcoloring property (Lovasz (1972)). We also study interesting economic examples ofcommunication andassignment games and provide direct proofs that their associated hypergraphs are strongly balanced.

Reduced games, consistency, and the core

Abstract: This paper establishes an axiomatization of the core by means of an internal consistency property with respect to a new reduced game introduced by Moulin (1985). Given a payoff vector chosen by a solution for some game, and given a subgroup of agents, we define thereduced game as that in which each coalition in the subgroup could attain payoffs to its members only if they are compatible with the initial payoffs toall the members outside of the subgroup. The solution isconsistent if it selects the same payoff distribution for the reduced game as initially. We show that consistency together with individual rationality characterizes the core of both transferable and non-transferable utility games.

Constitutional selection of collective-choice rules in a cooperative enterprise

Abstract: Constitutional choice in a cooperative enterprise as embodied in its articles of incorporation is viewed as a problem of selecting an efficient long-term contract. Economizing on transaction costs dictates recourse to collective choice rules, each of which defines a game among cooperative members. Multiple game solutions along with individual members' ignorance as to their future socio-economic position create uncertainty with respect to members' payoff at the operating phase. Constitutional selection of collective choice rules and their assignment to collective choice problem areas seeks to minimize the sum of bargaining costs and the members' risk premia.

Monitoring cooperative equilibria in a stochastic differential game

Abstract: The authors study a class of equilibria for stochastic differential games which are based on monitoring an implicit cooperative solution. The equilibrium is reached by implementing a random triggering scheme which changes the mode of play when there is an indication that at least one player may be departing from the cooperative solution. These equilibria make use of a particular kind of memory strategy and exploit the nonuniqueness of the solution to an associated Hamilton-Jacobi-Bellman (HJB) equation. In particular, it is shown that these equilibria, which satisfy a continuous-time dynamic programming equation, can be designed in such a way that they generate payoffs which dominate those obtained via the classical feedback Nash equilibrium for the original stochastic diffusion game. >

Some comments on a theorem of Hardy and Littlewood

Abstract: In this note, we reconstruct a proof of a classical result due to Hardy and Littlewood. While this result has played an important role in the modern theories of Markov decision processes and stochastic games, it is not that easy to find its proof in the literature in the format in which it has been applied. Furthermore, we supply either examples or complete citations for the other related cases which are not covered by the Hardy-Littlewood theorem.

Correlated equilibria in two-player repeated games with nonobservable actions

Abstract: Four kinds of correlated equilibrium payoff sets in undiscounted repeated games with nonobservable actions are studied. Three of them. the upper, the uniform, and Banach lead to the same payoff set, whereas the lower one in general is associated with a larger set. The extensive form correlated equilibrium is also explored. It turns out that both the regular and extensive form correlated equilibria yield the same sets of payoffs.

Two-player repeated games with nonobservable actions and observable payoffs

Abstract: This paper studies two-person repeated games in which after each stage a player is informed about the payoff he received at the previous stage. The information can, in some cases, include more than that. Four kinds of Nash-equilibrium concepts are defined by the limit of the means. A characterization of the equilibrium-payoffs sets and several properties of these sets are given. As a specific example, the standard information case, that of the folk theorem, is provided.

Computing simply stable equilibria

Abstract: For each two-player game, a linear-programming algorithm finds a component of the Nash equilibria and a subset of its perfect equilibria that are simply stable in the sense that there are nearby equilibria for each nearby game that perturbs one strategy's probability or payoff more than others. Copyright 1992 by The Econometric Society.

On the equilibrium payoffs set of two player repeated games with imperfect monitoring

Abstract: We show that any payoff, sustainable by a joint strategy of finitely repeated games, from which no player can deviate and gain by a non-detectable deviation, is a uniform equilibrium of the infinite repeated game This provides a characterization of the uniform equilibrium payoffs in terms of the finitely repeated games

Zero-sum two-person semi-markov games

Abstract: Semi-Markov games are investigated under discounted and limiting average payoff criteria. The issue of the existence of the value and a pair of stationary optimal strategies are settled; the optimality equation is studied and under a natural ergodic condition the existence of a solution to the optimality equation is proved for the limiting average case. Semi-Markov games provide useful flexibility in constructing recursive game models. All the work on Markov/semi-Markov decision processes and Markov (stochastic) games can be viewed as special cases of the developments in this paper.

An operator solution of stochastic games

Abstract: A class of zero-sum, two-person stochastic games is shown to have a value which can be calculated by transfinite iteration of an operator. The games considered have a countable state space, finite action spaces for each player, and a payoff sufficiently general to include classical stochastic games as well as Blackwell’s infiniteGδ games of imperfect information.

Chapter 4 Repeated games with complete information

Abstract: 0 Summary The theory of repeated games is concerned with the analysis of behavior in long-term interactions as opposed to one-shot situations; in this framework new objects occur in the form of threats, cooperative plans, signals, etc that are deeply related to “real life” phenomena like altruism, reputation or cooperation More precisely, repeated games with complete information, also called supergames, describe situations where a play corresponds to a sequence of plays of the same stage game and where the payoffs are some long-run average of the stage payoffs Note that unlike general repeated games [see, for example, Mertens, Sorin and Zamir (1992) ] the stage game is the same (the state is constant; compare with stochastic games; see the chapter on ‘stochastic games’ in a forthcoming volume of this Handbook) and known to the players (the state is certain; compare with games of incomplete information, Chapters 5 and 6 in this Handbook)

Weighted reward criteria in competitive Markov decision processes

Abstract: We consider Competitive Markov Decision Processes in which the controllers/players are antagonistic and aggregate their sequences of expected rewards according to “weighted” or “horizonsensitive” criteria. These are either a convex combination of two discounted objectives, or of one discounted and one limiting average reward objective. In both cases we establish the existence of the game-theoretic value vector, and supply a description of 6-optimal non-stationary strategies.

Linear complementarity and discounted switching controller stochastic games

Abstract: The class of discounted switching controller stochastic games can be solved in one step by a linear complementarity program (LCP). Following the proof of this technical result is a discussion of a special formulation and initialization of a standard LCP pivoting algorithm which has, in numerical experiments, always terminated in a complementary solution. That the LCP algorithm as formulated always finds a complementary solution has not yet been proven, but these theoretical and experimental results have the potential to provide an alternative proof of the ordered field property for these games. Numerical experimentation with the reformulated LCP is reviewed.

A remark on the princess and Monster search game

Abstract: We consider two zero-sum search games in which a searcher moves along a continuous trajectory in a search setQ. The probability of detection depends on the distance between the two players. The problem is “open loop”, i.e. neither player receives any information about the other as the game progresses. The payoff to a hider is the elapsed time before detection. Optimal mixed strategies are obtained.

On the Non-Cooperative Foundations of Cooperative Bargaining

Abstract: In this note we challenge the non-cooperative foundations of cooperative bargaining solutions on the grounds that the limit operation for approaching a frictionless world is not robusto We show that when discounting almost ceases to play a role, any individually rational payoff can be supported by some subgame perfect equilibrium. To select the "correct" point imposes excessive informationaL requirements on the analyst.

Some time-invariant stopping rule problems

Abstract: Let X 1, X 2, … be an i.i.d. sequence. We consider three stopping rule problems for stopping the sequence of partial sums each of which has a time-invariance for the payoff that allows us to describe the optimal stopping rule in a particularly simple form, depending on one or two parameters. For certain distributions of the Xn , the optimal rules are found explicitly. The three problems are: (1) stopping with payoff equal to the absolute value of the sum with a cost of time , (2) stopping with payoff equal to the maximum of the partial sums with a cost of time , and (3) deciding when to give up trying to attain a goal or set a record . For each of these problems, the corresponding problems repeated in time, where the objective is to maximize the rate of return, can also be solved.