Showing papers on "Stochastic game published in 1991"

Evolutionary games in economics

Abstract: Evolutionary games are introduced as models for repeated anonymous strategic interaction: actions (or behaviors) which are more "fit," given the current distribution of behaviors, tend over time to displace less fit behaviors. Cone fields characterize the continuous-time processes compatible with a given fitness (or payoff) function. For large classes of dynamics, it is shown that all stable steady states are Nash equilibria and that all Nash equilibria are steady states. The biologists' evolutionarily stable strategy condition is shown to be less closely related to the dynamic equilibria. Economic examples and a literature survey are also provided. Copyright 1991 by The Econometric Society.

Chip-firing games on graphs

Abstract: We analyse the following (solitaire) game: each node of a graph contains a pile of chips, and a move consists of selecting a node with at least as many chips on it as its degree, and letting it send one chip to each of its neighbors. The game terminates if there is no such node. We show that the finiteness of the game and the terminating configuration are independent of the moves made. If the number of chips is less than the number of edges, the game is always finite. If the number of chips is at least the number of edges, the game can be infinite for an appropriately chosen initial configuration. If the number of chips is more than twice the number of edges minus the number of nodes, then the game is always infinite. The independence of the finiteness and the terminating position follows from simple but powerful ‘exchange properties’ of the sequences of legal moves, and from some general results on ‘antimatroids with repetition’, i.e. languages having these exchange properties. We relate the number of steps in a finite game to the least positive eigenvalue of the Laplace operator of the graph.

Optimal Learning by Experimentation

Abstract: This paper considers a problem of optimal learning by experimentation by a single decision maker. Most of the analysis is concerned with the characterisation of limit beliefs and actions. We take a two-stage approach to this problem: first, understand the case where the agent's payoff function is deterministic; then, address the additional issues arising when noise is present. Our analysis indicates that local properties of the payoff function (such as smoothness) are crucial in determining whether the agent eventually attains the true maximum payoff or not. The paper also makes a limited attempt at characterising optimal experimentation strategies.

Algorithms for stochastic games — A survey

Abstract: We consider finite state, finite action, stochastic games over an infinite time horizon. We survey algorithms for the computation of minimax optimal stationary strategies in the zerosum case, and of Nash equilibria in stationary strategies in the nonzerosum case. We also survey those theoretical results that pave the way towards future development of algorithms.

Computational methods for task-directed sensor data fusion and sensor planning

Abstract: In this article we consider the problem of task-directed information gathering. We first develop a decision-theo retic model of task-directed sensing in which sensors are modeled as noise-contaminated, uncertain measurement systems, and sensing tasks are inodeled by a transforma tion describing the type of information required by the task, a utility function describing sensitivity to error, and a cost function describing time or resource constraints on the system.This description allows us to develop a standard condi tional Bayes decision-making model where the value of information, or payoff, of an estimate is defined as the average utility (the expected value of some function of decision or estimation error) relative to the current proba bility distribution and the best estimate is that which max imizes payoff. The optimal sensor viewing strategy is that which maximizes the net payoff (decision value minus observation costs) of the final estimate. The advantage of this solution is generality—it does not...

Internal correlation in repeated games

Abstract: This paper characterizes the set of all the Nash equilibrium payoffs in two player repeated games where the signal that the players get after each stage is either trivial (does not reveal any information) or standard (the signal is the pair of actions played). It turns out that if the information is not always trivial then the set of all the Nash equilibrium payoffs coincides with the set of the correlated equilibrium payoffs. In particular, any correlated equilibrium payoff of the one shot game is also a Nash equilibrium payoff of the repeated game. For the proof we develop a scheme by which two players can generate any correlation device, using the signaling structure of the game. We present strategies with which the players internally correlate their actions without the need of an exogenous mediator.

Nonlinear programming and stationary equilibria in stochastic games

Abstract: Stationary equilibria in discounted and limiting average finite state/action space stochastic games are shown to be equivalent to global optima of certain nonlinear programs. For zero sum limiting average games, this formulation reduces to a linear objective, nonlinear constraints program, which finds the “best” stationary strategies, even whene-optimal stationary strategies do not exist, for arbitrarily smalle.

Learning automata approach to hierarchical multiobjective analysis

Abstract: A novel approach to hierarchical multiobjective analysis using the theory of learning automata is introduced. The problem is modeled as several hierarchies of automata involved in stochastic identical payoff games at the various levels. It is shown that if suitable learning algorithms are chosen at all the levels, the overall performance of the system will improve at each stage. The relevance of the model to multilevel optimization problems is illustrated by considering a simple problem of labeling images consistently. >

Demand Commitment Bargaining: - The Case Of Apex Games -

Abstract: An apex game is a bargaining situation in which there is one major (apex) player and n “minor” players. The only profitable coalitions contain either the apex player and any one of the minor players or else all of the minor players. The demand commitment model is a bargaining procedure, i.e. an extensive form game. This paper investigates the payoffs that result (as subgame perfect outcomes) for apex games when players use the demand commitment bargaining procedure. We show that whenever the apex player has the first move he forms a coalition with a minor player and obtains the fraction (n − 1)/n of the coalition’s value while his (minor-player) partner obtains the remaining 1/n. When a minor player has the first move he either forms a coalition with the apex player (and obtains 1/n) or else forms a coalition with all of the remaining minor players. When this minor-player coalition forms there are many subgame perfect payoff distributions. A refinement of subgame perfection is proposed and is shown to select a unique payoff distribution (1/n for each minor player) for the minor-player coalition.

Nonzero-Sum Stochastic Games

Abstract: Nonzero-sum discounted stochastic games have equilibrium strategies when the state space is uncountable

The evolution of intelligent decision making in gaming

Abstract: Intelligent decision making requires an ability to predict one's environment and respond in an optimal manner with respect to some underlying purpose. Decision making becomes more difficult when facing an intelligently interactive player who may be always cooperative, neutral, competitive, or change as required by the circumstance. Further difficulties are encountered when the other player's behavior can be determined only through interaction. Such problems can be addressed using a technique that simulates the logic of evolution. An environment is created that presents the prisoner's dilemma. “Organisms” that optimize behavior are evolved over generations. The results indicate that this “evolutionary programming” can be useful in interactive gaming with respect to arbitrary payoff functions.

A Perspective on Renegotiation in Repeated Games

Abstract: In this paper we discuss the conceptual foundations of one approach to modelling renegotiation in repeated games. Renegotiation-proof equilibria are viewed as social conventions that players continue to find beneficial after every history. The theory can be understood in terms of stationary stable sets of credible deviations.

Stochastic games with average payoff criterion

Abstract: We study two-person stochastic games on a Polish state and compact action spaces and with average payoff criterion under a certain ergodicity condition. For the zero-sum game we establish the existence of a value and stationary optimal strategies for both players. For the nonzero-sum case the existence of Nash equilibrium in stationary strategies is established under certain separability conditions.

On the Algorithm of Pollatschek and Avi-ltzhak

Abstract: We present a modification of the Pollatschek and Avi-Itzhak’s algorithm for solving the discounted (and terminating), zero-sum, stochastic games. We call our algorithm the Modified Newton’s Method and demonstrate that it always converges to the value-vector of the stochastic game, and from an arbitrary starting point. The step-size in our method is selected according to the well-known Armijo’s Rule.

Algorithms for Stochastic Games

Abstract: In this paper, we present algorithms for the solution of finite discounted stochastic games, without special structure. Three equilibrium concepts are considered: saddle points in two-person zero-sum games, Nash equilibrium points in N-person non-cooperative games and finally Stackelberg equilibrium in two-person games.

Crime and punishment: Further reflections on the counter­intuitive results of mixed equilibria games

Abstract: In a series of related articles, George Tsebelis (1989, 1990, 1991) challenges political theorists to rethink the foundations of policy analysis. His major critique of policy analyses, based on decision theory (where one individual decides in an inanimate but not certain environment) rather than on game theory (where one individual decides in an environment with other strategic individuals), has weathered the storm of commentaries made on his work. Tsebelis's argument, that payoff changes for one player do not affect the behavior of that player at a mixed-strategy equilibrium, holds in some cases but not in others. Whether changes in the payoffs of one player affect that player's behavior at a mixed-strategy equilibrium depends upon the type of linkages that exist between that player and the others in the frame or in a similar position in a game. In this note we have stated the general conditions under which changes in the payoffs of one player do not affect that player's behavior at equilibrium and when they do.

Making Money From Fair Games

Abstract: Suppose that a casino offers p fair games. Can we make money by playing the games (a) according to a fixed schedule, (b) using a strategy that depends on our wealth? The answers depend on the moments of the payoff distribution.

Easy initial states in stochastic games

Abstract: In this paper we deal with limiting average stochastic games with finite state and action spaces. For any nonzero-sum stochastic game of this type, there exists a subset of initial states for which an almost stationary ∈-equilibrium exists. For any zero-sum stochastic game there exists for each player a subset of initial states for which this player has an optimal stationary strategy.

Optimal control of the running max

Abstract: A class of stochastic control problems where the payoff depends on the running maximum of a diffusion process is described. Such processes are appealing models for physical processes that evolve in a continuous and increasing manner. Dynamic programming conditions of optimality for these nonstandard problems are investigated and applied to particular examples.

On the optimal pure strategy sets for a mixed missile guidance law synthesis

Abstract: The terminal phase of a missile-versus-aircraft engagement in a noise-corrupted environment is formulated as an imperfect information zero-sum differential game. The payoff of the game is the single shot kill probability of the missile to be maximized by its designer and minimized by the pilot of the evading aircraft. In the past, it was shown that in such a game the optimal strategy of the evader is mixed. The proposed formulation allows a mixed strategy for the pursuer. The mathematical basis for such an analysis is provided, and a constructive methodology for the synthesis of missile guidance laws based on the concept of mixed strategies is outlined. >

Non-Cooperative Dynamic Games with General Utility Functions

Abstract: The present paper is concerned with a general non-cooperative two-person dynamic game with Borel state and action spaces, non-Markovian transition law and with utility functions depending on the whole sequence of states and actions. The motivation for a general utility function is that in several problems in economic theory, additivity or separability of the utility function is a restrictive assumption and hard to justify, e.g. in problems of consumption and production choices over time and in the closely related problems of optimal economic growth. Dynamic games with additive utility functions have been introduced by Shapley [22] and have then been investigated by many authors (see the survey paper of Parthasarathy and Stern [16] or Kiienle [9]). In recent years several authors have considered dynamic games with more general utility functions, e.g. Sengupta [21], Iwamoto [7], Schal [19].

On Stochastic Games with Uncountable State and Action Spaces

Abstract: We consider stochastic games with uncountable state space and prove the existence of a Nash equilibrium in Markovian strategies, reached as a limit, in an appropriate sense, of finite horizon Markovian equilibria (the latter exist even with compact metric action spaces). Our approach is based on Glicksberg’s fixed-point theorem, basic measurable selection results and a generalized Fatou Lemma.

N-person differential games governed by semilinear stochastic evolution systems

Abstract: In this paper the problem ofN-person infinite-dimensional stochastic differential games governed by semilinear stochastic evolution control systems is discussed. First the minimax principle which is the necessary condition for the existence of open-loop Nash equilibrium is proved. Then the necessary and sufficient conditions of open-loop and closed-loop Nash equilibrium for linear quadratic infinite-dimensional stochastic differential games are derived.

Positive Stochastic Games and a Theorem of Ornstein

Abstract: Stochastic games were first formulated by Shapley in 1953. In his fundamental paper Shapley [13] established the existence of value and optimal stationary strategies for zero-sum β-discounted stochastic games with finitely many states and actions for the two players.

Existence of Correlated Weak Equilibria in Discounted Stochastic Games with General State Space

Abstract: This paper treats of nonzero-sum discounted stochastic games with general state space where the players are allowed to use correlated strategies. The concept of correlated weak equilibrium of Moulin and Vial (being an extension of the Nash equilibrium point) is adopted to such games. An existence theorem is proved for a class of discounted stochastic games with a Borel state space where stationary Nash equilibria are not known to exist.

On a duel with time lag and arbitrary accuracy functions

Abstract: This paper deals with a two-person zero-sum game called duel with the following structure: Each of two players I and II has a gun with one bullet and he can fire his bullet at any time in [0, 1], aiming at his opponent. If I or II fires at timex, he hits his opponent with probabilityp (x) orq(x), respectively. The gun of I is silent, and hence, II does not know whether his opponent has fired or not, and the gun of II is noisy with time lagt, wheret is a positive constant,i.e., if II fires at timex then I knows it at timex +t. Further, if I hits II without being hit himself before, the payoff is 1; if I is hit by II without hitting II before, the payoff is −1; if they hit each other at the same time or both survive, the payoff is 0. This paper gives optimal strategy for each player and the value of the game.

A Two-Person Repeated Bargaining Game with Long-Term Contracts

Abstract: Does a noncooperative equilibrium point necessarily lead to a Pareto efficient outcome in a supergame if binding agreements on actions are possible among players? We present a two-person repeated bargaining game in which players can negotiate for a long-term contract on their actions in the supergame model. We show that a subgame perfect equilibrium point of our game necessarily leads to a Pareto efficient outcome if the equilibrium strategies for both players have zero-memory. We also point out that the question above is answered negatively if the equilibrium strategies for players have complete memory.