Showing papers on "Stochastic game published in 2005"

Who's Who in Networks: Wanted - The Key Player

Abstract: Finite population non-cooperative games with linear-quadratic utilities, where each player decides how much action she exerts, can be interpreted as a network game with local payoff complementarities, together with a globally uniform payoff substitutability component and an own concavity effect. For these games, the Nash equilibrium action of each player is proportional to her Bonacich centrality in the network of local complementarities, thus establishing a bridge with the sociology literature on social networks. This Bonacich-Nash linkage implies that aggregate equilibrium increases with network size and density. We then analyze a policy that consists in targeting the key player, that is, the player who, once removed, leads to the optimal change in aggregate activity. We provide a geometric characterization of the key player identified with an inter-centrality measure, which takes into account both a player's centrality and her contribution to the centrality of the others.

Models of cooperation based on the Prisoner's Dilemma and the Snowdrift game

Abstract: Understanding the mechanisms that can lead to the evolution of cooperation through natural selection is a core problem in biology. Among the various attempts at constructing a theory of cooperation, game theory has played a central role. Here, we review models of cooperation that are based on two simple games: the Prisoner's Dilemma, and the Snowdrift game. Both games are two-person games with two strategies, to cooperate and to defect, and both games are social dilemmas. In social dilemmas, cooperation is prone to exploitation by defectors, and the average payoff in populations at evolutionary equilibrium is lower than it would be in populations consisting of only cooperators. The difference between the games is that cooperation is not maintained in the Prisoner's Dilemma, but persists in the Snowdrift game at an intermediate frequency. As a consequence, insights gained from studying extensions of the two games differ substantially. We review the most salient results obtained from extensions such as iteration, spatial structure, continuously variable cooperative investments, and multi-person interactions. Bridging the gap between theoretical and empirical research is one of the main challenges for future studies of cooperation, and we conclude by pointing out a number of promising natural systems in which the theory can be tested experimentally.

SYNTHESES Models of cooperation based on the Prisoner's Dilemma and the Snowdrift game

Abstract: Understanding the mechanisms that can lead to the evolution of cooperation through natural selection is a core problem in biology. Among the various attempts at constructing a theory of cooperation, game theory has played a central role. Here, we review models of cooperation that are based on two simple games: the Prisoner’s Dilemma, and the Snowdrift game. Both games are two-person games with two strategies, to cooperate and to defect, and both games are social dilemmas. In social dilemmas, cooperation is prone to exploitation by defectors, and the average payoff in populations at evolutionary equilibrium is lower than it would be in populations consisting of only cooperators. The difference between the games is that cooperation is not maintained in the Prisoner’s Dilemma, but persists in the Snowdrift game at an intermediate frequency. As a consequence, insights gained from studying extensions of the two games differ substantially. We review the most salient results obtained from extensions such as iteration, spatial structure, continuously variable cooperative investments, and multi-person interactions. Bridging the gap between theoretical and empirical research is one of the main challenges for future studies of cooperation, and we conclude by pointing out a number of promising natural systems in which the theory can be tested experimentally.

On Adaptation, Maximization, and Reinforcement Learning among Cognitive Strategies.

Abstract: Analysis of binary choice behavior in iterated tasks with immediate feedback reveals robust deviations from maximization that can be described as indications of 3 effects: (a) a payoff variability effect, in which high payoff variability seems to move choice behavior toward random choice; (b) underweighting of rare events, in which alternatives that yield the best payoffs most of the time are attractive even when they are associated with a lower expected return; and (c) loss aversion, in which alternatives that minimize the probability of losses can be more attractive than those that maximize expected payoffs. The results are closer to probability matching than to maximization. Best approximation is provided with a model of reinforcement learning among cognitive strategies (RELACS). This model captures the 3 deviations, the learning curves, and the effect of information on uncertainty avoidance. It outperforms other models in fitting the data and in predicting behavior in other experiments.

Game strategies in network security

Abstract: This paper presents a game-theoretic method for analyzing the security of computer networks. We view the interactions between an attacker and the administrator as a two-player stochastic game and construct a model for the game. Using a nonlinear program, we compute Nash equilibria or best-response strategies for the players (attacker and administrator). We then explain why the strategies are realistic and how administrators can use these results to enhance the security of their network.

Cooperation, social networks, and the emergence of leadership in a prisoner's dilemma with adaptive local interactions.

Abstract: Cooperative behavior among a group of agents is studied assuming adaptive interactions. Each agent plays a Prisoner's Dilemma game with its local neighbors, collects an aggregate payoff, and imitates the strategy of its best neighbor. Agents may punish or reward their neighbors by removing or sustaining the interactions, according to their satisfaction level and strategy played. An agent may dismiss an interaction, and the corresponding neighbor is replaced by another randomly chosen agent, introducing diversity and evolution to the network structure. We perform an extensive numerical and analytical study, extending results in M. G. Zimmermann, V. M. Eguiluz, and M. San Miguel, Phys. Rev. E 69, 065102(R) (2004). We show that the system typically reaches either a full-defective state or a highly cooperative steady state. The latter equilibrium solution is composed mostly by cooperative agents, with a minor population of defectors that exploit the cooperators. It is shown how the network adaptation dynamics favors the emergence of cooperators with the highest payoff. These "leaders" are shown to sustain the global cooperative steady state. Also we find that the average payoff of defectors is larger than the average payoff of cooperators. Whenever "leaders" are perturbed (e.g., by addition of noise), an unstable situation arises and global cascades with oscillations between the nearly full defection network and the fully cooperative outcome are observed.

Strategic bidding of transmission-constrained GENCOs with incomplete information

Abstract: This work describes a method for analyzing the competition among transmission-constrained generating companies (GENCOs) with incomplete information. Each GENCO models its opponents' unknown information with specific types for transforming the incomplete game into a complete game with imperfect information. The proposed methodology employs the supply function equilibrium for modeling a GENCO's bidding strategy. The competition is modeled as a bilevel problem with the upper subproblem representing individual GENCOs and the lower subproblem representing the independent system operator (ISO). The upper subproblem maximizes the individual GENCOs' payoffs and the lower subproblem solves the ISO's market clearing problem for minimizing consumers' payments. The bilevel problem is solved by developing sensitivity functions for a GENCO's payoff with respect to its bidding strategies. An eight-bus system is employed to illustrate the proposed method, and the numerical results show the impact of transfer capability on GENCOs' bidding strategies.

Strategic Experimentation with Exponential Bandits

Abstract: This paper studies a game of strategic experimentation with two-armed bandits whose risky arm might yield a payoff only after some exponentially distributed random time. Because of free-riding, there is an inefficiently low level of experimentation in any equilibrium where the players use stationary Markovian strategies with posterior beliefs as the state variable. After characterizing the unique symmetric Markovian equilibrium of the game, which is in mixed strategies, we construct a variety of pure-strategy equilibria. There is no equilibrium where all players use simple cut-off strategies. Equilibria where players switch finitely often between the roles of experimenter and free-rider all lead to the same pattern of information acquisition; the efficiency of these equilibria depends on the way players share the burden of experimentation among them. In equilibria where players switch roles infinitely often, they can acquire an approximately efficient amount of information, but the rate at which it is acquired still remains inefficient; moreover, the expected payoff of an experimenter exhibits the novel feature that it rises as players become more pessimistic. Finally, over the range of beliefs where players use both arms a positive fraction of the time, the symmetric equilibrium is dominated by any asymmetric one in terms of aggregate payoffs.

Dynamic Psychological Games

Abstract: Building on recent work on dynamic interactive epistemology, we extend the analysis of extensive-form psychological games (Geneakoplos, Pearce & Stacchetti, Games and Economic Behavior, 1989) to include conditional higher-order beliefs and enlarged domains of payoff functions. The approach allows modeling dynamic psychological effects (such as sequential reciprocity, psychological forward induction, and regret) that are ruled out when epistemic types are identified with hierarchies of initial beliefs. We define a notion of psychological sequential equilibrium, which generalizes the sequential equilibrium notion for traditional games, for which we prove existence under mild assumptions. Our framework also allows us to directly formulate assumptions about 'dynamic' rationality and interactive beliefs in order to explore strategic interaction without assuming that players beliefs are coordinated on an equilibrium. In particular, we provide an exploration of (extensive-form) rationalizability in psychological games.

On the nature of reciprocal motives

Abstract: I. INTRODUCTION The most widely applied models in economics and game theory are based on the assumption of "self-regarding preferences," which are characterized by an exclusive concern about one's own material payoff. Models of self-regarding preferences capture behavior quite well in many contexts, including double auctions as in Smith (1982) and Davis and Holt (1993), one-sided auctions with independent private values as in Cox and Oaxaca (1996), procurement contracting as in Cox et al. (1996), and search as in Cox and Oaxaca (1989, 2000), Harrison and Morgan (1990), and Cason and Friedman (2003). But there is now a large body of literature that reports systematic inconsistencies with the implications of the self-regarding preferences model. (1) These replicable patterns of behavior are observed in experimental games involving decisions about the division of material payoffs among the participants. One explanation for the observed behavior that has received considerable attention is reciprocity. We report experiments designed to yield insight into the nature and robustness of reciprocal motives. By observing decisions in a group of related experiments we are able to discriminate between behavior motivated by reciprocity and behavior motivated by nonreciprocal other-regarding preferences over outcomes. Some treatments introduce the possibility of behavior motivated by positive reciprocity, whereas other treatments introduce the possibility of negatively reciprocal motivation. By "positive reciprocity" we mean a motivation to adopt a generous action that benefits someone else because that person's intentional behavior was perceived to be beneficial to oneself within the decision context of the experiment. Similarly, by "negative reciprocity" we mean a motivation to adopt a costly action that harms someone else because that person's intentional behavior was perceived to be harmful to oneself within the decision context of the experiment. Perhaps the most familiar experiment in the reciprocity literature is the ultimatum game. In this game, the first mover proposes a division of a fixed sum of money and the second mover either accepts this proposal or vetoes it. In the event of a veto, both players get a money payoff of zero. The self-regarding preferences model predicts extremely unequal payoffs for this game, with the first mover offering the second mover the smallest feasible positive amount of money and the second mover accepting this offer. However, observed behavior in the ultimatum game contrasts sharply with these predictions. Under a wide variety of conditions, first movers in ultimatum games tend to propose relatively equal splits. This has been observed by Guth et al. (1982), Hoffman and Spitzer (1985), Hoffman et al. (1994), and Bornstein and Yaniv (1998). (2) First movers may make generous proposals in ultimatum games because they have inequality-averse other-regarding preferences, as suggested by Fehr and Schmidt (1999) and Bolton and Ockenfels (2000), or altruistic other-regarding preferences, as suggested by Cox and Sadiraj (2005). Alternatively, first movers may make generous proposals because they are afraid that second movers will veto lopsided proposals. Second movers may veto such proposals because of inequality-averse preferences over outcomes or because of negative reciprocity. The implications for modeling behavior are different if the behavior is motivated by preferences over outcomes that are unconditional on perceived intentions than if it is motivated by negative reciprocity or fear of negative reciprocity. To discriminate among alternative motivations, we use a triadic experimental design that includes a mini-ultimatum game, which we call the Punishment mini-ultimatum game (Punishment-MUG), and two dictator control treatments. (3) Additional insight into the nature of alternative motives is gained from a systematic comparison of our data with data from the different experimental design of Falk et al. …

Legislative Bargaining under Weighted Voting

Abstract: Organizations often distribute resources through weighted voting. We analyze this setting using a non-cooperative bargaining game based on the Baron-Ferejohn (1989) model. Unlike analyses derived from cooperative game theory, we find that each voter's expected payoff is proportional to her voting weight. An exception occurs when many high-weight voters exist, as low-weight voters may expect disproportionately high payoffs due to proposal power. The model also predicts that, ex post, the coalition formateur (the party chosen to form a coalition) will receive a disproportionately high payoff. Using data from coalition governments from 1946 to 2001, we find strong evidence ofsuch formateur effects.

Comparison of basic assumptions embedded in learning models for experience-based decision making.

Abstract: The present study examined basic assumptions embedded in learning models for predicting behavior in decisions based on experience. In such decisions, the probabilities and payoffs are initially unknown and are learned from repeated choice with payoff feedback. We examined combinations of two rules for updating past experience with new payoff feedback and of two choice rule assumptions for mapping experience onto choices. The combination of these assumptions produced four classes of models that were systematically compared. Two methods were employed to evaluate the success of learning models for approximating players' choices: One was based on estimating parameters from each person's data to maximize the prediction of choices one step ahead, conditioned by the observed past history of feedback. The second was based on making a priori predictions for the entire sequence of choices using parameters estimated from a separate experiment. The results indicated the advantage of a class of models incorporating decay of previous experience, whereas the ranking of choice rules depended on the evaluation method used.

Recursive markov decision processes and recursive stochastic games

Abstract: We introduce Recursive Markov Decision Processes (RMDPs) and Recursive Simple Stochastic Games (RSSGs), and study the decidability and complexity of algorithms for their analysis and verification. These models extend Recursive Markov Chains (RMCs), introduced in [EY05a, EY05b] as a natural model for verification of probabilistic procedural programs and related systems involving both recursion and probabilistic behavior. RMCs define a class of denumerable Markov chains with a rich theory generalizing that of stochastic context-free grammars and multi-type branching processes, and they are also intimately related to probabilistic pushdown systems. RMDPs & RSSGs extend RMCs with one controller or two adversarial players, respectively. Such extensions are useful for modeling nondeterministic and concurrent behavior, as well as modeling a system’s interactions with an environment. We provide upper and lower bounds for deciding, given an RMDP (or RSSG) A and probability p, whether player 1 has a strategy to force termination at a desired exit with probability at least p. We also address “qualitative” termination, where p=1, and model checking questions.

Evolutionary prisoner's dilemma game on hierarchical lattices.

Abstract: An evolutionary prisoner's dilemma (PD) game is studied with players located on a hierarchical structure of layered square lattices. The players can follow two strategies [ D (defector) and C (cooperator)] and their income comes from PD games with the "neighbors." The adoption of one of the neighboring strategies is allowed with a probability dependent on the payoff difference. Monte Carlo simulations are performed to study how the measure of cooperation is affected by the number of hierarchical levels (Q) and by the temptation to defect. According to the simulations the highest frequency of cooperation can be observed at the top level if the number of hierarchical levels is low (Q<4) . For larger Q , however, the highest frequency of cooperators occurs in the middle layers. The four-level hierarchical structure provides the highest average (total) income for the whole community.

Fairness in bargaining

Abstract: We consider new three player games to test existing models of fairness. Our games consist of a proposer who offers an allocation of $10 between two players, either himself and the responder or the responder and a third party. In each case, the responder either accepts or rejects this allocation. In case of a rejection, the player who was not part of the initial division (the third party and the proposer, respectively) receives a rejection payoff (of $0, $5 or $10, depending on the game). Our results cast some doubt on existing fairness theories.

How individuals learn to take turns: emergence of alternating cooperation in a congestion game and the prisoner's dilemma

Abstract: In many social dilemmas, individuals tend to generate a situation with low payoffs instead of a system optimum ("tragedy of the commons"). Is the routing of traffic a similar problem? In order to address this question, we present experimental results on humans playing a route choice game in a computer laboratory, which allow one to study decision behavior in repeated games beyond the Prisoner's Dilemma. We will focus on whether individuals manage to find a cooperative and fair solution compatible with the system-optimal road usage. We find that individuals tend towards a user equilibrium with equal travel times in the beginning. However, after many iterations, they often establish a coherent oscillatory behavior, as taking turns performs better than applying pure or mixed strategies. The resulting behavior is fair and compatible with system-optimal road usage. In spite of the complex dynamics leading to coordinated oscillations, we have identified mathematical relationships quantifying the observed transition process. Our main experimental discoveries for 2- and 4-person games can be explained with a novel reinforcement learning model for an arbitrary number of persons, which is based on past experience and trial-and-error behavior. Gains in the average payoff seem to be an important driving force for the innovation of time-dependent response patterns, i.e. the evolution of more complex strategies. Our findings are relevant for decision support systems and routing in traffic or data networks.

How individuals learn to take turns: Emergence of alternating cooperation in a congestion game and the prisoner's dilemma

Abstract: In many social dilemmas, individuals tend to generate a situation with low payoffs instead of a system optimum ("tragedy of the commons"). Is the routing of traffic a similar problem? In order to address this question, we present experimental results on humans playing a route choice game in a computer laboratory, which allow one to study decision behavior in repeated games beyond the Prisoner's Dilemma. We will focus on whether individuals manage to find a cooperative and fair solution compatible with the system-optimal road usage. We find that individuals tend towards a user equilibrium with equal travel times in the beginning. However, after many iterations, they often establish a coherent oscillatory behavior, as taking turns performs better than applying pure or mixed strategies. The resulting behavior is fair and compatible with system-optimal road usage. In spite of the complex dynamics leading to coordinated oscillations, we have identified mathematical relationships quantifying the observed transition process. Our main experimental discoveries for 2- and 4-person games can be explained with a novel reinforcement learning model for an arbitrary number of persons, which is based on past experience and trial-and-error behavior. Gains in the average payoff seem to be an important driving force for the innovation of time-dependent response patterns, i.e. the evolution of more complex strategies. Our findings are relevant for decision support systems and routing in traffic or data networks.

The generalized Stackelberg equilibrium of the all-pay auction with complete information

Abstract: We characterize the equilibrium of the all-pay auction with general convex cost of effort and sequential effort choices. We consider a set of n players who are arbitrarily partitioned into a group of players who choose their efforts ’early’ and a group of players who choose ’late’. Only the player with the lowest cost of effort has a positive payoff in any equilibrium. This payoff depends on his own timing vis-a-vis the timing of others. We also show that the choice of timing can be endogenized, in which case the strongest player typically chooses ’late’, whereas all other players are indifferent with respect to their choice of timing. In the most prominent equilibrium the player with the lowest cost of effort wins the auction at zero aggregate cost.

The Expected Number of Nash Equilibria of a Normal Form Game

Abstract: Fix finite pure strategy sets S1,..., S,,, and let S = S, x... x S-n. In our model of a random game the agents' payoffs are statistically independent, with each agent's payoff uniformly distributed on the unit sphere in R-A. For given nonempty T-1 subset of S-1,..., T-n subset of S-n we give a computationally implementable formula for the mean number of Nash equilibria in which each agent i's mixed strategy has support T-i. The formula is the product of two expressions. The first is the expected number of totally mixed equilibria for the truncated game obtained by eliminating pure strategies outside the sets Ti. The second may be construed as the "probability" that such an equilibrium remains an equilibrium when the strategies in the sets S-i\T-i become available.

Games where you can play optimally without any memory

Abstract: Reactive systems are often modelled as two person antagonistic games where one player represents the system while his adversary represents the environment. Undoubtedly, the most popular games in this context are parity games and their cousins (Rabin, Streett and Muller games). Recently however also games with other types of payments, like discounted or mean-payoff [5,6], previously used only in economic context, entered into the area of system modelling and verification. The most outstanding property of parity, mean-payoff and discounted games is the existence of optimal positional (memoryless) strategies for both players. This observation raises two questions: (1) can we characterise the family of payoff mappings for which there always exist optimal positional strategies for both players and (2) are there other payoff mappings with practical or theoretical interest and admitting optimal positional strategies. This paper provides a complete answer to the first question by presenting a simple necessary and sufficient condition on payoff mapping guaranteeing the existence of optimal positional strategies. As a corollary to this result we show the following remarkable property of payoff mappings: if both players have optimal positional strategies when playing solitary one-player games then also they have optimal positional strategies for two-player games.

Marginal contributions and externalities in the value

Abstract: For games in partition function form, we explore the implications of distinguishing between the concepts of intrinsic marginal contributions and externalities. If one requires efficiency for the grand coalition, we provide several results concerning extensions of the Shapley value. Using the axioms of efficiency, anonymity, marginality and monotonicity, we provide upper and lower bounds to players' payoffs when affected by external effects, and a characterization of an ''externality-free'' value. If the grand coalition does not form, we characterize a payoff configuration on the basis of the principle of balanced contributions. We also analyze a game of coalition formation that yields sharp predictions

The Value of Zero-Sum Stopping Games in Continuous Time

Abstract: We study two-player zero-sum stopping games in continuous time and infinite horizon. We prove that the value in randomized stopping times exists as soon as the payoff processes are right-continuous. In particular, as opposed to existing literature, we do not assume any conditions on the relations between the payoff processes.

A stress test of fairness measures in models of social utility

Abstract: Current social utility models posit fairness as a motive for certain types of strategic behavior. The models differ, however, with respect to how fairness is measured. Distribution models measure fairness in terms of relative payoff comparisons. Reciprocal-kindness models measure fairness in terms of gifts given and gifts received. Reference points play an important role in both measures, but the reference points in reciprocal-kindness models are conditioned on the actions available to players, whereas those in distributive models are not. Data from an ultimatum game experiment that stress tests the kindness measure is consistent with the distributive measure. Data from an experiment that stress tests the distributive measure is inconsistent with the distributive measure, but moves in the direction opposite that implied by the kindness measure. A measure that combines relative payoff comparisons with a reference point conditioned on feasible actions provides a first approximation to our data.

Complexity and competition

Abstract: Extensive-form market games typically have a large number of noncompetitive equilibria. In this paper, we argue that the complexity of noncompetitive behavior provides a justification for competitive equilibrium in the sense that if rational agents have an aversion to complexity (at the margin), then maximizing behavior will result in simple behavioral rules and hence in a competitive outcome. For this purpose, we use a class of extensive-form dynamic matching and bargaining games with a finite number of agents. In particular, we consider markets with heterogeneous buyers and sellers and deterministic, exogenous, sequential matching rules, although the results can be extended to other matching processes. If the complexity costs of implementing strategies enter players' preferences lexicographically with the standard payoff, then every equilibrium strategy profile induces a competitive outcome.

Using Stochastic Game Theory to Compute the Expected Behavior of Attackers

Abstract: This paper presents ongoing work on using stochastic game theory as a mathematical tool for computing the expected behavior of attackers. The possible use of the Nash Equilibrium as a part of the transition probabilities in state transition models is defined and motivated. To demonstrate the approach, a simple example of an attack against a computer network is modelled and analyzed.

The Generalized Stackelberg Equilibrium of the All-Pay Auction with Complete Information

Abstract: We characterize the equilibrium of the all-pay auction with general convex cost of effort and sequential effort choices. We consider a set of n players who are arbitrarily partitioned into a group of players who choose their efforts 'early' and a group of players who choose 'late.' Only the player with the lowest cost of effort has a positive payoff in any equilibrium. This payoff depends on his own timing vis-a-vis the timing of others. We also show that the choice of timing can be endogenized, in which case the strongest player typically chooses 'late,' whereas all other players are indifferent with respect to their choice of timing. In the most prominent equilibrium the player with the lowest cost of effort wins the auction at zero aggregate cost.