Showing papers on "Stochastic game published in 2006"

The Colonel Blotto game

Abstract: In the Colonel Blotto game, two players simultaneously distribute forces across n battlefields. Within each battlefield, the player that allocates the higher level of force wins. The payoff of the game is the proportion of wins on the individual battlefields. An equilibrium of the Colonel Blotto game is a pair of n-variate distributions. This paper characterizes the unique equilibrium payoffs for all (symmetric and asymmetric) configurations of the players’ aggregate levels of force, characterizes the complete set of equilibrium univariate marginal distributions for most of these configurations, and constructs entirely new and novel equilibrium n-variate distributions.

Cognition and Behavior in Two-Person Guessing Games: An Experimental Study

Abstract: This paper reports an experiment that elicits subjects' initial responses to 16 dominance-solvable two-person guessing games. The structure is publicly announced except for varying payoff parameters, to which subjects are given free access. Varying the parameters allows very strong separation of the behavior implied by leading decision rules. Subjects' decisions and searches show that most subjects understood the games and sought to maximize payoffs, but many had simplified models of others' decisions that led to systematic deviations from equilibrium. The predictable component of their deviations is well explained by a structural nonequilibrium model of initial responses based on level-k thinking.

Gaming machine with sorting feature

Abstract: A gaming machine comprises at least one visual display (mechanical or video) and a game of chance controlled by a processor in response to a wager. The game of chance includes a primary game and a sorting feature. The sorting feature is triggered by certain start-feature outcomes of the primary game. The sorting feature includes a collection of scrambled objects, such as letters, symbols, pictures, or puzzle pieces, that are at least partially sorted during operation of the sorting feature. The sorting feature generates an award, such as a payoff, a payoff multiplier, or extended play, if the sorted objects match predetermined criteria.

Memory-based snowdrift game on networks.

Abstract: We present a memory-based snowdrift game (MBSG) taking place on networks. We found that, when a lattice is taken to be the underlying structure, the transition of spatial patterns at some critical values of the payoff parameter is observable for both four- and eight-neighbor lattices. The transition points as well as the styles of spatial patterns can be explained by local stability analysis. In sharp contrast to previously reported results, cooperation is promoted by the spatial structure in the MBSG. Interestingly, we found that the frequency of cooperation of the MBSG on a scale-free network peaks at a specific value of the payoff parameter. This phenomenon indicates that properly encouraging selfish behaviors can optimally enhance the cooperation. The memory effects of individuals are discussed in detail and some non-monotonous phenomena are observed on both lattices and scale-free networks. Our work may shed some new light on the study of evolutionary games over networks.

Robust game theory

Abstract: We present a distribution-free model of incomplete-information games, both with and without private information, in which the players use a robust optimization approach to contend with payoff uncertainty. Our ``robust game'' model relaxes the assumptions of Harsanyi's Bayesian game model, and provides an alternative distribution-free equilibrium concept, which we call ``robust-optimization equilibrium,'' to that of the ex post equilibrium. We prove that the robust-optimization equilibria of an incomplete-information game subsume the ex post equilibria of the game and are, unlike the latter, guaranteed to exist when the game is finite and has bounded payoff uncertainty set. For arbitrary robust finite games with bounded polyhedral payoff uncertainty sets, we show that we can compute a robust-optimization equilibrium by methods analogous to those for identifying a Nash equilibrium of a finite game with complete information. In addition, we present computational results.

Collaborative Multiagent Reinforcement Learning by Payoff Propagation

Abstract: In this article we describe a set of scalable techniques for learning the behavior of a group of agents in a collaborative multiagent setting. As a basis we use the framework of coordination graphs of Guestrin, Koller, and Parr (2002a) which exploits the dependencies between agents to decompose the global payoff function into a sum of local terms. First, we deal with the single-state case and describe a payoff propagation algorithm that computes the individual actions that approximately maximize the global payoff function. The method can be viewed as the decision-making analogue of belief propagation in Bayesian networks. Second, we focus on learning the behavior of the agents in sequential decision-making tasks. We introduce different model-free reinforcement-learning techniques, unitedly called Sparse Cooperative Q-learning, which approximate the global action-value function based on the topology of a coordination graph, and perform updates using the contribution of the individual agents to the maximal global action value. The combined use of an edge-based decomposition of the action-value function and the payoff propagation algorithm for efficient action selection, result in an approach that scales only linearly in the problem size. We provide experimental evidence that our method outperforms related multiagent reinforcement-learning methods based on temporal differences.

Evolutionary games and population dynamics: maintenance of cooperation in public goods games

Abstract: The emergence and abundance of cooperation in nature poses a tenacious and challenging puzzle to evolutionary biology. Cooperative behaviour seems to contradict Darwinian evolution because altruistic individuals increase the fitness of other members of the population at a cost to themselves. Thus, in the absence of supporting mechanisms, cooperation should decrease and vanish, as predicted by classical models for cooperation in evolutionary game theory, such as the Prisoner’s Dilemma and public goods games. Traditional approaches to studying the problem of cooperation assume constant population sizes and thus neglect the ecology of the interacting individuals. Here, we incorporate ecological dynamics into evolutionary games and reveal a new mechanism for maintaining cooperation. In public goods games, cooperation can gain a foothold if the population density depends on the average population payoff. Decreasing population densities, due to defection leading to small payoffs, results in smaller interaction group sizes in which cooperation can be favoured. This feedback between ecological dynamics and game dynamics can generate stable coexistence of cooperators and defectors in public goods games. However, this mechanism fails for pairwise Prisoner’s Dilemma interactions and the population is driven to extinction. Our model represents natural extension of replicator dynamics to populations of varying densities.

Cooperation in the noisy case: Prisoner's dilemma game on two types of regular random graphs

Abstract: We have studied an evolutionary prisoner's dilemma game with players located on two types of random regular graphs with a degree of 4. The analysis is focused on the effects of payoffs and noise (temperature) on the maintenance of cooperation. When varying the noise level and/or the highest payoff, the system exhibits a second-order phase transition from a mixed state of cooperators and defectors to an absorbing state where only defectors remain alive. For the random regular graph (and Bethe lattice) the behavior of the system is similar to those found previously on the square lattice with nearest neighbor interactions, although the measure of cooperation is enhanced by the absence of loops in the connectivity structure. For low noise the optimal connectivity structure is built up from randomly connected triangles.

Coherence resonance in a spatial prisoner's dilemma game

Abstract: We study effects of additive spatiotemporal random variations, introduced to the payoffs of a spatial prisoner's dilemma game, on the evolution of cooperation. In the absence of explicit payoff variations the system exhibits a phase transition from a mixed state of cooperators and defectors to a homogenous state of defectors belonging to the directed percolation universality class. By introducing nonzero random variations to the payoffs, this phase transition can be reverted in a resonance-like manner depending on the variance of noise, thus marking coherence resonance in the system. We argue that explicit random payoff variations present a viable mechanism that promotes cooperation for defection temptation values substantially exceeding the one marking the transition point to homogeneity by deterministic payoffs.

Evolutionary game dynamics in a Wright-Fisher process

Abstract: Evolutionary game dynamics in finite populations can be described by a frequency dependent, stochastic Wright-Fisher process. We consider a symmetric game between two strategies, A and B. There are discrete generations. In each generation, individuals produce offspring proportional to their payoff. The next generation is sampled randomly from this pool of offspring. The total population size is constant. The resulting Markov process has two absorbing states corresponding to homogeneous populations of all A or all B. We quantify frequency dependent selection by comparing the absorption probabilities to the corresponding probabilities under random drift. We derive conditions for selection to favor one strategy or the other by using the concept of total positivity. In the limit of weak selection, we obtain the 1/3 law: if A and B are strict Nash equilibria then selection favors replacement of B by A, if the unstable equilibrium occurs at a frequency of A which is less than 1/3.

Unequal connections

Abstract: Empirical work suggests that social and economic networks are characterized by an unequal distribution of connections across individuals. This paper explores the circumstances under which networks will or will not exhibit inequality. Two specific models of network formation are explored. The first is a playing the field game in which the aggregate payoffs of an individual depend only on the number of his links and the aggregate number of links of the rest of the population. The second is a local spillovers game in which the aggregate payoffs of an individual depend on the distribution of links of all players and the identity of neighbors. For both class of games we develop results on existence and characterize equilibrium networks under different combinations of externalities/spillovers. We also examine conditions under which having more connections implies a higher payoff.

The Speed of Learning in Noisy Games: Partial Reinforcement and the Sustainability of Cooperation

Abstract: In an experiment, players’ ability to learn to cooperate in the repeated prisoner’s dilemma was substantially diminished when the payoffs were noisy, even though players could monitor one another's past actions perfectly. In contrast, in one-time play against a succession of opponents, noisy payoffs increased cooperation, by slowing the rate at which cooperation decays. These observations are consistent with the robust observation from the psychology literature that partial reinforcement (adding randomness to the link between an action and its consequences while holding expected payoffs constant) slows learning. This effect is magnified in the repeated game: When others are slow to learn to cooperate, the benefits of cooperation are reduced, which further hampers cooperation. These results show that a small change in the payoff environment, which changes the speed of individual learning, can have a large effect on collective behavior. And they show that there may be interesting comparative dynamics that can be derived from careful attention to the fact that at least some economic behavior is learned from experience.

Regret Minimization Under Partial Monitoring

Abstract: We consider repeated games in which the player, instead of observing the action chosen by the opponent in each game round, receives a feedback generated by the combined choice of the two players. We study Hannan-consistent players for these games, that is, randomized playing strategies whose per-round regret vanishes with probability one as the number n of game rounds goes to infinity. We prove a general lower bound of Ω(n-1/3) for the convergence rate of the regret, and exhibit a specific strategy that attains this rate for any game for which a Hannan-consistent player exists.

Double resonance in cooperation induced by noise and network variation for an evolutionary prisoner's dilemma

Abstract: We study effects of slowly varying small-world topology and additive spatiotemporal random variations, introduced to the payoffs of a spatial prisoner's dilemma game, on the evolution of cooperation. We show that there exists an optimal fraction of shortcut links, constituting the variable complex network of participating players of the game, for which noise-induced cooperation is resonantly enhanced, thus marking a double resonance phenomenon in the studied system. The double resonance is attributed to the time-dependence of the connectivity structure that induces a tendency towards the mean-field behaviour in the limit of random graphs. We argue that random payoff disturbances and complex network topology are two potent extrinsic factors able to boost cooperation, thus representing a viable escape hatch out of evolutionary stalemate.

Improved Second-Order Bounds for Prediction with Expert Advice

Abstract: This work studies external regret in sequential prediction games with both positive and negative payoffs. External regret measures the difference between the payoff obtained by the forecasting strategy and the payoff of the best action. In this setting, we derive new and sharper regret bounds for the well-known exponentially weighted average forecaster and for a new forecaster with a different multiplicative update rule. Our analysis has two main advantages: first, no preliminary knowledge about the payoff sequence is needed, not even its range; second, our bounds are expressed in terms of sums of squared payoffs, replacing larger first-order quantities appearing in previous bounds. In addition, our most refined bounds have the natural and desirable property of being stable under rescalings and general translations of the payoff sequence.

Effects of average degree on cooperation in networked evolutionary game

Abstract: We study effects of average degree on cooperation in the networked prisoner's dilemma game Typical structures are considered, including random networks, small-world networks and scale-free networks Simulation results show that the average degree plays a universal role in cooperation occurring on all these networks, that is the density of cooperators peaks at some specific values of the average degree Moreover, we investigated the average payoff of players through numerical simulations together with theoretical predictions and found that simulation results agree with the predictions Our work may be helpful in understanding network effects on the evolutionary games

Extensive games with possibly unaware players

Abstract: Standard game theory assumes that the structure of the game is common knowledge among players. We relax this assumption by considering extensive games where agents may be unaware of the complete structure of the game. In particular, they may not be aware of moves that they and other agents can make. We show how such games can be represented; the key idea is to describe the game from the point of view of every agent at every node of the game tree. We provide a generalization of Nash equilibrium and show that every game with awareness has a generalized Nash equilibrium. Finally, we extend these results to games with awareness of unawareness, where a player i may be aware that a player j can make moves that i is not aware of.

Pay One or Pay All:Random Selection of One Choice for Payment

Abstract: It has become increasingly common in economics experiments to elicit a series of choices from participants, and then pay for only one, selected at random, after all have been made. This allows the researcher to observe a large number of individual decisions, and to increase payoffs for each decision since only one of them will be used for payment. It has not been demonstrated, however, whether subjects behave as if each of these choices involves the stated payoffs, or if subjects scale-down payoffs to account for the random selection that is made. This paper reports an experiment that tests this directly. In an environment where payoff scale effects have been demonstrated to matter, three treatments are conducted: pay for one of 10 choices under low payoffs, pay for all 10 choices under low payoffs, and pay for 1 of 10 choices under 10x the low payoff level. Increasing payoff scale has a significant effect on choices compared with the low payoff treatments where all 10 decisions are paid, or where one decision is paid. However, there is no significant difference in choices between paying for one or all 10 decisions at the low payoff level. This supports the validity of using the random-choice payment method.

Evolutionary and dynamical coherence resonances in the pair approximated prisoner's dilemma game

Abstract: Stochasticity has recently emerged as being a potent promoter of cooperative behaviour in systems developed under the framework of evolutionary game theory. In the spatial prisoner's dilemma game, the fitness of players adopting the cooperative strategy was found to be resonantly dependent on the intensity of payoff fluctuations. Evidently, the phenomenon resembles classical coherence resonance, whereby the noise-induced order, or coherence, of the dynamics is substituted with the noise-induced prevalence of the 'good' strategy, thus marking a constructive effect of noise on the system. The connection between the former 'dynamical' coherence resonance and the latter so-called 'evolutionary' coherence resonance, however, has not yet been established. The two different definitions of coherence resonance appear to provoke some discomfort. The goal of the present paper is therefore, on one hand, to draw a clear line between the two different perceptions of coherence resonance, and on the other, to show that the two apparently disjoint phenomena, that are currently related only by name, can in fact be observed simultaneously, sharing an identical mechanism of emergence.

Market sharing games applied to content distribution in ad hoc networks

Abstract: In third-generation (3G) wireless data networks, repeated requests for popular data items can exacerbate the already scarce wireless spectrum. In this paper, we propose an architectural and protocol framework that allows 3G service providers to host efficient content distribution services. We offload the spectrum intensive task of content distribution to an ad hoc network. Less mobile users (resident subscribers) are provided incentives to cache popular data items, while mobile users (transit subscribers) access this data from resident subscribers through the ad hoc network. Since the participants of this data distribution network act as selfish agents, they may collude to maximize their individual payoff. Our proposed protocol discourages potential collusion scenarios. In this architecture, the goal (social function) of the 3G service provider is to have the selfishly motivated resident subscribers service as many data requests as possible. However, the choice of which set of items to cache is left to the individual user. The caching activity among the different users can be modeled as a market sharing game. In this work, we study the Nash equilibria of market sharing games and the performance of such equilibria in terms of a social function. These games are a special case of congestion games that have been studied in the economics literature. In particular, pure strategy Nash equilibria for this set of games exist. We give a polynomial-time algorithm to find a pure strategy Nash equilibrium for a special case, while it is NP-hard to do so in the general case. As for the performance of Nash equilibria, we show that the price of anarchy-the worst case ratio between the social function at any Nash equilibrium and at the social optimum-can be upper bounded by a factor of 2. When the popularity follows a Zipf distribution, the price of anarchy is bounded by 1.45 in the special case where caching any item has a positive reward for all players. We prove that the selfish behavior of computationally bounded agents converges to an approximate Nash equilibrium in a finite number of improvements. Furthermore, we prove that, after each agent computes its response function once using a constant factor approximation algorithm, the outcome of the game is within a factor of O(logn) of the optimal social value, where n is the number of agents. Our simulation scenarios show that the price of anarchy is 30% better than that of the worst case analysis and that the system quickly (1 or 2 steps) converges to a Nash equilibrium.

A game-theoretic study of CSMA/CA under a backoff attack

Abstract: CSMA/CA, the contention mechanism of the IEEE 802.11 DCF medium access protocol, has recently been found vulnerable to selfish backoff attacks consisting in nonstandard configuration of the constituent backoff scheme. Such attacks can greatly increase a selfish station's bandwidth share at the expense of honest stations applying a standard configuration. The paper investigates the distribution of bandwidth among anonymous network stations, some of which are selfish. A station's obtained bandwidth share is regarded as a payoff in a noncooperative CSMA/CA game. Regardless of the IEEE 802.11 parameter setting, the payoff function is found similar to a multiplayer Prisoners' Dilemma; moreover, the number (though not the identities) of selfish stations can be inferred by observation of successful transmission attempts. Further, a repeated CSMA/CA game is defined, where a station can toggle between standard and nonstandard backoff configurations with a view of maximizing a long-term utility. It is argued that a desirable station strategy should yield a fair, Pareto efficient, and subgame perfect Nash equilibrium. One such strategy, called CRISP, is described and evaluated.

John Maynard Smith and the importance of consistency in evolutionary game theory

Abstract: John Maynard Smith was the founder of evolutionary game theory. He has also been the major influence on the direction of this field, which now pervades behavioural ecology and evolutionary biology. In its original formulation the theory had three components: a set of strategies, a payoff structure, and a concept of evolutionary stability. These three key components are still the basis of the theory, but what is assumed about each component is often different to the original assumptions. We review modern approaches to these components. We emphasis that if a game is considered in isolation, and arbitrary payoffs are assumed, then the payoffs may not be consistent with other components of the system which are not modelled. Modelling the whole system, including not only the focal game, but also the future behaviour of the players and the behaviour of other population members, allows a consistent model to be constructed. We illustrate this in the case of two models of parental care, showing how linking a focal game to other aspects of the system alters what is predicted.

Mixed Zero-Sum Stochastic Differential Game and American Game Options

Abstract: In this paper we solve the mixed zero-sum stochastic differential game problem in the general case. The main tool is the notion of a local solution of backward stochastic differential equations (BSDEs) with two reflecting barriers. As an application we deal with the American game options.

Pay One or Pay All: Random Selection of One Choice for Payment

Abstract: It has become increasingly common in economics experiments to elicit a series of choices from participants, and then pay for only one, selected at random, after all have been made. This allows the researcher to observe a large number of individual decisions, and to increase payoffs for each decision since only one of them will be used for payment. It has not been demonstrated, however, whether subjects behave as if each of these choices involves the stated payoffs, or if subjects scale-down payoffs to account for the random selection that is made. This paper reports an experiment that tests this directly. In an environment where payoff scale effects have been demonstrated to matter, three treatments are conducted: pay for one of 10 choices under low payoffs, pay for all 10 choices under low payoffs, and pay for 1 of 10 choices under 10x the low payoff level. Increasing payoff scale has a significant effect on choices compared with the low payoff treatments where all 10 decisions are paid, or where one decision is paid. However, there is no significant difference in choices between paying for one or all 10 decisions at the low payoff level. This supports the validity of using the random-choice payment method.

The Value of Markov Chain Games with Lack of Information on One Side

Abstract: We consider a two-player zero-sum game, given by a Markov chain over a finite set of states and a family of matrix games indexed by states. The sequence of states follows the Markov chain. At the beginning of each stage, only Player 1 is informed of the current state, then the corresponding matrix game is played, and the actions chosen are observed by both players before proceeding to the next stage. We call such a game a Markov chain game with lack of information on one side. This model generalizes the model of Aumann and Maschler of zero-sum repeated games with lack of information on one side (which corresponds to the case where the transition matrix of the Markov chain is the identity matrix). We generalize the proof of Aumann and Maschler and, from the definition and the study of appropriate nonrevealing auxiliary games with infinitely many stages, show the existence of the uniform value. An important difference with Aumann and Maschler's model is that here the notions for Player 1 of using the information and revealing a relevant information are distinct.

Polynomial algorithms for approximating nash equilibria of bimatrix games

Abstract: We focus on the problem of computing an e-Nash equilibrium of a bimatrix game, when e is an absolute constant. We present a simple algorithm for computing a $\frac{3}{4}$-Nash equilibrium for any bimatrix game in strongly polynomial time and we next show how to extend this algorithm so as to obtain a (potentially stronger) parameterized approximation. Namely, we present an algorithm that computes a $\frac{2+\lambda}{4}$-Nash equilibrium, where λ is the minimum, among all Nash equilibria, expected payoff of either player. The suggested algorithm runs in time polynomial in the number of strategies available to the players.

Finding All Nash Equilibria of a Finite Game Using Polynomial Algebra

Abstract: The set of Nash equilibria of a finite game is the set of nonnegative solutions to a system of polynomial equations. In this survey article we describe how to construct certain special games and explain how to find all the complex roots of the corresponding polynomial systems, including all the Nash equilibria. We then explain how to find all the complex roots of the polynomial systems for arbitrary generic games, by polyhedral homotopy continuation starting from the solutions to the specially constructed games. We describe the use of Groebner bases to solve these polynomial systems and to learn geometric information about how the solution set varies with the payoff functions. Finally, we review the use of the Gambit software package to find all Nash equilibria of a finite game.