Showing papers on "Stochastic game published in 2008"

Stochastic games with lossy channels

Abstract: We consider turn-based stochastic games on infinite graphs induced by game probabilistic lossy channel systems (GPLCS), the game version of probabilistic lossy channel systems (PLCS). We study games with Buchi (repeated reachability) objectives and almost-sure winning conditions. These games are pure memoryless determined and, under the assumption that the target set is regular, a symbolic representation of the set of winning states for each player can be effectively constructed. Thus, turn-based stochastic games on GPLCS are decidable. This generalizes the decidability result for PLCS-induced Markov decision processes in [10].

Competitive spectrum sharing in cognitive radio networks: a dynamic game approach

Abstract: "Cognitive radio" is an emerging technique to improve the utilization of radio frequency spectrum in wireless networks. In this paper, we consider the problem of spectrum sharing among a primary user and multiple secondary users. We formulate this problem as an oligopoly market competition and use a noncooperative game to obtain the spectrum allocation for secondary users. Nash equilibrium is considered as the solution of this game. We first present the formulation of a static game for the case where all secondary users have the current information of the adopted strategies and the payoff of each other. However, this assumption may not be realistic in some cognitive radio systems. Therefore, we consider the case of bounded rationality in which the secondary users gradually and iteratively adjust their strategies based on the observations on their previous strategies. The speed of adjustment of the strategies is controlled by the learning rate. The stability condition of the dynamic behavior for this spectrum sharing scheme is investigated. The numerical results reveal the dynamics of distributed dynamic adaptation of spectrum sharing strategies.

Deal or No Deal? Decision-making under Risk in a Large-payoff Game Show

Abstract: We examine the risky choices of contestants in the popular TV game show “Deal or No Deal” and related classroom experiments Contrary to the traditional view of expected utility theory, the choices can be explained in large part by previous outcomes experienced during the game Risk aversion decreases after earlier expectations have been shattered by unfavorable outcomes or surpassed by favorable outcomes Our results point to reference-dependent choice theories such as prospect theory, and suggest that path-dependence is relevant, even when the choice problems are simple and well-defined, and when large real monetary amounts are at stake

Multi-Armed Bandits in Metric Spaces

Abstract: In a multi-armed bandit problem, an online algorithm chooses from a set of strategies in a sequence of trials so as to maximize the total payoff of the chosen strategies. While the performance of bandit algorithms with a small finite strategy set is quite well understood, bandit problems with large strategy sets are still a topic of very active investigation, motivated by practical applications such as online auctions and web advertisement. The goal of such research is to identify broad and natural classes of strategy sets and payoff functions which enable the design of efficient solutions. In this work we study a very general setting for the multi-armed bandit problem in which the strategies form a metric space, and the payoff function satisfies a Lipschitz condition with respect to the metric. We refer to this problem as the "Lipschitz MAB problem". We present a complete solution for the multi-armed problem in this setting. That is, for every metric space (L,X) we define an isometry invariant which bounds from below the performance of Lipschitz MAB algorithms for X, and we present an algorithm which comes arbitrarily close to meeting this bound. Furthermore, our technique gives even better results for benign payoff functions.

Promotion of cooperation induced by appropriate payoff aspirations in a small-world networked game.

Abstract: Based on learning theory, we adopt a stochastic learning updating rule to investigate the evolution of cooperation in the Prisoner's Dilemma game on Newman-Watts small-world networks with different payoff aspiration levels. Interestingly, simulation results show that the mechanism of intermediate aspiration promoting cooperation resembles a resonancelike behavior, and there exists a ping-pong vibration of cooperation for large payoff aspiration. To explain the nontrivial dependence of the cooperation level on the aspiration level, we investigate the fractions of links, provide analytical results of the cooperation level, and find that the simulation results are in close agreement with analytical ones. Our work may be helpful in understanding the cooperative behavior induced by the aspiration level in society.

Coevolution of teaching activity promotes cooperation

Abstract: Evolutionary games are studied where the teaching activity of players can evolve in time. Initially all players following either the cooperative or defecting strategy are distributed on a square lattice. The rate of strategy adoption is determined by the payoff difference and a teaching activity characterizing the donor's capability to enforce its strategy on the opponent. Each successful strategy adoption process is accompanied by an increase in the donor's teaching activity. By applying an optimum value of the increment, this simple mechanism spontaneously creates relevant inhomogeneities in the teaching activities that support the maintenance of cooperation for both the prisoner's dilemma and the snowdrift game.

Towards effective payoffs in the prisoner's dilemma game on scale-free networks

Abstract: We study the transition towards effective payoffs in the prisoner’s dilemma game on scale-free networks by introducing a normalization parameter guiding the system from accumulated payoffs to payoffs normalized with the connectivity of each agent. We show that during this transition the heterogeneity-based ability of scale-free networks to facilitate cooperative behavior deteriorates continuously, eventually collapsing with the results obtained on regular graphs. The strategy donations and adaptation probabilities of agents with different connectivities are studied. Results reveal that strategies generally spread from agents with larger towards agents with smaller degree. However, this strategy adoption flow reverses sharply in the fully normalized payoff limit. Surprisingly, cooperators occupy the hubs even if the averaged cooperation level due to partly normalized payoffs is moderate.

Coevolution of teaching activity promotes cooperation

Abstract: Evolutionary games are studied where the teaching activity of players can evolve in time. Initially all players following either the cooperative or defecting strategy are distributed on a square lattice. The rate of strategy adoption is determined by the payoff difference and a teaching activity characterizing the donor's capability to enforce its strategy on the opponent. Each successful strategy adoption process is accompanied with an increase in the donor's teaching activity. By applying an optimum value of the increment this simple mechanism spontaneously creates relevant inhomogeneities in the teaching activities that support the maintenance of cooperation for both the prisoner's dilemma and the snowdrift game.

Stochastic Differential Games and Viscosity Solutions of Hamilton-Jacobi-Bellman-Isaacs Equations

Abstract: In this paper we study zero-sum two-player stochastic differential games with the help of the theory of backward stochastic differential equations (BSDEs). More precisely, we generalize the results of the pioneering work of Fleming and Souganidis [Indiana Univ. Math. J., 38 (1989), pp. 293-314] by considering cost functionals defined by controlled BSDEs and by allowing the admissible control processes to depend on events occurring before the beginning of the game. This extension of the class of admissible control processes has the consequence that the cost functionals become random variables. However, by making use of a Girsanov transformation argument, which is new in this context, we prove that the upper and the lower value functions of the game remain deterministic. Apart from the fact that this extension of the class of admissible control processes is quite natural and reflects the behavior of the players who always use the maximum of available information, its combination with BSDE methods, in particular that of the notion of stochastic “backward semigroups" introduced by Peng [BSDE and stochastic optimizations, in Topics in Stochastic Analysis, Science Press, Beijing, 1997], allows us then to prove a dynamic programming principle for both the upper and the lower value functions of the game in a straightforward way. The upper and the lower value functions are then shown to be the unique viscosity solutions of the upper and the lower Hamilton-Jacobi-Bellman-Isaacs equations, respectively. For this Peng's BSDE method is extended from the framework of stochastic control theory into that of stochastic differential games.

The Power of Focal Points Is Limited: Even Minute Payoff Asymmetry May Yield Large Coordination Failures

Abstract: Since Schelling, it has often been assumed that players make use of salient decision labels to achieve coordination. Consistent with previous work, we find that given equal payoffs, salient labels yield frequent coordination. However, given even minutely asymmetric payoffs, labels lose much of their effectiveness and miscoordination abounds. This raises questions about the extent to which the effectiveness of focal points based on label salience persists beyond the special case of symmetric games. The patterns of miscoordination we observe vary with the magnitude of payoff differences in intricate ways that suggest nonequilibrium accounts based on "level-k" thinking and "team reasoning." (JEL C12, C92)

Tug-of-war with noise: A game-theoretic view of the $p$-Laplacian

Abstract: Fix a bounded domain Ω⊂Rd, a continuous function F:∂Ω→R, and constants e>0 and 1

Asymptotically Optimal Decentralized Control for Large Population Stochastic Multiagent Systems

Abstract: The interaction of interest-coupled decision-makers and the uncertainty of individual behavior are prominent characteristics of multiagent systems (MAS). How to break through the framework of conventional control theory, which aims at single decision-maker and single decision objective, and to extend the methodology and tools in the stochastic adaptive control theory to analyze MAS are of great significance. In this paper, a preliminary exploration is made in this direction, and the decentralized control problem is considered for large population stochastic MAS with coupled cost functions. Different from the deterministic discounted costs in the existing differential game models, a time-averaged stochastic cost function is adopted for each agent. The decentralized control law is constructed based on the state aggregation method and tracking-like quadratic optimal control. By using probability limit theory, the stability and optimality of the closed-loop system are analyzed. The main contributions of this paper include the following points. 1) The concepts of asymptotic Nash-equilibrium in probability and almost surely, respectively, are introduced and the relationship between these concepts is illuminated, which provide necessary tools for analyzing the optimality of the decentralized control laws. 2) The closed-loop system is shown to be almost surely uniformly stable, and bounded independently of the number of agents N . 3) The population state average (PSA) is shown to converge to the infinite population mean (IPM) trajectory in the sense of both L2-norm and time average almost surely, as N increases to infinity. 4) The decentralized control law is designed and shown to be almost surely asymptotically optimal; the cost of each agent based on local measurements converges to that based on global measurements almost surely, as N increases to infinity.

Decision-Theoretic and Game-Theoretic Approaches to IT Security Investment

Abstract: Firms have been increasing their information technology (IT) security budgets significantly to deal with increased security threats. An examination of current practices reveals that managers view security investment as any other and use traditional decision-theoretic risk management techniques to determine security investments. We argue in this paper that this method is incomplete because of the problem's strategic nature-hackers alter their hacking strategies in response to a firm's investment strategies. We propose game theory for determining IT security investment levels and compare game theory and decision theory approaches on several dimensions such as the investment levels, vulnerability, and payoff from investments. We show that the sequential game results in the maximum payoff to the firm, but requires that the firm move first before the hacker. Even if a simultaneous game is played, the firm enjoys a higher payoff than that in the decision theory approach, except when the firm's estimate of the hacker effort in the decision theory approach is sufficiently close to the actual hacker effort. We also show that if the firm learns from prior observations of hacker effort and uses these to estimate future hacker effort in the decision theory approach, then the gap between the results of decision theory and game theory approaches diminishes over time. The rate of convergence and the extent of loss the firm suffers before convergence depend on the learning model employed by the firm to estimate hacker effort.

Fast convergence to nearly optimal solutions in potential games

Abstract: We study the speed of convergence of decentralized dynamics to approximately optimal solutions in potential games. We consider α-Nash dynamics in which a player makes a move if the improvement in his payoff is more than an α factor of his own payoff. Despite the known polynomial convergence of α-Nash dynamics to approximate Nash equilibria in symmetric congestion games [7], it has been shown that the convergence time to approximate Nash equilibria in asymmetric congestion games is exponential [25]. In contrast to this negative result, and as the main result of this paper, we show that for asymmetric congestion games with linear and polynomial delay functions, the convergence time of α-Nash dynamics to an approximate optimal solution is polynomial in the number of players, with approximation ratio that is arbitrarily close to the price of anarchy of the game. In particular, we show this polynomial convergence under the minimal liveness assumption that each player gets at least one chance to move in every T steps. We also prove that the same polynomial convergence result does not hold for (exact) best-response dynamics, showing the α-Nash dynamics is required. We extend these results for congestion games to other potential games including weighted congestion games with linear delay functions, cut games (also called party affiliation games) and market sharing games.

Evolutionary prisoner's dilemma game on Newman-Watts networks.

Abstract: Maintenance of cooperation was studied for a two-strategy evolutionary prisoner's dilemma game where the players are located on a one-dimensional chain and their payoff comes from games with the nearest- and next-nearest-neighbor interactions. The applied host geometry makes it possible to study the impacts of two conflicting topological features. The evolutionary rule involves some noise affecting the strategy adoptions between the interacting players. Using Monte Carlo simulations and the extended versions of dynamical mean-field theory we determined the phase diagram as a function of noise level and a payoff parameter. The peculiar feature of the diagram is changed significantly when the connectivity structure is extended by extra links as suggested by Newman and Watts.

Strategic Communication Networks

Abstract: In this paper, we consider situations in which individuals want to choose an action close to others' actions as well as close to a payoff relevant state of nature with the ideal proximity to the common state varying across the agents. Before this coordination game with heterogeneous preferences is played, a cheap talk communication stage is offered to players who decide to whom they reveal the private information they hold about the state. The strategic information transmission taking place in the communication stage is characterized by a strategic communication network. We provide a direct link between players' preferences and the strategic communication network emerging at equilibrium, depending on the strength of the coordination motive and the prior information structure. Equilibrium strategic communication networks are characterized in a very tractable way and compared in term of efficiency. In general, a maximal strategic communication network may not exist and communication networks cannot be ordered in the sense of Pareto. However, expected social welfare always increases when the communication network expands. Strategic information transmission can be improved when group or public communication is allowed, and/or when information is certifiable.

Transversality of the Shapley value

Abstract: A few applications of the Shapley value are described The main choice criterion is to look at quite diversified fields, to appreciate how wide is the terrain that has been explored and colonized using this and related tools

An Optimization Approach for Approximate Nash Equilibria

Abstract: In this paper we propose a new methodology for determining approximate Nash equilibria of noncooperative bimatrix games, and based on that, we provide an efficient algorithm that computes 0.3393-approximate equilibria, the best approximation to date. The methodology is based on the formulation of an appropriate function of pairs of mixed strategies reflecting the maximum deviation of the players' payoffs from the best payoff each player could achieve given the strategy chosen by the other. We then seek to minimize such a function using descent procedures. Because it is unlikely to be able to find global minima in polynomial time, given the recently proven intractability of the problem, we concentrate on the computation of stationary points and prove that they can be approximated arbitrarily closely in polynomial time and that they have the above-mentioned approximation property. Our result provides the best e to date for polynomially computable e-approximate Nash equilibria of bimatrix games. Furthermore,...

Individual differences in ewa learning with partial payoff information

Abstract: We extend experience-weighted attraction (EWA) learning to games in which only the set of possible foregone payoffs from unchosen strategies are known, and estimate parameters separately for each player to study heterogeneity We assume players estimate unknown foregone payoffs from a strategy, by substituting the last payoff actually received from that strategy, by clairvoyantly guessing the actual foregone payoff, or by averaging the set of possible foregone payoffs conditional on the actual outcomes All three assumptions improve predictive accuracy of EWA Individual parameter estimates suggest that players cluster into two separate subgroups (which differ from traditional reinforcement and belief learning)

A Dynamic Principal-Agent Model with Hidden Information: Sequential Optimality Through Truthful State Revelation

Abstract: This paper proposes a general framework for a large class of multiperiod principal-agent problems. In this framework, a principal has a primary stake in the performance of a system but delegates its control to an agent. The underlying system is a Markov decision process, where the state of the system can only be observed by the agent but the agent's action is observed by both parties. This paper develops a dynamic programming algorithm to derive optimal long-term contracts for the principal. The principal indirectly controls the underlying system by offering the agent a menu of continuation utility vectors along public information paths; the agent's best response, expressed in his choice of continuation utilities, induces truthful state revelation and results in actions that maximize the principal's expected payoff. This problem is meaningful to the operations research community because it can be framed as the problem of optimally designing the reward structure of a Markov decision process with hidden states and has many applications of interest as discussed in this paper.

Bayesian stackelberg games and their application for security at Los Angeles international airport

Abstract: Many multiagent settings are appropriately modeled as Stackelberg games [Fudenberg and Tirole 1991; Paruchuri et al. 2007], where a leader commits to a strategy first, and then a follower selfishly optimizes its own reward, considering the strategy chosen by the leader. Stackelberg games are commonly used to model attacker-defender scenarios in security domains [Brown et al. 2006] as well as in patrolling [Paruchuri et al. 2007; Paruchuri et al. 2008]. For example, security personnel patrolling an infrastructure commit to a patrolling strategy first, before their adversaries act taking this committed strategy into account. Indeed, Stackelberg games are being used at the Los Angeles International Airport to schedule security checkpoints and canine patrols [Murr 2007; Paruchuri et al. 2008; Pita et al. 2008a]. They could potentially be used in network routing, pricing in transportation systems and many other situations [Korilis et al. 1997; Cardinal et al. 2005]. Although the follower in a Stackelberg game is allowed to observe the leader’s strategy before choosing its own strategy, there is often an advantage for the leader over the case where both players must choose their moves simultaneously. To see the advantage of being the leader in a Stackelberg game, consider the game with the payoff as shown in Table I. The leader is the row player and the follower is the column player. The only pure-strategy Nash equilibrium for this game is when the leader plays a and the follower plays c which gives the leader a payoff of 2. However, if the leader commits to a mixed strategy of playing a and b with equal (0.5) probability, then the follower will play d, leading to an expected payoff for the leader of 3.5.

Cooperation Between Multiple Newsvendors with Warehouses

Abstract: This study considers a supply chain that consists of n retailers, each facing a newsvendor problem, and m warehouses. The retailers are supplied with a single product via some warehouses. In these warehouses, the ordered amounts of goods of these retailers become available after some lead time. At the time that the goods arrive at the warehouses, demand realizations are known by the retailers. The retailers can increase their expected joint profits if they can coordinate their orders and make allocations after demand realization. For this setting, we consider an associated cooperative game between the retailers. We show that this associated cooperative game has a nonempty core. Finally, we introduce a noncooperative game, where the retailers decide on their order quantities individually, and show that the set of payoff vectors resulting from strong Nash equilibria corresponds to the core of the associated cooperative game.

Phase transition and hysteresis loop in structured games with global updating

Abstract: We present a global payoff-based strategy updating model for studying cooperative behavior of a networked population. We adopt the Prisoner's Dilemma game and the snowdrift game as paradigms for characterizing the interactions among individuals. We investigate the model on regular, small-world, and scale-free networks, and find multistable cooperation states depending on the initial cooperator density. In particular for the snowdrift game on small-world and scale-free networks, there exist a discontinuous phase transition and hysteresis loops of cooperator density. We explain the observed properties by theoretical predictions and simulation results of the average number of neighbors of cooperators and defectors, respectively. Our work indicates that individuals with more neighbors have a trend to preserve their initial strategies, which has strong impacts on the strategy updating of individuals with fewer neighbors; while the fact that individuals with few neighbors have to become cooperators to avoid gaining the lowest payoff plays significant roles in maintaining and spreading of cooperation strategy.

Norm emergence under constrained interactions in diverse societies

Abstract: Effective norms, emerging from sustained individual interactions over time, can complement societal rules and significantly enhance performance of individual agents and agent societies Researchers have used a model that supports the emergence of social norms via learning from interaction experiences where each interaction is viewed as a stage game In this social learning model, which is distinct from an agent learning from repeated interactions against the same player, an agent learns a policy to play the game from repeated interactions with multiple learning agents The key research question is to characterize when and how the entire population of homogeneous learners converge to a consistent norm when multiple action combinations yield the same optimal payoff In this paper we study two extensions to the social learning model that significantly enhances its applicability We first explore the effects of heterogeneous populations where different agents may be using different learning algorithms We also investigate norm emergence when agent interactions are physically constrained We consider agents located on a grid where an agent is more likely to interact with other agents situated closer to it than those that are situated afar The key new results include the surprising acceleration in learning with limited interaction ranges We also study the effects of pure-strategy players, ie, nonlearners in the environment

Conservation laws and asymptotic behavior of a model of social dynamics

Abstract: A conservative social dynamics model is developed within a discrete kinetic framework for active particles, which has been proposed in [M.L. Bertotti, L. Delitala, From discrete kinetic and stochastic game theory to modelling complex systems in applied sciences, Math. Mod. Meth. Appl. Sci. 14 (2004) 1061–1084]. The model concerns a society in which individuals, distinguished by a scalar variable (the activity) which expresses their social state, undergo competitive and/or cooperative interactions. The evolution of the discrete probability distribution over the social state is described by a system of nonlinear ordinary differential equations. The asymptotic trend of their solutions is investigated both analytically and computationally. Existence, stability and attractivity of certain equilibria are proved.