Showing papers in "Dynamic Games and Applications in 2017"

Two Numerical Approaches to Stationary Mean-Field Games

Abstract: Here, we consider numerical methods for stationary mean-field games (MFG) and investigate two classes of algorithms. The first one is a gradient-flow method based on the variational characterization of certain MFG. The second one uses monotonicity properties of MFG. We illustrate our methods with various examples, including one-dimensional periodic MFG, congestion problems, and higher-dimensional models.

Mean-Field-Game Model of Corruption

Abstract: A simple model of corruption that takes into account the effect of the interaction of a large number of agents by both rational decision making and myopic behavior is developed. Its stationary version turns out to be a rare example of an exactly solvable model of mean-field-game type. The results show clearly how the presence of interaction (including social norms) influences the spread of corruption by creating certain phase transition from one to three equilibria.

One-Way Flow Networks with Decreasing Returns to Linking

Abstract: This paper presents a model of noncooperative network formation in which the benefit of new links eventually decreases. Agents link with each other to gain information and update their links according to better-reply dynamics. We show that all equilibrium networks display a center-periphery structure and may be disconnected. The only equilibrium networks that satisfy some strictness conditions are constellations of starred wheels, where central agents form possibly several optimally sized disjoint wheels and peripheral agents are linked to one of the wheels from outside. In the long run, the system settles to such a network architecture. The main features of a constellation of starred wheels are reminiscent of some well-known real-world facts. Collections of smaller disjoint networks connecting only a few agents are more common than global networks connecting all the agents in a community. Also differences within a connected component such as the center and the periphery are often found.

Zero-Sum Pursuit-Evasion Differential Games with Many Objects: Survey of Publications

Abstract: If a pursuit game with many persons can be formalized in the framework of zero-sum differential games, then general methods can be applied to solve it. But difficulties arise connected with very high dimension of the phase vector when there are too many objects. Just due to this problem, special formulations and methods have been elaborated for conflict interaction of groups of objects. This paper is a survey of publications and results on group pursuit games.

Evasion Differential Game of Infinitely Many Evaders from Infinitely Many Pursuers in Hilbert Space

Abstract: We consider a simple motion evasion differential game of infinitely many evaders and infinitely many pursuers in Hilbert space \(\ell _2\). Control functions of the players are subjected to integral constraints. If the position of an evader never coincides with the position of any pursuer, then evasion is said to be possible. Problem is to find conditions of evasion. The main result of the paper is that if either (i) the total resource of evaders is greater than that of pursuers or (ii) the total resource of evaders is equal to that of pursuers and initial positions of all the evaders are not limit points for initial positions of the pursuers, then evasion is possible. Strategies for the evaders are constructed.

On a Class of Hybrid Differential Games

Abstract: This paper is intended to present a systematic application of the hybrid systems framework to differential games. A special class of bimodal linear-quadratic differential games is presented and illustrated with examples; two particular classes of switching rules, time-dependent and state-dependent switches are discussed. The main contribution of the paper consists in formulating necessary optimality conditions for determining optimal strategies in both cooperative and non-cooperative cases. A practically relevant hybrid differential game of pollution reduction is considered to illustrate the developed framework.

Evolutionary Games in Interacting Communities

Abstract: In this paper, we extend the evolutionary games framework by considering a population composed of communities with each having its set of strategies and payoff functions. Assuming that the interactions among the communities occur with different probabilities, we define new evolutionarily stable strategies (ESS) with different levels of stability against mutations. In particular, through the analysis of two-community two-strategy model, we derive the conditions of existence of ESSs under different levels of stability. We also study the evolutionary game dynamics both in its classic form and with delays. The delays may be strategic, i.e., associated with the strategies, spatial, i.e., associated with the communities, or spatial strategic. We apply our model to the Hawk–Dove game played in two communities with an asymmetric level of aggressiveness, and we characterize the regions of ESSs as function of the interaction probabilities and the parameters of the model. We also show through numerical examples how the delays and the game parameters affect the stability of the mixed ESS.

Fixation Probabilities of Strategies for Bimatrix Games in Finite Populations

Abstract: Recent developments in stochastic evolutionary game theory in finite populations yield insights that complement the conventional deterministic evolutionary game theory in infinite populations. However, most studies of stochastic evolutionary game theory have investigated dynamics of symmetric games, although not all social and biological phenomena are described by symmetric games, e.g., social interactions between individuals having conflicting preferences or different roles. In this paper, we describe the stochastic evolutionary dynamics of two-player $$2 \times 2$$ bimatrix games in finite populations. The stochastic process is modeled by a frequency-dependent Moran process without mutation. We obtained the fixation probability that the evolutionary dynamics starting from a given initial state converges to a specific absorbing state. Applying the formula to the ultimatum game, we show that evolutionary dynamics favors fairness. Furthermore, we present two novel concepts of stability for bimatrix games, based on our formula for the fixation probability, and demonstrate that one of the two serves as a criterion for equilibrium selection.

A Semi-Lagrangian Scheme for a Modified Version of the Hughes’ Model for Pedestrian Flow

Abstract: In this paper, we present a semi-Lagrangian scheme for a regularized version of the Hughes’ model for pedestrian flow. Hughes originally proposed a coupled nonlinear PDE system describing the evolution of a large pedestrian group trying to exit a domain as fast as possible. The original model corresponds to a system of a conservation law for the pedestrian density and an eikonal equation to determine the weighted distance to the exit. We consider this model in the presence of small diffusion and discuss the numerical analysis of the proposed semi-Lagrangian scheme. Furthermore, we illustrate the effect of small diffusion on the exit time with various numerical experiments.

Cooperative Advertising with Accrual Rate in a Dynamic Supply Chain

Abstract: We consider a supply chain in which a manufacturer stimulates his retailers investing more local advertising expenditures through a cooperative advertising program (Co-op). Co-op advertising programs usually involve two important contractual terms, a participation rate and an accrual rate. While previous literatures have discussed the participation rate excessively, they seldom study the role of the accrual rate in cooperative advertising. To investigate the impact of the accrual rate on cooperative advertising decisions, we develop a dynamic cooperative advertising model for a single manufacturer–single retailer supply chain that incorporates the participation rate and the accrual rate simultaneously. We derive the equilibrium co-op decisions of two channel members, including the manufacturer’s national advertising efforts and his participation rate, as well as the retailer’s local advertising expenditure. Our analysis of the equilibrium solutions shows that an increase in participation rate will not always increase the retailer’s advertising efforts because of the accrual rate. Also, both the manufacturer and retailer can benefit from a high accrual rate.

International Fisheries Agreements with a Shifting Stock

Abstract: When a fish stock shifts from one nation to another nation, e.g., due to climate change, the nation that loses the resource has incentives to deplete it, while the other nation, receiving the resource, has incentives to conserve it. We propose an analytical model to study under which circumstances self-enforcing agreements can align incentives. Our setup allows to distinguish between a fast and a slow shift and between a smooth or a sudden shift in ownership. We show that the shorter the expected duration of the transition, the higher the total equilibrium exploitation rate. Similarly, a sudden shift implies—by and large—more aggressive non-cooperative exploitation than a gradual shift. However, a self-enforcing agreement without side-payments is more likely for a sudden than for a smooth shift. Further, the scope for cooperation increases with the expected duration of the transition, and it decreases with the renewability of the resource and the discount rate. Most importantly, we show that concentrating on in-kind transfers can be very detrimental for shifting renewable resources: In some cases, there is no efficient bargaining solution without side-payments, even when there are only two players.

On Dynamic Games with Randomly Arriving Players

Abstract: We consider a dynamic game where additional players (assumed identical, even if there will be a mild departure from that hypothesis) join the game randomly according to a Bernoulli process. The problem solved here is that of computing their expected payoff as a function of time and the number of players present when they arrive, if the strategies are given. We consider both a finite horizon game and an infinite horizon, discounted game. As illustrations, we discuss some examples relating to oligopoly theory (Cournot, Stackelberg, cartel).

Aircraft Control During Cruise Flight in Windshear Conditions: Viability Approach

Abstract: This paper addresses the analysis of aircraft control capabilities during the cruise phase (flying at the established level with practically constant configuration and speed) in the presence of windshears. The study uses a point-mass aircraft model describing flight in a vertical plane. The problem is formulated as a differential game against wind disturbances. The first player, autopilot, controls the angle of attack and the power setting, whereas the second player, wind, produces dangerous gusts. The state variables of the model are subjected to constraints expressing aircraft safety conditions. Namely, the altitude, path inclination, and velocity are constrained. Viability theory is used to find the so-called viability kernel, the maximal subset of the state constraint where the aircraft trajectories can remain arbitrary long if the first player utilizes an appropriate feedback control, and the second player generates any admissible disturbances. The computations are based on grid methods developed by the authors and implemented on a multiprocessor computer system.

Large Population Aggregative Potential Games

Abstract: We consider population games in which payoff depends upon the aggregate strategy level and which admit a potential function. Examples of such aggregative potential games include the tragedy of the commons and the Cournot competition model. These games are technically simple as they can be analyzed using a one-dimensional variant of the potential function. We use such games to model the presence of externalities, both positive and negative. We characterize Nash equilibria in such games as socially inefficient. Evolutionary dynamics in such games converge to socially inefficient Nash equilibria.

Numerical Analysis of Markov-Perfect Equilibria with Multiple Stable Steady States: A Duopoly Application with Innovative Firms

Abstract: This paper presents a numerical method for the characterization of Markov-perfect equilibria of symmetric differential games exhibiting coexisting stable steady states. The method relying on the calculation of ‘local value functions’ through collocation in overlapping parts of the state space, is applicable for games with multiple state variables. It is applied to analyze a piecewise deterministic game capturing the dynamic competition between two oligopolistic firms, which are active in an established market and invest in R&D. Both R&D investment and an evolving public knowledge stock positively influence a breakthrough probability, where the breakthrough generates the option to introduce an innovative product on the market. Additionally, firms engage in activities influencing the appeal of the established and new product to consumers. Markov-perfect equilibrium profiles are numerically determined for different parameter settings and it is shown that for certain constellations the new product is introduced with probability one if the initial strength of the established market is below a threshold, which depends on the initial level of public knowledge. In case, the initial strength of the established market is above this threshold, and the R&D effort of both firms quickly goes to zero and with a high probability the new product is never introduced. Furthermore, it is shown that after the introduction of the new product the innovator engages in activities weakening the established market, although it is still producing positive quantities of that product.

Extremal Shift Rule for Continuous-Time Zero-Sum Markov Games

Abstract: In the paper we consider the controlled continuous-time Markov chain describing the interacting particles system with the finite number of types. The system is controlled by two players with the opposite purposes. This Markov game converges to a zero-sum differential game when the number of particles tends to infinity. Krasovskii–Subbotin extremal shift provides the optimal strategy in the limiting game. The main result of the paper is the near optimality of the Krasovskii–Subbotin extremal shift rule for the original Markov game.

Total Reward Semi-Markov Mean-Field Games with Complementarity Properties

Abstract: We study a class of dynamic games with a continuum of atomless players where each player controls a semi-Markov process of individual states, while the global state of the game is the aggregation of individual states of all the players. The model differs from standard models of dynamic games with continuum of players known as mean field or anonymous games in that the moments when the decisions are made are discrete, but different for each of the players. As a result, the individual states of each player follow a continuous time Markov chain, but the global state follows an ordinary differential equation. Games of this type were introduced by Gomes et al. (Appl Math Optim 68:99–143, 2013) and received some attention in the literature in last few years. In our paper we introduce a novel model of this type where players maximize their cumulative payoffs over their lifetime. We show that the payoffs of the players using any stationary strategy of a certain class in a game with continuum of players are close to those obtained in n-person counterparts of this game for n large enough. This implies that equilibrium strategies in the anonymous model can well approximate equilibria in related games with large finite number of players. In the rest of the paper we concentrate on a subclass of games where the payoff and transition probability functions exhibit some strategic complementarities between players. In that case we prove that the game possesses a stationary equilibrium. Moreover, largest and smallest equilibrium strategies are nondecreasing in the states. It also turns out that these equilibria can be well approximated using a distributed iterative procedure.

On the Evolution of Continuous Types Under Replicator and Gradient Dynamics: Two Examples

Abstract: This paper illustrates techniques for assessing the dynamic stability of games where a continuum of types might be present by re-analyzing two models under incomplete information, the Lohman et al. (Unpublished manuscript, 2001) public goods game and the Kopel et al. (J Econ Dyn Control 48:394–409, 2014) Cournot duopoly game. The evolution of continuous types follows either replicator dynamics Oechssler and Riedel (Econ Theory 17:141–162, 2001; J Econ Theory 107:223–252 2002) or gradient dynamics Friedman and Ostrov (J Math Econ 46:691–707, 2010; J Econ Theory 148:743–777, 2013). The techniques rely on a system of partial differential equations. Numerical solutions obtained through replicator and gradient dynamics highlight the differences and the similarities that arise under both approaches. In the public goods game, the dynamic system affects the stationary distribution of types while in the Cournot duopoly model, the types evolve to a single mass point regardless of the dynamics used. Lastly, these techniques allow us to endogenize the distribution of player types.

On Derivation and Evolutionary Classification of Social Dilemma Games

Abstract: We propose an axiomatic derivation and a classification of multiperson social dilemma games. For the two-person symmetric games, the axiomatization leads to three types of the social dilemmas only: the Prisoner’s Dilemma (PD) game, the Chicken (Snowdrift) game and the Stag Hunt game. The popular multiperson games, such as for example the N-person PD, the Public Goods, the Tragedy of the Commons, the Volunteer’s Dilemma and the Assurance game, are included in the proposed frame. For general social dilemma games with arbitrary payoffs, their simple classification is proposed, based on the number of stable equilibria of the corresponding replicator dynamics.

Option Fund Market Dynamics for Threshold Public Goods

Abstract: Economic agents have the possibility to fund the protection of environmental public goods, such as natural ecosystems and biodiversity, facing unknown risks of collapse, which could help to back them up. On the base of the prediction markets, which meet with a degree of success since their introduction, we propose an evolutionary model of an option fund market for the threshold environmental public goods. We consider population dynamics of agents distributed into proportional fair-share contributors and free riders. The model outcomes show that the public goods could be provided when the agents exchanging option contracts are equally divided into buyers and sellers. This result only holds for a specific social belief over the probability of the public good safeguard, the strict equality between bids and asks, and the equality of all payoffs. Otherwise, providing public goods through option markets turns out to be inoperative.

Sporadic Overtaking Optimality in Markov Decision Problems

Abstract: This paper examines a notion of sporadic overtaking optimality in the context of Markov decision problems (MDP). For the class of deterministic MDPs, we prove the existence of pure stationary sporadic overtaking optimal strategies under both the discounted and the average payoff evaluations. Moreover, we examine logical connections between sporadic overtaking optimality and Blackwell optimality. In the class of nondeterministic MDPs, we give examples that admit no sporadic overtaking optimal strategy and discuss a number of alternative definitions of this concept.

Pricing Strategies of Complementary Products in Distribution Channels: A Dynamic Approach

Abstract: This paper investigates the dynamic pricing strategies of firms selling complementary products in a marketing channel. The problem is modelled as a non-cooperative differential game that takes place between decisions makers controlling transfer and retail prices. We computed and compared prices and sales rates of channel members under two scenarios: (i) The first involves a single retailer that sells a unique brand produced by a monopolist manufacturer and (ii) in the second, a complementary product is introduced by an additional manufacturer. We found that in both scenarios, transfer and retail prices decrease over time, but prices decrease faster when the complementary product is introduced into the market. Furthermore, the entry of the complementary product onto the market boosts the sales rate of the existing product. Finally, we found that the retailer in the second scenario always has a non-negative retail margin, meaning that practicing a loss-leadership strategy is not optimal.

Approachability in Stackelberg Stochastic Games with Vector Costs

Abstract: The notion of approachability was introduced by Blackwell (Pac J Math 6(1):1–8, 1956) in the context of vector-valued repeated games. The famous ‘Blackwell’s approachability theorem’ prescribes a strategy for approachability, i.e., for ‘steering’ the average vector cost of a given agent toward a given target set, irrespective of the strategies of the other agents. In this paper, motivated by the multi-objective optimization/decision-making problems in dynamically changing environments, we address the approachability problem in Stackelberg stochastic games with vector-valued cost functions. We make two main contributions. Firstly, we give a simple and computationally tractable strategy for approachability for Stackelberg stochastic games along the lines of Blackwell’s. Secondly, we give a reinforcement learning algorithm for learning the approachable strategy when the transition kernel is unknown. We also recover as a by-product Blackwell’s necessary and sufficient conditions for approachability for convex sets in this setup and thus a complete characterization. We give sufficient conditions for non-convex sets.

Differential Terror Queue Games

Abstract: We present models of differential terror queue games, wherein terrorists seek to determine optimal attack rates over time, while simultaneously the government develops optimal counterterror staffing levels. The number of successful and interdicted terror attacks is determined via an underlying dynamic terror queue model. Different information structures and commitment abilities derive from different assumptions regarding what the players in the game can and cannot deduce about the underlying model. We compare and explain the impact of different information structures, i.e., open loop, closed loop, and asymmetric. We characterize the optimal controls for both the terrorists and the government in terms of the associated state and costate variables and deduce the costate equations that must be solved numerically to yield solutions to the game for the different cases. Using recently assembled data describing both terror attack and staffing levels, we compare the differential game models to each other as well as to the optimal control model of Seidl et al. (Eur J Oper Res 248:246–256, 2016). The paper concludes with a discussion of the lessons learned from the entire modeling exercise.

LQG Dynamic Games with a Control-Sharing Information Pattern

Abstract: “Zero-sum” linear-quadratic Gaussian dynamic games (LQGDGs) where the players have partial information are considered. The players’ initial state information and their measurements are private information, but each player is able to observe his antagonist’s past inputs: The protagonists’ past controls are shared information. Although this is a game with partial information, the control-sharing information pattern renders the game amenable to solution by Dynamic Programming. Three Riccati equations and a Lyapunov equation must be solved. The solution of LQGDGs with a control-sharing information pattern is obtained in closed-form.

A numerical algorithm to calculate the unique feedback nash equilibrium in a large scalar LQ differential game

Abstract: In this paper, we study scalar linear quadratic differential games with state feedback information structure. We present a numerical algorithm which determines whether this game will have no, one, or multiple equilibria. Furthermore, in case there is a unique equilibrium, the algorithm provides this equilibrium. The algorithm is efficient in the sense that it is capable of handling a large number of players. The analysis is restricted to the case the involved cost depend only on the state and control variables.

Intertemporal Versus Spatial Externalities in Counterterror Policy Games

Abstract: This paper studies a dynamic game between two national governments that fight a common terrorist organization that is seeking to mount a transnational terror campaign. It is the first examination that combines the temporal externalities associated with a sustained campaign with the spatial externalities that occur when the effects of one government’s counterterror policy spill over into another country. We consider two types of noncooperative behavior; one in which national authorities are sensitive to the reactions of the terrorists on foreign soil and another in which they are insensitive. It is shown that foreign terrorist sensitivity is preferred to insensitivity. Moreover, unilaterally accounting for terrorist reactions on foreign soil can be preferred to full policy coordination between governments. This then feeds into policy recommendations as to when each nation finds it desirable to coordinate transnational counterterror policy.

On nonzero-sum game considered on solutions of a hybrid system with frequent random jumps

Abstract: We study a non-zero sum game considered on the solutions of a hybrid dynamical system that evolves in continuous time and that is subjected to abrupt changes of parameters. The changes of the parameters are synchronized with (and determined by) the changes of the states/actions of two Markov decision processes, each of which is controlled by a player that aims at minimizing his or her objective function. The lengths of the time intervals between the " jumps " of the parameters are assumed to be small. We show that an asymptotic Nash equilibrium of such hybrid game can be constructed on the basis of a Nash equilibrium of a deterministic averaged dynamic game.

Cooperative and Noncooperative Extraction in a Common Pool with Habit Formation

Abstract: The present paper considers the exploitation of a common-property, nonrenewable resource, by individuals subject to habit formation. We formalize their behavior by means of a utility function, depending on the difference between the individuals’ current consumption and the consumption level which they aspire, the latter being a weighted average of past consumptions in the population. We derive and compare the benchmark cooperative solution and a noncooperative Markov-perfect Nash equilibrium of the differential game. We investigate how the intensity, persistence and initial level of habits shape the cooperative and noncooperative solutions. We prove that habit formation may either mitigate or worsen the tragedy of the commons.

Completely Mixed Strategies for Generalized Bimatrix and Switching Controller Stochastic Game

Abstract: In this paper, we revisit a result by Jurg et al. (Linear Algebra Appl 141:61–74, 1990) where the necessary and sufficient condition for a bimatrix game to be weakly completely mixed is given. We present an alternate proof of this result using linear complementarity approach. We extend this result to a generalization of bimatrix game introduced by Gowda and Sznajder (Int J Game Theory 25:1–12, 1996) via a generalization of linear complementarity problem introduced by Cottle and Dantzig (J Comb Theory 8:79–90, 1970). We further study completely mixed switching controller stochastic game (in which transition structure is a natural generalization of the single controller games) and extend the results obtained by Filar (Proc Am Math Soc 95:585–594, 1985) for completely mixed single controller stochastic game to completely mixed switching controller stochastic game. A numerical method is proposed to compute a completely mixed strategy for a switching controller stochastic game.