Stochastic game

About: Stochastic game is a research topic. Over the lifetime, 9493 publications have been published within this topic receiving 202664 citations.

On the Complexity of Nash Equilibria and Other Fixed Points

Abstract: We reexamine what it means to compute Nash equilibria and, more generally, what it means to compute a fixed point of a given Brouwer function, and we investigate the complexity of the associated problems. Specifically, we study the complexity of the following problem: given a finite game, $\Gamma$, with 3 or more players, and given $\epsilon>0$, compute an approximation within $\epsilon$ of some (actual) Nash equilibrium. We show that approximation of an actual Nash equilibrium, even to within any nontrivial constant additive factor $\epsilon<1/2$ in just one desired coordinate, is at least as hard as the long-standing square-root sum problem, as well as a more general arithmetic circuit decision problem that characterizes P-time in a unit-cost model of computation with arbitrary precision rational arithmetic; thus, placing the approximation problem in P, or even NP, would resolve major open problems in the complexity of numerical computation. We show similar results for market equilibria: it is hard to estimate with any nontrivial accuracy the equilibrium prices in an exchange economy with a unique equilibrium, where the economy is given by explicit algebraic formulas for the excess demand functions. We define a class, FIXP, which captures search problems that can be cast as fixed point computation problems for functions represented by algebraic circuits (straight line programs) over basis $\{+,*,-,/,\max,\min\}$ with rational constants. We show that the (exact or approximate) computation of Nash equilibria for 3 or more players is complete for FIXP. The price equilibrium problem for exchange economies with algebraic demand functions is another FIXP-complete problem. We show that the piecewise linear fragment of FIXP equals PPAD. Many other problems in game theory, economics, and probability theory can be cast as fixed point problems for such algebraic functions. We discuss several important such problems: computing the value of Shapley's stochastic games and the simpler games of Condon, extinction probabilities of branching processes, probabilities of stochastic context-free grammars, and termination probabilities of recursive Markov chains. We show that for some of them, the approximation, or even exact computation, problem can be placed in PPAD, while for others, they are at least as hard as the square-root sum and arithmetic circuit decision problems.

Tug-of-war with noise: A game-theoretic view of the $p$-Laplacian

Abstract: Fix a bounded domain Ω⊂Rd, a continuous function F:∂Ω→R, and constants e>0 and 1

Learning to Cooperate via Policy Search

Abstract: Cooperative games are those in which both agents share the same payoff structure. Value-based reinforcement-learning algorithms, such as variants of Q-learning, have been applied to learning cooperative games, but they only apply when the game state is completely observable to both agents. Policy search methods are a reasonable alternative to value-based methods for partially observable environments. In this paper, we provide a gradient-based distributed policy-search method for cooperative games and compare the notion of local optimum to that of Nash equilibrium. We demonstrate the effectiveness of this method experimentally in a small, partially observable simulated soccer domain.

Asymptotically Optimal Importance Sampling and Stratification for Pricing Path‐Dependent Options

Abstract: This paper develops a variance reduction technique for Monte Carlo simulations of path-dependent options driven by high-dimensional Gaussian vectors. The method combines importance sampling based on a change of drift with stratified sampling along a small number of key dimensions. The change of drift is selected through a large deviations analysis and is shown to be optimal in an asymptotic sense. The drift selected has an interpretation as the path of the underlying state variables which maximizes the product of probability and payoff—the most important path. The directions used for stratified sampling are optimal for a quadratic approximation to the integrand or payoff function. Indeed, under differentiability assumptions our importance sampling method eliminates variability due to the linear part of the payoff function, and stratification eliminates much of the variability due to the quadratic part of the payoff. The two parts of the method are linked because the asymptotically optimal drift vector frequently provides a particularly effective direction for stratification. We illustrate the use of the method with path-dependent options, a stochastic volatility model, and interest rate derivatives. The method reveals novel features of the structure of their payoffs.

An Equilibrium-Point Interpretation of Stable Sets and a Proposed Alternative Definition

Abstract: The paper argues that the von Neumann-Morgenstern definition of stable sets is unsatisfactory because it neglects the destabilizing effect of indirect dominance relations. This argument is supported both by heuristic considerations and by construction of a bargaining game B(G), formalizing the bargaining process by which the players agree on their payoffs from an n-person cooperative game G. (G itself is assumed to be given in characteristic-function form allowing side payments.) The strategies ρi, the players use in this bargaining game will determine which payoff vectors x will be stationary, i.e., will have the property that, should such a payoff vector x be proposed to the players, all further bargaining will stop and x will be accepted as the outcome of the game. It will be suggested that a stable set should be defined as the set V of all stationary payoff vectors x, on the assumption that the players' bargaining strategies p, will form a canonical equilibrium point ρ in the bargaining game B(G). In ...