Proceedings Article•
Finding optimal strategies for imperfect information games
01 Jul 1998-pp 500-507
TL;DR: These algorithms theoretically and experimentally are compared using both simple game trees and a large database of problems from the game of Bridge, showing that the new algorithms both out-perform Monte-carlo sampling, with the superiority of payoff-reduction minimaxing being especially marked.
Abstract: We examine three heuristic algorithms for games with imperfect information: Monte-carlo sampling, and two new algorithms we call vector minimaxing and payoff-reduction minimaxing. We compare these algorithms theoretically and experimentally, using both simple game trees and a large database of problems from the game of Bridge. Our experiments show that the new algorithms both out-perform Monte-carlo sampling, with the superiority of payoff-reduction minimaxing being especially marked. On the Bridge problem set, for example, Monte-carlo sampling only solves 66% of the problems, whereas payoff-reduction minimaxing solves over 95%. This level of performance was even good enough to allow us to discover five errors in the expert text used to generate the test database.
Citations
More filters
TL;DR: GIB, the program being described, involves five separate technical advances: partition search, the practical application of Monte Carlo techniques to realistic problems, a focus on achievable sets to solve problems inherent in the Monte Carlo approach, an extension of alpha-beta pruning from total orders to arbitrary distributive lattices, and the use of squeaky wheel optimization to find approximately optimal solutions to cardplay problems.
Abstract: This paper investigates the problems arising in the construction of a program to play the game of contract bridge. These problems include both the difficulty of solving the game's perfect information variant, and techniques needed to address the fact that bridge is not, in fact, a perfect information game. GIB, the program being described, involves five separate technical advances: partition search, the practical application of Monte Carlo techniques to realistic problems, a focus on achievable sets to solve problems inherent in the Monte Carlo approach, an extension of alpha-beta pruning from total orders to arbitrary distributive lattices, and the use of squeaky wheel optimization to find approximately optimal solutions to cardplay problems.
GIB is currently believed to be of approximately expert caliber, and is currently the strongest computer bridge program in the world.
184 citations
Cites methods from "Finding optimal strategies for impe..."
...Frank et.al, for example, are only capable of solving single suit combinations (13 cards left, give or take) using an algorithm that appears to take several minutes to run ( Frank, Basin, & Matsubara, 1998 )....
[...]
04 May 2015
TL;DR: It is shown that OOS can overcome the problem of non-locality encountered by previous search algorithms and perform well against its worst-case opponents and that preexisting Information Set Monte Carlo tree search (ISMCTS) can get more exploitable over time.
Abstract: Online search in games has been a core interest of artificial intelligence. Search in imperfect information games (e.g., Poker, Bridge, Skat) is particularly challenging due to the complexities introduced by hidden information. In this paper, we present Online Outcome Sampling, an online search variant of Monte Carlo Counterfactual Regret Minimization, which preserves its convergence to Nash equilibrium. We show that OOS can overcome the problem of non-locality encountered by previous search algorithms and perform well against its worst-case opponents. We show that exploitability of the strategies played by OOS decreases as the amount of search time increases, and that preexisting Information Set Monte Carlo tree search (ISMCTS) can get more exploitable over time. In head-to-head play, OOS outperforms ISMCTS in games where non-locality plays a significant role, given a sufficient computation time per move.
56 citations
Cites background from "Finding optimal strategies for impe..."
...Two commonly reported problems are strategy fusion and non-locality [9]....
[...]
...Strategy fusion can be overcome by imposing the proper information constraints during search [9, 6, 30, 26, 8]....
[...]
01 Jul 2004
TL;DR: This thesis considers some aspects of multi-agent systems, seen as a metaphor for reasoning about the world, and providing a conceptual machinery that can be used to model and analyze the reality in which an agent is embedded.
Abstract: This thesis considers some aspects of multi-agent systems, seen as a metaphor for reasoning about the world, and providing a conceptual machinery that can be used to model and analyze the reality in which an agent is embedded First, we study several modal logics for multi-agent systems; in particular, Alternating-time Temporal Logic (ATL) is studied in various contexts Then, a concept of multi-level modeling of reality and multi-level decision making is proposed in the second part of the thesis
45 citations
Cites background from "Finding optimal strategies for impe..."
...Investigation of similar concepts in the context of ATL can prove worthwhile, and lead to new research questions, concerning phenomena like non-locality (Frank and Basin, 1998), and design of efficient suboptimal algorithms (Frank et al., 1998) in the scope of logics for multi-agent systems....
[...]
...ATEL and ATOL: probabilistic outcomes, best defense criteria for games with incomplete information (Frank, 1996; Frank and Basin, 1998; Jamroga, 2001a), non-locality (Frank and Basin, 1998), efficient suboptimal algorithms for games with uncertainty (Frank et al., 1998; Ginsberg, 1999) etc....
[...]
...Similar techniques, like vector minimaxing and payoff-reduction minimaxing (Frank et al., 1998), and generalized vector minimaxing (Jamroga, 2001a) can be tried as well....
[...]
..., Iak , π〉, 4It is worth noting that this sort of structures resembles to some extent the representation proposed independently in (Frank et al., 1998) to investigate search algorithms for games with incomplete information....
[...]
TL;DR: On a set of Bridge problems drawn from a definitive expert text, the heuristics consistently identify strategies as good as, or superior to, the expert solutions – the first time a game-general tree search algorithm has been capable of such performance.
35 citations
Cites result from "Finding optimal strategies for impe..."
...This paper gives a complete account of our work on search algorithms for imperfect information games and supersedes previous results reported in [7] [10] [11]....
[...]
25 Jun 2012
TL;DR: The development process and testing of an agent able to compete against human players on Poker --- one of the most popular IIG field --- is described, indicating that after a training phase the developed strategy is capable of outperforming basic/intermediate playing strategies thus validating this approach.
Abstract: Researching into the incomplete information games (IIG) field requires the development of strategies which focus on optimizing the decision making process, as there is no unequivocal best choice for a particular play. As such, this paper describes the development process and testing of an agent able to compete against human players on Poker --- one of the most popular IIG. The used methodology combines pre-defined opponent models with a reinforcement learning approach. The decision-making algorithm creates a different strategy against each type of opponent by identifying the opponent's type and adjusting the rewards of the actions of the corresponding strategy. The opponent models are simple classifications used by Poker experts. Thus, each strategy is constantly adapted throughout the games, continuously improving the agent's performance. In light of this, two agents with the same structure but different rewarding conditions were developed and tested against other agents and each other. The test results indicated that after a training phase the developed strategy is capable of outperforming basic/intermediate playing strategies thus validating this approach.
22 citations
References
More filters
Book•
01 Jan 1944
TL;DR: Theory of games and economic behavior as mentioned in this paper is the classic work upon which modern-day game theory is based, and it has been widely used to analyze a host of real-world phenomena from arms races to optimal policy choices of presidential candidates, from vaccination policy to major league baseball salary negotiations.
Abstract: This is the classic work upon which modern-day game theory is based. What began more than sixty years ago as a modest proposal that a mathematician and an economist write a short paper together blossomed, in 1944, when Princeton University Press published "Theory of Games and Economic Behavior." In it, John von Neumann and Oskar Morgenstern conceived a groundbreaking mathematical theory of economic and social organization, based on a theory of games of strategy. Not only would this revolutionize economics, but the entirely new field of scientific inquiry it yielded--game theory--has since been widely used to analyze a host of real-world phenomena from arms races to optimal policy choices of presidential candidates, from vaccination policy to major league baseball salary negotiations. And it is today established throughout both the social sciences and a wide range of other sciences.
19,337 citations
2,044 citations
01 Jan 1944
1,200 citations
Additional excerpts
...Neumann & Morgenstern 1944)....
[...]
01 Jan 1988
TL;DR: The problem of constructing a computing routine or "program" for a modern general purpose computer which enables it to play chess is addressed in this article, where the authors propose a set of possibilities in this direction.
Abstract: This paper is concerned with the problem of constructing a computing routine or “program” for a modern general purpose computer which will enable it to play chess. Although perhaps of no practical importance, the question is of theoretical interest, and it is hoped that a satisfactory solution of this problem will act as a wedge in attacking other problems of a similar nature and of greater significance. Some possibilities in this direction are:-
(1)
Machines for designing filters, equalizers, etc.
(2)
Machines for designing relay and switching circuits.
(3)
Machines which will handle routing of telephone calls based on the individual circumstances rather than by fixed patterns.
(4)
Machines for performing symbolic (non-numerical) mathematical operations.
(5)
Machines capable of translating from one language to another.
(6)
Machines for making strategic decisions in simplified military operations.
(7)
Machines capable of orchestrating a melody.
(8)
Machines capable of logical deduction.
798 citations