Showing papers presented at "Advances in Computer Games in 2009"

Monte-Carlo tree search in settlers of catan

Abstract: Games are considered important benchmark opportunities for artificial intelligence research. Modern strategic board games can typically be played by three or more people, which makes them suitable test beds for investigating multi-player strategic decision making. Monte-Carlo Tree Search (MCTS) is a recently published family of algorithms that achieved successful results with classical, two-player, perfect-information games such as Go. In this paper we apply MCTS to the multi-player, non-deterministic board game Settlers of Catan. We implemented an agent that is able to play against computer-controlled and human players. We show that MCTS can be adapted successfully to multi-agent environments, and present two approaches of providing the agent with a limited amount of domain knowledge. Our results show that the agent has a considerable playing strength when compared to game implementation with existing heuristics. So, we may conclude that MCTS is a suitable tool for achieving a strong Settlers of Catan player.

Creating an upper-confidence-tree program for havannah

Abstract: Monte-Carlo Tree Search and Upper Confidence Bounds provided huge improvements in computer-Go. In this paper, we test the generality of the approach by experimenting on the game, Havannah, which is known for being especially difficult for computers. We show that the same results hold, with slight differences related to the absence of clearly known patterns for the game of Havannah, in spite of the fact that Havannah is more related to connection games like Hex than to territory games like Go.

Adding expert knowledge and exploration in monte-carlo tree search

Abstract: We present a new exploration term, more efficient than classical UCT-like exploration terms. It combines efficiently expert rules, patterns extracted from datasets, All-Moves-As-First values, and classical online values. As this improved bandit formula does not solve several important situations (semeais, nakade) in computer Go, we present three other important improvements which are central in the recent progress of our program MoGo. We show an expert-based improvement of Monte-Carlo simulations for nakade situations; we also emphasize some limitations of this modification. We show a technique which preserves diversity in the Monte-Carlo simulation, which greatly improves the results in 19x19. Whereas the UCB-based exploration term is not efficient in MoGo, we show a new exploration term which is highly efficient in MoGo. MoGo recently won a game with handicap 7 against a 9Dan Pro player, Zhou JunXun, winner of the LG Cup 2007, and a game with handicap 6 against a 1Dan pro player, Li-Chen Chien.

A lock-free multithreaded monte-carlo tree search algorithm

Abstract: With the recent success of Monte-Carlo tree search algorithms in Go and other games, and the increasing number of cores in standard CPUs, the efficient parallelization of the search has become an important issue. We present a new lock-free parallel algorithm for Monte-Carlo tree search which takes advantage of the memory model of the IA-32 and Intel-64 CPU architectures and intentionally ignores rare faulty updates of node values. We show that this algorithm significantly improves the scalability of the Fuego Go program.

Incongruity-based adaptive game balancing

Abstract: Commercial games possess various methods of game balancing. Each of them modifies the game's entertainment value for players of different skill levels. This paper deals with one of them, viz. a way of automatically adapting a game's balance which is based on the theory of incongruity. We tested our approach on a group of subjects, who played a game with three difficulty settings. The idea is to maintain a specific difference in incongruity automatically. We tested our idea extensively and may report that the results coincide with the theory of incongruity as far as positive incongruity is concerned. The main conclusion is that, owing to our automatically maintained balanced difficulty setting, we can avoid that a game becomes boring or frustrating.

Evaluation function based monte-carlo LOA

Abstract: Recently, Monte-Carlo Tree Search (MCTS) has advanced the field of computer Go substantially Also in the game of Lines of Action (LOA), which has been dominated so far by αβ, MCTS is making an inroad In this paper we investigate how to use a positional evaluation function in a Monte-Carlo simulation-based LOA program (MC-LOA) Four different simulation strategies are designed, called Evaluation Cut-Off, Corrective, Greedy, and Mixed They use an evaluation function in several ways Experimental results reveal that the Mixed strategy is the best among them This strategy draws the moves randomly based on their transition probabilities in the first part of a simulation, but selects them based on their evaluation score in the second part of a simulation Using this simulation strategy the MC-LOA program plays at the same level as the αβ program MIA, the best LOA-playing entity in the world

Randomized parallel proof-number search

Abstract: Proof-Number Search (PNS) is a powerful method for solving games and game positions. Over the years, the research on PNS has steadily produced new insights and techniques. With multi-core processors becoming established in the recent past, the question of parallelizing PNS has gained new urgency. This article presents a new technique called Randomized Parallel Proof-Number Search (RPPNS) for parallelizing PNS on multi-core systems with shared memory. The parallelization is based on randomizing the move selection of multiple threads, which operate on the same search tree. RPPNS is tested on a set of complex Lines-of-Action endgame positions. Experiments show that RPPNS scales well. Four directions for future research are given.

Performance and prediction: bayesian modelling of fallible choice in chess

Abstract: Evaluating agents in decision-making applications requires assessing their skill and predicting their behaviour. Both are well developed in Poker-like situations, but less so in more complex game and model domains. This paper addresses both tasks by using Bayesian inference in a benchmark space of reference agents. The concepts are explained and demonstrated using the game of chess but the model applies generically to any domain with quantifiable options and fallible choice. Demonstration applications address questions frequently asked by the chess community regarding the stability of the rating scale, the comparison of players of different eras and/or leagues, and controversial incidents possibly involving fraud. The last include alleged under-performance, fabrication of tournament results, and clandestine use of computer advice during competition. Beyond the model world of games, the aim is to improve fallible human performance in complex, high-value tasks.

Monte-Carlo kakuro

Abstract: Kakuro consists in filling a grid with integers that sum up to predefined values. Sums are predefined for each row and column and all integers have to be different in the same row or column. Kakuro can be modeled as a constraint satisfaction problem. Monte-Carlo methods can improve on traditional search methods for Kakuro. We show that a Nested Monte-Carlo Search at level 2 gives good results. This is the first time a nested search of level 2 gives good results for a Constraint Satisfaction problem.

A study of UCT and its enhancements in an artificial game

Abstract: Monte-Carlo tree search, especially the UCT algorithm and its enhancements, have become extremely popular. Because of the importance of this family of algorithms, a deeper understanding of when and how the different enhancements work is desirable. To avoid the hard to analyze intricacies of tournament-level programs in complex games, this work focuses on a simple abstract game, which is designed to be ideal for history-based heuristics such as RAVE. Experiments show the influence of game complexity and of enhancements on the performance of Monte-Carlo Tree Search.

Deriving concepts and strategies from chess tablebases

Abstract: Complete tablebases, indicating best moves for every position, exist for chess endgames. There is no doubt that tablebases contain a wealth of knowledge, however, mining for this knowledge, manually or automatically, proved as extremely difficult. Recently, we developed an approach that combines specialized minimax search with the argument-based machine learning (ABML) paradigm. In this paper, we put this approach to test in an attempt to elicit human-understandable knowledge from tablebases. Specifically, we semi-automatically synthesize knowledge from the KBNK tablebase for teaching the difficult king, bishop, and knight versus the lone king endgame.

Automated discovery of search-extension features

Abstract: One of the main challenges with selective search extensions is designing effective move categories (features). Usually, it is a manual trial-and-error task, which requires both intuition and expert human knowledge. Automating this task potentially enables the discovery of both more complex and more effective move categories. The current work introduces Gradual Focus, an algorithm for automatically discovering interesting move categories for selective search extensions. The algorithm iteratively creates new more refined move categories by combining features from an atomic feature set. Empirical data is presented for the game Breakthrough showing that Gradual Focus looks at a number of combinations that is two orders of magnitude fewer than a brute-force method does, while preserving adequate precision and recall.

Solving kriegspiel endings with brute force: the case of KR vs. k

Abstract: Retrograde analysis is a tool for reconstructing a game tree starting from its leaves; with these techniques one can solve specific subsets of a complex game, achieving optimal play in these situations, for example a chess endgame. Position values can then be stored in “tablebases” for instant access, as is the norm in professional chess programs. While this technique is supposed to be only used in games of perfect information, this paper shows that retrograde analysis can be applied to certain Kriegspiel (invisible chess) endgames, such as King and Rook versus King. Using brute force and a suitable data representation, one can achieve perfect play, with perfection meaning fastest checkmate in the worst case and without making any assumptions on the opponent.

Data assurance in opaque computations

Abstract: The chess endgame is increasingly being seen through the lens of, and therefore effectively defined by, a data ‘model' of itself It is vital that such models are clearly faithful to the reality they purport to represent This paper examines that issue and systems engineering responses to it, using the chess endgame as the exemplar scenario A structured survey has been carried out of the intrinsic challenges and complexity of creating endgame data by reviewing the past pattern of errors during work in progress, surfacing in publications and occurring after the data was generated Specific measures are proposed to counter observed classes of error-risk, including a preliminary survey of techniques for using state-of-the-art verification tools to generate EGTs that are correct by construction The approach may be applied generically beyond the game domain

6-man chess and zugzwangs

Abstract: With 6-man Chess essentially solved, the available 6-man Endgame Tables (EGTs) have been scanned for zugzwang positions where, unusually, having the move is a disadvantage. Review statistics together with some highlights and positions are provided here: the complete information is available on the ICGA website. An outcome of the review is the observation that the definition of zugzwang should be revisited, if only because the presence of en passant capture moves gives rise to three new, asymmetric types of zugzwang.

Conflict resolution of chinese chess endgame knowledge base

Abstract: Endgame heuristics are often incorperated as part of the evaluation function used in Chinese Chess programs. In our program, Contemplation, we have proposed an automatic strategy to construct a large set of endgame heuristics. In this paper, we propose a conflict resolution strategy to eliminate the conflicts among the constructed heuristic databases, which is called endgame knowledge base. In our experiment, the correctness of the obtained constructed endgame knowledge base is sufficiently high for practical usage.

Hex, braids, the crossing rule, and XH-search

Abstract: We present XH-search, a Hex connection finding algorithm. XH-search extends Anshelevich's H-search by incorporating a new Crossing Rule to find braids, connections built from overlapping subconnections.

Plans, patterns, and move categories guiding a highly selective search

Abstract: In this paper we present our ideas for an Arimaa-playing program (also called a bot) that uses plans and pattern matching to guide a highly selective search. We restrict move generation to moves in certain move categories to reduce the number of moves considered by the bot significantly. Arimaa is a modern board game that can be played with a standard Chess set. However, the rules of the game are not at all like those of Chess. Furthermore, Arimaa was designed to be as simple and intuitive as possible for humans, yet challenging for computers. While all established Arimaa bots use alpha-beta search with a variety of pruning techniques and other heuristics ending in an extensive positional leaf node evaluation, our new bot, Rat, starts with a positional evaluation of the current position. Based on features found in the current position – supported by pattern matching using a directed position graph – our bot Rat decides which of a given set of plans to follow. The plan then dictates what types of moves can be chosen. This is another major difference from bots that generate “all” possible moves for a particular position. Rat is only allowed to generate moves that belong to certain categories. Leaf nodes are evaluated only by a straightforward material evaluation to help avoid moves that lose material. This highly selective search looks, on average, at only 5 moves out of 5,000 to over 40,000 possible moves in a middle game position.

On drawn k-in-a-row games

Abstract: In 2005, Wu and Huang [9] presented a generalized family of k-in-a-row games. The current paper simplifies the family to Connect(k, p). Two players alternately place p stones on empty squares of an infinite board in each turn. The player who first obtains k consecutive stones of his own horizontally, vertically, diagonally wins. A Connect(k, p)game is drawn if both have no winning strategy. Given p, this paper derives the value kdraw(p), such that Connect(kdraw(p), p) is drawn, as follows. (1) kdraw(2) = 11. (2) For all p ≥ 3, kdraw(p) = 3p+3d+8, where d is a logarithmic function of p. So, the ratio kdraw(p)/p is approximate to 3 for sufficiently large p. To our knowledge, our kdraw(p) are currently the smallest for all 2 ≤ p < 1000, except for p = 3.

Optimal analyses for 3× n AB games in the worst case

Abstract: The past decades have witnessed a growing interest in research on deductive games such as Mastermind and AB game. Because of the complicated behavior of deductive games, tree-search approaches are often adopted to find their optimal strategies. In this paper, a generalized version of deductive games, called 3×n AB games, is introduced. However, traditional tree-search approaches are not appropriate for solving this problem since it can only solve instances with smaller n. For larger values of n, a systematic approach is necessary. Therefore, intensive analyses of playing 3×n AB games in the worst case optimally are conducted and a sophisticated method, called structural reduction, which aims at explaining the worst situation in this game is developed in the study. Furthermore, a worthwhile formula for calculating the optimal numbers of guesses required for arbitrary values of n is derived and proven to be final.