The game of Go has long been viewed as the most challenging of classic games for artificial intelligence owing to its enormous search space and the difficulty of evaluating board positions and moves. Here we introduce a new approach to computer Go that uses ‘value networks’ to evaluate board positions and ‘policy networks’ to select moves. These deep neural networks are trained by a novel combination of supervised learning from human expert games, and reinforcement learning from games of self-play. Without any lookahead search, the neural networks play Go at the level of stateof-the-art Monte Carlo tree search programs that simulate thousands of random games of self-play. We also introduce a new search algorithm that combines Monte Carlo simulation with value and policy networks. Using this search algorithm, our program AlphaGo achieved a 99.8% winning rate against other Go programs, and defeated the human European Go champion by 5 games to 0. This is the first time that a computer program has defeated a human professional player in the full-sized game of Go, a feat previously thought to be at least a decade away.

/pdf/mastering-the-game-of-go-with-deep-neural-networks-and-tree-3viuxo6uxm.pdf

Mastering the game of Go with deep neural networks and tree search

Data Mining - Concepts and Techniques.

We have devised and implemented a novel computational strategy for de novo design of molecules with desired properties termed ReLeaSE (Reinforcement Learning for Structural Evolution). On the basis of deep and reinforcement learning (RL) approaches, ReLeaSE integrates two deep neural networks—generative and predictive—that are trained separately but are used jointly to generate novel targeted chemical libraries. ReLeaSE uses simple representation of molecules by their simplified molecular-input line-entry system (SMILES) strings only. Generative models are trained with a stack-augmented memory network to produce chemically feasible SMILES strings, and predictive models are derived to forecast the desired properties of the de novo–generated compounds. In the first phase of the method, generative and predictive models are trained separately with a supervised learning algorithm. In the second phase, both models are trained jointly with the RL approach to bias the generation of new chemical structures toward those with the desired physical and/or biological properties. In the proof-of-concept study, we have used the ReLeaSE method to design chemical libraries with a bias toward structural complexity or toward compounds with maximal, minimal, or specific range of physical properties, such as melting point or hydrophobicity, or toward compounds with inhibitory activity against Janus protein kinase 2. The approach proposed herein can find a general use for generating targeted chemical libraries of novel compounds optimized for either a single desired property or multiple properties.

Deep reinforcement learning for de novo drug design

Monte-Carlo Tree Search (MCTS) is a new best-first search guided by the results of Monte-Carlo simulations. In this article, we introduce two progressive strategies for MCTS, called progressive bias and progressive unpruning. They enable the use of relatively time-expensive heuristic knowledge without speed reduction. Progressive bias directs the search according to heuristic knowledge. Progressive unpruning first reduces the branching factor, and then increases it gradually again. Experiments assess that the two progressive strategies significantly improve the level of our Go program Mango. Moreover, we see that the combination of both strategies performs even better on larger board sizes.

/pdf/progressive-strategies-for-monte-carlo-tree-search-5givz0d985.pdf

Progressive Strategies for Monte-Carlo Tree Search

The game of checkers has roughly 500 billion billion possible positions (5 × 1020). The task of solving the game, determining the final result in a game with no mistakes made by either player, is daunting. Since 1989, almost continuously, dozens of computers have been working on solving checkers, applying state-of-the-art artificial intelligence techniques to the proving process. This paper announces that checkers is now solved: Perfect play by both sides leads to a draw. This is the most challenging popular game to be solved to date, roughly one million times as complex as Connect Four. Artificial intelligence technology has been used to generate strong heuristic-based game-playing programs, such as Deep Blue for chess. Solving a game takes this to the next level by replacing the heuristics with perfection.

https://science.sciencemag.org/content/sci/317/5844/1518.full.pdf

Checkers Is Solved

http://web.mit.edu/6.034/wwwbob/GamesSolved.pdf

Games solved: now and in the future

In 1981 Claude Berge asked about combinatorial properties that might be used to solve Hex puzzles. In response, we establish properties of dead, or negligible, cells in Hex and the Shannon game.

https://webdocs.cs.ualberta.ca/~hayward/papers/bergeParis.pdf

Dead Cell Analysis in Hex and the Shannon Game

Hex and combinatorics

Traditionally, chess has been called the "fruitfly of Artificial Intelligence". In recent years, however, the focus of game playing research has gradually shifted away from chess towards games that offer new challenges. These challenges include large branching factors and imperfect information. The game of Hex offers some interesting properties that make it an attractive research subject. This paper presents the key ideas behind Queenbee, the first Hex playing program to play at the level of strong human players.

Are Bees Better than Fruitflies

We present an algorithm that determines the outcome of an arbitrary Hex game-state by finding a winning virtual connection for the winning player Our algorithm recursively searches the game-tree, combining fixed and dynamic game-state virtual connection composition rules to find a winning virtual connection for one of the two players The search is enhanced by pruning the game-tree according to two new Hex game-state reduction results: under certain conditions, (i) some moves dominate others, and (ii) some board-cells can be "filled-in" without changing the game's outcomeThe algorithm is powerful enough to solve arbitrary 7 × 7 game-states In particular, we use it to determine the outcome of a 7 × 7 Hex game after each of the 49 possible opening moves, in each case finding an explicit proof-tree for the winning player

Jack van Rijswijck

Papers

Games solved: now and in the future

Dead Cell Analysis in Hex and the Shannon Game

Hex and combinatorics

Are Bees Better than Fruitflies

Solving 7×7 Hex with domination, fill-in, and virtual connections