A long-standing goal of artificial intelligence is an algorithm that learns, tabula rasa, superhuman proficiency in challenging domains. Recently, AlphaGo became the first program to defeat a world champion in the game of Go. The tree search in AlphaGo evaluated positions and selected moves using deep neural networks. These neural networks were trained by supervised learning from human expert moves, and by reinforcement learning from self-play. Here we introduce an algorithm based solely on reinforcement learning, without human data, guidance or domain knowledge beyond game rules. AlphaGo becomes its own teacher: a neural network is trained to predict AlphaGo’s own move selections and also the winner of AlphaGo’s games. This neural network improves the strength of the tree search, resulting in higher quality move selection and stronger self-play in the next iteration. Starting tabula rasa, our new program AlphaGo Zero achieved superhuman performance, winning 100–0 against the previously published, champion-defeating AlphaGo. Starting from zero knowledge and without human data, AlphaGo Zero was able to teach itself to play Go and to develop novel strategies that provide new insights into the oldest of games. To beat world champions at the game of Go, the computer program AlphaGo has relied largely on supervised learning from millions of human expert moves. David Silver and colleagues have now produced a system called AlphaGo Zero, which is based purely on reinforcement learning and learns solely from self-play. Starting from random moves, it can reach superhuman level in just a couple of days of training and five million games of self-play, and can now beat all previous versions of AlphaGo. Because the machine independently discovers the same fundamental principles of the game that took humans millennia to conceptualize, the work suggests that such principles have some universal character, beyond human bias.

/pdf/mastering-the-game-of-go-without-human-knowledge-19y9mw638s.pdf

Mastering the game of Go without human knowledge

How to Do Things With Words

An Introduction to MultiAgent Systems.

Monte Carlo tree search (MCTS) is a recently proposed search method that combines the precision of tree search with the generality of random sampling. It has received considerable interest due to its spectacular success in the difficult problem of computer Go, but has also proved beneficial in a range of other domains. This paper is a survey of the literature to date, intended to provide a snapshot of the state of the art after the first five years of MCTS research. We outline the core algorithm's derivation, impart some structure on the many variations and enhancements that have been proposed, and summarize the results from the key game and nongame domains to which MCTS methods have been applied. A number of open research questions indicate that the field is ripe for future work.

/pdf/a-survey-of-monte-carlo-tree-search-methods-2mj9olkdni.pdf

A Survey of Monte Carlo Tree Search Methods

An empirical comparison of four initialization methods for the K-Means algorithm

This article presents a task allocation protocol that is efficient in time and tolerates crash failures in multi-agent systems. The protocol is an extension of the negotiation protocol defined by Smith and Davis [25, 26] for task allocation. Our extension of the Contract Net Protocol (1) enables an agent to manage several negotiation processes in parallel; (2) optimizes the length of the negotiation processes among agents; (3) reduces the contractors' decommitment situations; (4) enables the detection of failures of an agent participating in a negotiation process and prevents a negotiation process with blocked agents.

/pdf/an-extended-multi-agent-negotiation-protocol-12nkxcwjyh.pdf

An Extended Multi-Agent Negotiation Protocol

Recent advances in artificial intelligence, particularly in the field of multi-agent theory, offer great promises in modeling strategic planning processes. In this domain, the ability to introduce distinct cognitive agents which cooperate to solve the problem enables the processing of more complex and ill-structured problems. This paper presents a general framework for building a distributed strategic decision support system (DSDSS) which integrates both advances in distributed decision making and distributed artificial intelligence. It goes on to describe a cooperative and distributed system with two specific features: the users intervene as human agents in the solution formation, and strategic knowledge and domain knowledge are distributed in different agents which communicate through various blackboards and message passing. An example is provided in the field of strategic marketing which illustrates how the system operates.

A distributed decision support system for strategic planning

this paper presents the modeling of a disturbance on a public transportation line. The proposed model allows the synthetis, evaluation and update of available information in order to help human regulators in their monitoring task. It begins with a formal modeling of the disturbance concept. This modeling makes it possible to capitalize the knowledge available within a monitoring station and to follow up the evolution of the disturbances in real time. The paper goes on to propose a multi-agent representation of an incident allowing the integration of the disturbance processing within the activity of a network system.

https://hal.archives-ouvertes.fr/hal-00017160/document

Dynamic Modeling of a Disturbance in a Multi-agent System for Traffic Regulation (decembre 2002)

This paper presents the modeling of a disturbance on a public transportation line. The proposed model allows the synthesis, evaluation and update of available information in order to help human regulators in their monitoring task. It begins with a formal modeling of the disturbance concept. This modeling makes it possible to capitalize the knowledge available within a monitoring station and to follow up the evolution of the disturbances in real time. The paper goes on to propose a multi-agent representation of an incident allowing the integration of the disturbance processing within the activity of a network system.

/pdf/dynamic-modeling-of-a-disturbance-in-a-multi-agent-system-4tvj8849ye.pdf

Dynamic modeling of a disturbance in a multi-agent system for traffic regulation

This paper presents an agent-based approach used to design a Transportation Regulation Support System (TRSS), that reports the network activity in real-time and thus assists the bus network regulators. The objective is to combine the functionalities of the existing information system with the functionalities of a decision support system in order to propose a generic model of a traffic regulation support system. Unlike the other approaches that only deal with a specific task, the original feature of our generic model is that it proposes a global approach to the regulation function under normal conditions (network monitoring, dynamic timetable management) and under disrupted conditions (disturbance assessment and action planning of feasible solutions). Following the introduction, the second section presents the notions of the domain and highlights the main regulation problems. The third section details and motivates our choice of the components of the generic model. Based on our generic model, in the fourth section, we present a TRSS prototype called SATIR (Systeme Automatique de Traitement des Incidents en Reseau - Automatic System for Network Incident Processing) that we have developed. SATIR has been tested on the Brussels transportation network (STIB). The results are presented in the fifth section. Lastly, we show how using the multi-agent paradigm opens perspectives regarding the development of new functionalities to improve the management of a bus network.

Suzanne Pinson

Papers

An Extended Multi-Agent Negotiation Protocol

A distributed decision support system for strategic planning

Dynamic Modeling of a Disturbance in a Multi-agent System for Traffic Regulation (decembre 2002)

Dynamic modeling of a disturbance in a multi-agent system for traffic regulation

Using intelligent agents for Transportation Regulation Support System design