Book ChapterDOI
Multi-agent reinforcement learning: independent vs. cooperative agents
Ming Tan
- pp 487-494
TLDR
This paper shows that additional sensation from another agent is beneficial if it can be used efficiently, sharing learned policies or episodes among agents speeds up learning at the cost of communication, and for joint tasks, agents engaging in partnership can significantly outperform independent agents although they may learn slowly in the beginning.Abstract:
Intelligent human agents exist in a cooperative social environment that facilitates learning They learn not only by trial-and-error, but also through cooperation by sharing instantaneous information, episodic experience, and learned knowledge The key investigations of this paper are, “Given the same number of reinforcement learning agents, will cooperative agents outperform independent agents who do not communicate during learning?” and “What is the price for such cooperation?” Using independent agents as a benchmark, cooperative agents are studied in following ways: (1) sharing sensation, (2) sharing episodes, and (3) sharing learned policies This paper shows that (a) additional sensation from another agent is beneficial if it can be used efficiently, (b) sharing learned policies or episodes among agents speeds up learning at the cost of communication, and (c) for joint tasks, agents engaging in partnership can significantly outperform independent agents although they may learn slowly in the beginning These tradeoff's are not just limited to multi-agent reinforcement learningread more
Citations
More filters
Book ChapterDOI
Markov games as a framework for multi-agent reinforcement learning
TL;DR: A Q-learning-like algorithm for finding optimal policies and its application to a simple two-player game in which the optimal policy is probabilistic is demonstrated.
Journal ArticleDOI
A Comprehensive Survey of Multiagent Reinforcement Learning
TL;DR: The benefits and challenges of MARL are described along with some of the problem domains where the MARL techniques have been applied, and an outlook for the field is provided.
Posted Content
Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments
TL;DR: An adaptation of actor-critic methods that considers action policies of other agents and is able to successfully learn policies that require complex multi-agent coordination is presented.
Journal ArticleDOI
Cooperative mobile robotics: antecedents and directions
TL;DR: A critical survey of existing works in cooperative robotics is given and open problems in this field are discussed, emphasizing the various theoretical issues that arise in the study of cooperative robotics.
Journal ArticleDOI
Cooperative Multi-Agent Learning: The State of the Art
Liviu Panait,Sean Luke +1 more
TL;DR: This survey attempts to draw from multi-agent learning work in a spectrum of areas, including RL, evolutionary computation, game theory, complex systems, agent modeling, and robotics, and finds that this broad view leads to a division of the work into two categories.
References
More filters
Journal ArticleDOI
Technical Note Q-Learning
Chris Watkins,Peter Dayan +1 more
TL;DR: In this article, it is shown that Q-learning converges to the optimum action-values with probability 1 so long as all actions are repeatedly sampled in all states and the action values are represented discretely.
Journal ArticleDOI
Automatic programming of behavior-based robots using reinforcement learning
TL;DR: In this article, two algorithms for behavior learning are described that combine Q learning, a well-known scheme for propagating reinforcement values temporally across actions, with statistical clustering and Hamming distance.
Proceedings Article
Programming robots using reinforcement learning and teaching
TL;DR: This paper presents a general approach to making robots which can improve their performance from experiences as well as from being taught, and develops a simulated learning robot which could learn three moderately complex behaviors and use what was learned in the simulator to operate in the real world quite successfully.
Proceedings Article
A complexity analysis of cooperative mechanisms in reinforcement learning
TL;DR: The search time complexity of reinforcement learning algorithms, along with unbiased Q-learning, are analyzed for problem solving tasks on a restricted class of state spaces and shed light on the complexity of search in reinforcement learning in general and the utility of cooperative mechanisms for reducing search.
Toward parallel and distributed learning by meta-learning
TL;DR: Several meta-learning strategies for integrating independently learned classifiers by the same learner in a parallel and distributed computing environment are outlined, particularly suited for massive amounts of data that main-memory-based learning algorithms cannot efficiently handle.