Abstract: This thesis addresses situated, embodied agents interacting in complex domains. It focuses on two problems: (1) synthesis and analysis of intelligent group behavior, and (2) learning in complex group environments.
Behaviors are proposed as the appropriate level for control and learning. Basic behaviors are introduced as building blocks for synthesizing and analyzing system behavior. The thesis describes the process of selecting such basic behaviors, formally specifying them, algorithmically implementing them, and empirically evaluating them. All of the proposed ideas are validated with a group of up to 20 mobile robots using a basic behavior set consisting of: avoidance, following, aggregation, dispersion, and homing. The set of basic behaviors acts as a substrate for achieving more complex high-level goals and tasks. Two behavior combination operators are introduced, and verified by combining subsets of the above basic behavior set to implement collective flocking and foraging.
A methodology is introduced for automatically constructing higher-level behaviors by learning to select among the basic behavior set. A novel formulation of reinforcement learning is proposed that makes behavior selection learnable in noisy, uncertain multi-agent environments with stochastic dynamics. It consists of using conditions and behaviors for more robust control and minimized state-spaces, and a reinforcement shaping methodology that enables principled embedding of domain knowledge with two types of shaping functions: heterogeneous reward functions and progress estimators. The methodology outperforms two alternatives when tested on a collection of robots learning to forage. The proposed formulation enables and accelerates learning in complex multi-robot domains. The generality of the approach makes it compatible with the existing reinforcement learning algorithms, allowing it to accelerate learning in a variety of domains and applications.
The presented methodologies and results are aimed at extending our understanding of synthesis, analysis, and learning of group behavior. (Copies available exclusively from MIT Libraries, Rm. 14-0551, Cambridge, MA 02139-4307. Ph. 617-253-5668; Fax 617-253-1690.)