Reinforcement learning, one of the most active research areas in artificial intelligence, is a computational approach to learning whereby an agent tries to maximize the total amount of reward it receives when interacting with a complex, uncertain environment. In Reinforcement Learning, Richard Sutton and Andrew Barto provide a clear and simple account of the key ideas and algorithms of reinforcement learning. Their discussion ranges from the history of the field's intellectual foundations to the most recent developments and applications. The only necessary mathematical background is familiarity with elementary concepts of probability. The book is divided into three parts. Part I defines the reinforcement learning problem in terms of Markov decision processes. Part II provides basic solution methods: dynamic programming, Monte Carlo methods, and temporal-difference learning. Part III presents a unified view of the solution methods and incorporates artificial neural networks, eligibility traces, and planning; the two final chapters present case studies and consider the future of reinforcement learning.

/pdf/reinforcement-learning-an-introduction-rzxgej9p17.pdf

Reinforcement Learning: An Introduction

The concept of an agent has become important in both Artificial Intelligence (AI) and mainstream computer science. Our aim in this paper is to point the reader at what we perceive to be the most important theoretical and practical issues associated with the design and construction of intelligent agents. For convenience, we divide these issues into three areas (though as the reader will see, the divisions are at times somewhat arbitrary). Agent theory is concerned with the question of what an agent is, and the use of mathematical formalisms for representing and reasoning about the properties of agents. Agent architectures can be thought of as software engineering models of agents;researchers in this area are primarily concerned with the problem of designing software or hardware systems that will satisfy the properties specified by agent theorists. Finally, agent languages are software systems for programming and experimenting with agents; these languages may embody principles proposed by theorists. The paper is not intended to serve as a tutorial introduction to all the issues mentioned; we hope instead simply to identify the most important issues, and point to work that elaborates on them. The article includes a short review of current and potential applications of agent technology.

/pdf/intelligent-agents-theory-and-practice-5dchj7p8pc.pdf

Intelligent Agents: Theory and Practice

The contract net protocol has been developed to specify problem-solving communication and control for nodes in a distributed problem solver. Task distribution is affected by a negotiation process, a discussion carried on between nodes with tasks to be executed and nodes that may be able to execute those tasks.

/pdf/the-contract-net-protocol-high-level-communication-and-237lqg0ddq.pdf

The Contract Net Protocol: High-Level Communication and Control in a Distributed Problem Solver

This survey characterizes an emerging research area, sometimes called coordination theory, that focuses on the interdisciplinary study of coordination. Research in this area uses and extends ideas about coordination from disciplines such as computer science, organization theory, operations research, economics, linguistics, and psychology.A key insight of the framework presented here is that coordination can be seen as the process of managing dependencies among activities. Further progress, therefore, should be possible by characterizing different kinds of dependencies and identifying the coordination processes that can be used to manage them. A variety of processes are analyzed from this perspective, and commonalities across disciplines are identified. Processes analyzed include those for managing shared resources, producer/consumer relationships, simultaneity constraints, and task/subtask dependencies.Section 3 summarizes ways of applying a coordination perspective in three different domains:(1) understanding the effects of information technology on human organizations and markets, (2) designing cooperative work tools, and (3) designing distributed and parallel computer systems. In the final section, elements of a research agenda in this new area are briefly outlined.

/pdf/the-interdisciplinary-study-of-coordination-22jsxamy1h.pdf

The interdisciplinary study of coordination

From the Publisher:
An agent is an entity with domain knowledge, goals and actions. Multi-agent systems are a set of agents which
interact in a common environment. Multi-agent systems deal with the construction of complex systems involving multiple agents and their coordination. A multi-agent system (MAS) is a distributed computing system with
autonomous interacting intelligent agents that coordinate their actions so as to achieve its goal(s) jointly or competitively.

Introduction to Multiagent Systems

The contract net framework for distributed problem solving is discussed. Task distribution is viewed as a mutual selection process, a discussion carried on between a node with a task to be executed and a group of nodes that may be able to execute the task. This leads to the use of a control formalism based on a contract metaphore in which task distribution corresponds to contract negotiation.

The three primary components of the framework are described: communication, control, and knowledge organization. The use of the framework is illustrated for a distributed sensing system. We then discuss suitable applications and limitations of the framework.

A framework for distributed problem solving

The contract net protocol: high-level communication and control in a distributed problem solver

Enabling organizations to capture, share, and apply the collective experience and know-how of their people is seen as fundamental to competing in the knowledge economy. As a result, there has been a wave of enthusiasm and activity centered on knowledge management. To make progress in this area, issues of technology, process, people, and content must be addressed. In this article, we develop a road map for knowledge management. It begins with an assessment of the current state of the practice, using examples drawn from our experience at Schlumberger. It then sketches the possible evolution of technology and practice over a 10-year period. Along the way, we highlight ways in which AI technology, present and future, can be applied in knowledge management systems.

/pdf/the-road-ahead-for-knowledge-management-an-ai-perspective-49gi1d0zad.pdf

The Road Ahead for Knowledge Management: An AI Perspective

Fundamentals of expert systems

We use our experience with the Dipmeter Advisor system for well-log interpretation as a case study to examine the development of commercial expert system. We discuss the nature of these systems as we see them in the coming decade, characteristics of the evolution process, development methods, and skills required in the development team. We argue that the tools and ideas of rapid prototyping and successive refinement accelerate the development process. We note that different types of people are required at different stages of expert system development: Those who are primarily knowledgeable in the domain, but who can use the framework to expand the domain knowledge; and those who can actually design and build expert systems. Finally, we discuss the problem of technology transfer and compare our experience with some of the traditional wisdom of expert system development.

/pdf/on-the-development-of-commercial-expert-systems-279htnmsor.pdf

Reid G. Smith

Papers

A framework for distributed problem solving

The contract net protocol: high-level communication and control in a distributed problem solver

The Road Ahead for Knowledge Management: An AI Perspective

Fundamentals of expert systems

On the Development of Commercial Expert Systems