scispace - formally typeset
Search or ask a question

Showing papers on "Multi-agent system published in 2006"


Journal ArticleDOI
TL;DR: Fire, a trust and reputation model that integrates a number of information sources to produce a comprehensive assessment of an agent’s likely performance in open systems, is presented and is shown to help agents gain better utility than their benchmarks.
Abstract: Trust and reputation are central to effective interactions in open multi-agent systems (MAS) in which agents, that are owned by a variety of stakeholders, continuously enter and leave the system. This openness means existing trust and reputation models cannot readily be used since their performance suffers when there are various (unforseen) changes in the environment. To this end, this paper presents FIRE, a trust and reputation model that integrates a number of information sources to produce a comprehensive assessment of an agent's likely performance in open systems. Specifically, FIRE incorporates interaction trust, role-based trust, witness reputation, and certified reputation to provide trust metrics in most circumstances. FIRE is empirically evaluated and is shown to help agents gain better utility (by effectively selecting appropriate interaction partners) than our benchmarks in a variety of agent populations. It is also shown that FIRE is able to effectively respond to changes that occur in an agent's environment.

800 citations


Journal ArticleDOI
TL;DR: The evolution of agent technologies and manufacturing will probably proceed hand in hand and the former can receive real challenges from the latter, which will have more and more benefits in applying agent technologies, presumably together with well-established or emerging approaches of other disciplines.
Abstract: The emerging paradigm of agent-based computation has revolutionized the building of intelligent and decentralized systems. The new technologies met well the requirements in all domains of manufacturing where problems of uncertainty and temporal dynamics, information sharing and distributed operation, or coordination and cooperation of autonomous entities had to be tackled. In the paper software agents and multi-agent systems are introduced and through a comprehensive survey, their potential manufacturing applications are outlined. Special emphasis is laid on methodological issues and deployed industrial systems. After discussing open issues and strategic research directions, we conclude that the evolution of agent technologies and manufacturing will probably proceed hand in hand. The former can receive real challenges from the latter, which, in turn, will have more and more benefits in applying agent technologies, presumably together with well-established or emerging approaches of other disciplines.

668 citations


Journal ArticleDOI
TL;DR: An update review on the recent achievements in implementing agent-based manufacturing systems such as agent encapsulation, agent organization, agent coordination and negotiation, system dynamics, learning, optimization, security and privacy, tools and standards is provided.

560 citations


Journal ArticleDOI
01 Jul 2006
TL;DR: This paper reviews the research literature on manufacturing process planning, scheduling as well as their integration, particularly on agent-based approaches to these difficult problems and discusses major issues in these research areas.
Abstract: Manufacturing process planning is the process of selecting and sequencing manufacturing processes such that they achieve one or more goals and satisfy a set of domain constraints. Manufacturing scheduling is the process of selecting a process plan and assigning manufacturing resources for specific time periods to the set of manufacturing processes in the plan. It is, in fact, an optimization process by which limited manufacturing resources are allocated over time among parallel and sequential activities. Manufacturing process planning and scheduling are usually considered to be two separate and distinct phases. Traditional optimization approaches to these problems do not consider the constraints of both domains simultaneously and result in suboptimal solutions. Without considering real-time machine workloads and shop floor dynamics, process plans may become suboptimal or even invalid at the time of execution. Therefore, there is a need for the integration of manufacturing process-planning and scheduling systems for generating more realistic and effective plans. After describing the complexity of the manufacturing process-planning and scheduling problems, this paper reviews the research literature on manufacturing process planning, scheduling as well as their integration, particularly on agent-based approaches to these difficult problems. Major issues in these research areas are discussed, and research opportunities and challenges are identified

424 citations



Proceedings ArticleDOI
03 Dec 2006
TL;DR: This tutorial describes the foundations of ABMS, identifies ABMS toolkits and development methods illustrated through a supply chain example, and provides thoughts on the appropriate contexts for ABMS versus conventional modeling techniques.
Abstract: Agent-based modeling and simulation (ABMS) is a new approach to modeling systems comprised of interacting autonomous agents. ABMS promises to have far-reaching effects on the way that businesses use computers to support decision-making and researchers use electronic laboratories to do research. Some have gone so far as to contend that ABMS is a new way of doing science. Computational advances make possible a growing number of agent-based applications across many fields. Applications range from modeling agent behavior in the stock market and supply chains, to predicting the spread of epidemics and the threat of bio-warfare, from modeling the growth and decline of ancient civilizations to modeling the complexities of the human immune system, and many more. This tutorial describes the foundations of ABMS, identifies ABMS tool-kits and development methods illustrated through a supply chain example, and provides thoughts on the appropriate contexts for ABMS versus conventional modeling techniques.

405 citations


Journal ArticleDOI
TL;DR: A microscopic model of communicating autonomous agents performing language games without any central control is introduced and it is shown that the system undergoes a disorder/order transition, going through a sharp symmetry breaking process to reach a shared set of conventions.
Abstract: What processes can explain how very large populations are able to converge on the use of a particular word or grammatical construction without global coordination? Answering this question helps to understand why new language constructs usually propagate along an S-shaped curve with a rather sudden transition towards global agreement. It also helps to analyse and design new technologies that support or orchestrate self-organizing communication systems, such as recent social tagging systems for the web. The article introduces and studies a microscopic model of communicating autonomous agents performing language games without any central control. We show that the system undergoes a disorder/order transition, going through a sharp symmetry breaking process to reach a shared set of conventions. Before the transition, the system builds up non-trivial scale-invariant correlations, for instance in the distribution of competing synonyms, which display a Zipf-like law. These correlations make the system ready for the transition towards shared conventions, which, observed on the timescale of collective behaviours, becomes sharper and sharper with system size. This surprising result not only explains why human language can scale up to very large populations but also suggests ways to optimize artificial semiotic dynamics.

380 citations


Journal ArticleDOI
Feng Xiao1, Long Wang1
TL;DR: In this article, the authors investigated state consensus problems for discrete-time multi-agent systems with changing communication topologies and bounded time-varying communication delays. But their analysis was based on the properties of non-negative matrices.
Abstract: In this paper, we investigate state consensus problems for discrete-time multi-agent systems with changing communications topologies and bounded time-varying communication delays. The analysis in this paper is based on the properties of non-negative matrices. We first extend the model of networks of dynamic agents to the case with multiple time-delays and prove that if the communication topology, time-delays, and weighting factors are time-invariant, then the necessary and sufficient condition that the multi-agent system solves a consensus problem is that the communication topology, represented by a directed graph, has spanning trees. Then we allow for dynamically changing communication topologies and bounded time-varying communication delays, and present some sufficient conditions for state consensus of system. Finally, as a special case of our model, the problem of asynchronous information exchange is also discussed.

340 citations


Journal ArticleDOI
TL;DR: Simulation results show that the hybridmultiagent system provides significant improvement in traffic conditions when evaluated against an existing traffic signal control algorithm as well as the SPSA-NN-based multiagent system as the complexity of the simulation scenario increases.
Abstract: Real-time traffic signal control is an integral part of the urban traffic control system, and providing effective real-time traffic signal control for a large complex traffic network is an extremely challenging distributed control problem. This paper adopts the multiagent system approach to develop distributed unsupervised traffic responsive signal control models, where each agent in the system is a local traffic signal controller for one intersection in the traffic network. The first multiagent system is developed using hybrid computational intelligent techniques. Each agent employs a multistage online learning process to update and adapt its knowledge base and decision-making mechanism. The second multiagent system is developed by integrating the simultaneous perturbation stochastic approximation theorem in fuzzy neural networks (NN). The problem of real-time traffic signal control is especially challenging if the agents are used for an infinite horizon problem, where online learning has to take place continuously once the agent-based traffic signal controllers are implemented into the traffic network. A comprehensive simulation model of a section of the Central Business District of Singapore has been developed using PARAMICS microscopic simulation program. Simulation results show that the hybrid multiagent system provides significant improvement in traffic conditions when evaluated against an existing traffic signal control algorithm as well as the SPSA-NN-based multiagent system as the complexity of the simulation scenario increases. Using the hybrid NN-based multiagent system, the mean delay of each vehicle was reduced by 78% and the mean stoppage time, by 85% compared to the existing traffic signal control algorithm. The promising results demonstrate the efficacy of the hybrid NN-based multiagent system in solving large-scale traffic signal control problems in a distributed manner

334 citations


Proceedings ArticleDOI
01 Dec 2006
TL;DR: A decentralized algorithm to increase the connectivity of a multi-agent system is proposed, where each agent receives information only from its neighbors, and uses this information to update its control law at each step of the iteration.
Abstract: In this paper we propose a decentralized algorithm to increase the connectivity of a multi-agent system. The connectivity property of the multi-agent system is quantified through the second smallest eigenvalue of the state dependent Laplacian of the proximity graph of agents. An exponential decay model is used to characterize the connection between agents. A supergradient algorithm is then used in conjunction with a recently developed decentralized algorithm for eigenvector computation to maximize the second smallest eigenvalue of the Laplacian of the proximity graph. A potential based control law is utilized to achieve the distances dictated by the supergradient algorithm. The algorithm is completely decentralized, where each agent receives information only from its neighbors, and uses this information to update its control law at each step of the iteration. Simulations demonstrate the effectiveness of the algorithm.

334 citations


Journal ArticleDOI
TL;DR: Multi-robot coalition formation results based upon simulation and actual robot experiments are reported, including a multi-agent coalition formation algorithm that has been demonstrated on an actual robot system.
Abstract: As the community strives towards autonomous multi-robot systems, there is a need for these systems to autonomously form coalitions to complete assigned missions. Numerous coalition formation algorithms have been proposed in the software agent literature. Algorithms exist that form agent coalitions in both super additive and non-super additive environments. The algorithmic techniques vary from negotiation-based protocols in multi-agent system (MAS) environments to those based on computation in distributed problem solving (DPS) environments. Coalition formation behaviors have also been discussed in relation to game theory. Despite the plethora of MAS coalition formation literature, to the best of our knowledge none of the proposed algorithms have been demonstrated with an actual multi-robot system. There exists a discrepancy between the multi-agent algorithms and their applicability to the multi-robot domain. This paper aims to bridge that discrepancy by unearthing the issues that arise while attempting to tailor these algorithms to the multi-robot domain. A well-known multi-agent coalition formation algorithm has been studied in order to identify the necessary modifications to facilitate its application to the multi-robot domain. This paper reports multi-robot coalition formation results based upon simulation and actual robot experiments. A multi-agent coalition formation algorithm has been demonstrated on an actual robot system

Journal ArticleDOI
01 Feb 2006
TL;DR: Simulation results show that Masbiole can obtain various kinds of behaviors and better performances than conventional MAS in MTT by evolution, and its characteristics are examined especially with an emphasis on the behaviors of agents obtained by symbiotic evolution.
Abstract: Multiagent Systems with Symbiotic Learning and Evolution (Masbiole) has been proposed and studied, which is a new methodology of Multiagent Systems (MAS) based on symbiosis in the ecosystem. Masbiole employs a method of symbiotic learning and evolution where agents can learn or evolve according to their symbiotic relations toward others, i.e., considering the benefits/losses of both itself and an opponent. As a result, Masbiole can escape from Nash Equilibria and obtain better performances than conventional MAS where agents consider only their own benefits. This paper focuses on the evolutionary model of Masbiole, and its characteristics are examined especially with an emphasis on the behaviors of agents obtained by symbiotic evolution. In the simulations, two ideas suitable for the effective analysis of such behaviors are introduced; "Match Type Tile-world (MTT)" and "Genetic Network Programming (GNP)". MTT is a virtual model where tile-world is improved so that agents can behave considering their symbiotic relations. GNP is a newly developed evolutionary computation which has the directed graph type gene structure and enables to analyze the decision making mechanism of agents easily. Simulation results show that Masbiole can obtain various kinds of behaviors and better performances than conventional MAS in MTT by evolution.

Journal ArticleDOI
01 Mar 2006
TL;DR: BioWar, a scalable citywide multiagent network numerical model, is described in this paper and simulates individuals as agents who are embedded in social, health, and professional networks and tracks the incidence of background and maliciously introduced diseases.
Abstract: While structured by social and institutional networks, disease outbreaks are modulated by physical, economical, technological, communication, health, and governmental infrastructures. To systematically reason about the nature of outbreaks, the potential outcomes of media, prophylaxis, and vaccination campaigns, and the relative value of various early warning devices, social context, and infrastructure, must be considered. Numerical models provide a cost-effective ethical system for reasoning about such events. BioWar, a scalable citywide multiagent network numerical model, is described in this paper. BioWar simulates individuals as agents who are embedded in social, health, and professional networks and tracks the incidence of background and maliciously introduced diseases. In addition to epidemiology, BioWar simulates health-care-seeking behaviors, absenteeism patterns, and pharmaceutical purchases, information useful for syndromic and behavioral surveillance algorithms.

Journal ArticleDOI
TL;DR: The multi-agent system paradigm is applied to collaborative negotiation in a global manufacturing supply chain network to solve coordination and negotiation issues involving multiple autonomous or semiautonomous problem solving agents.
Abstract: This paper applies the multi-agent system paradigm to collaborative negotiation in a global manufacturing supply chain network. Multi-agent computational environments are suitable for dealing with a broad class of coordination and negotiation issues involving multiple autonomous or semiautonomous problem solving agents. An agent-based multi-contract negotiation system is proposed for global manufacturing supply chain coordination. Also reported is a case study of mobile phone global manufacturing supply chain management.

Journal ArticleDOI
TL;DR: The authors discuss the experience of developing a multi-agent system that is robust enough for continual online use within the power industry.
Abstract: This paper reports on the use of multi-agent system technology to automate the management and analysis of SCADA and digital fault recorder (DFR) data. The multi-agent system, entitled Protection Engineering Diagnostic Agents (PEDA), integrates legacy intelligent systems that analyze SCADA and DFR data to provide data management and online diagnostic information to protection engineers. Since November 2004, PEDA agents have been intelligently interpreting and managing data online at a transmission system operator in the U.K. As the results presented in this paper demonstrate, PEDA supports protection engineers by providing access to interpreted power systems data via the corporate intranet within minutes of the data being received. In this paper, the authors discuss their experience of developing a multi-agent system that is robust enough for continual online use within the power industry. The use of existing agent development toolsets and standards is also discussed.

Journal ArticleDOI
TL;DR: In this paper, a case study from Chile is used as an example to demonstrate the potential of the MAS framework for water resources management in an efficient, equitable, and sustainable way.
Abstract: Due to the hydrological and socio-economic complexity of water use within river basins and even sub-basins, it is a considerable challenge to manage water resources in an efficient, equitable and sustainable way. This paper shows that multi-agent simulation (MAS) is a promising approach to better understand the complexity of water uses and water users within sub-basins. This approach is especially suitable to take the collective action into account when simulating the outcome of technical innovation and policy change. A case study from Chile is used as an example to demonstrate the potential of the MAS framework. Chile has played a pioneering role in water policy reform by privatizing water rights and promoting trade in such rights, devolving irrigation management authority to user groups, and privatizing the provision of irrigation infrastructure. The paper describes the different components of a MAS model developed for four micro-watersheds in the Maule river basin. Preliminary results of simulation experiments are presented, which show the impacts of technical change and of informal rental markets on household income and water use efficiency. The paper also discusses how the collective action problems in water markets and in small-scale and large-scale infrastructure provision can be captured by the MAS model. To promote the use of the MAS approach for planning purposes, a collaborative research and learning framework has been established, with a recently created multi-stakeholder platform at the regional level (Comision Regional de Recursos Hidricos) as the major partner. Finally, the paper discusses the potentials of using MAS models for water resources management, such as increasing transparency as an aspect of good governance. The challenges, for example the need to build trust in the model, are discussed as well.

Proceedings ArticleDOI
14 Jun 2006
TL;DR: The trajectories of the leaders can be viewed as exogenous control inputs, which allows to state and study questions concerning controllability and optimal control in heterogenous multi-agent applications.
Abstract: In this paper, we consider the situation where a collection of leaders dictate the motion of the followers in heterogenous multi-agent applications. In particular, the followers move according to a decentralized averaging rule, while the leaders' motion is unconstrained. Thus, the trajectories of the leaders can be viewed as exogenous control inputs, which allows us to state and study questions concerning controllability and optimal control.

Journal ArticleDOI
13 Sep 2006-Synthese
TL;DR: This paper proposes a perspective on practical reasoning as presumptive justification of a course of action, along with critical questions of this justification, building on the account of Walton, and articulate an interaction protocol, which is called PARMA, for dialogues over proposed actions based on this theory.
Abstract: In this paper we consider persuasion in the context of practical reasoning, and discuss the problems associated with construing reasoning about actions in a manner similar to reasoning about beliefs. We propose a perspective on practical reasoning as presumptive justification of a course of action, along with critical questions of this justification, building on the account of Walton. From this perspective, we articulate an interaction protocol, which we call PARMA, for dialogues over proposed actions based on this theory. We outline an axiomatic semantics for the PARMA Protocol, and discuss two implementations which use this protocol to mediate a discussion between humans. We then show how our proposal can be made computational within the framework of agents based on the Belief-Desire-Intention model, and illustrate this proposal with an example debate within a multi agent system.

Journal ArticleDOI
TL;DR: Simulation runs are conducted to compare the performance of the proposed MAS-based IPPS approaches and that of an evolutionary algorithm and it is shown that the hybrid-based MAS, with the introduction of supervisory control, is able to provide integrated process plan and job shop scheduling solutions with a better global performance.
Abstract: This paper is on the development of an agent-based approach for the dynamic integration of the process planning and scheduling functions. In consideration of the alternative processes and alternative machines for the production of each part, the actual selection of the schedule and allocation of manufacturing resources is achieved through negotiation among the part and machine agents which represent the parts and manufacturing resources, respectively. The agents are to negotiate on a fictitious cost with the adoption of a currency function. Two MAS architectures are evaluated in this paper. One is a simple MAS architecture comprises part agents and machine agents only; the other one involves the addition of a supervisor agent to establish a hybrid-based MAS architecture. A hybrid contract net protocol is developed in the paper to support both types of MAS architectures. This new negotiation protocol enables multi-task many-to-many negotiations, it also incorporates global control into the decentralized negotiation. Simulation runs are conducted to compare the performance of the proposed MAS-based IPPS approaches and that of an evolutionary algorithm. It also shows that the hybrid-based MAS, with the introduction of supervisory control, is able to provide integrated process plan and job shop scheduling solutions with a better global performance.

Journal Article
TL;DR: This paper presents mcmas, a model checker for Multi-Agent Systems that permits the automatic verification of specifications that use epistemic, correctness, and cooperation modalities, in addition to the standard temporal modalities.
Abstract: This paper presents mcmas, a model checker for Multi-Agent Systems (MAS). Differently from traditional model checkers, mcmas permits the automatic verification of specifications that use epistemic, correctness, and cooperation modalities, in addition to the standard temporal modalities. These additional modalities are used to capture properties of various scenarios (including communication and security protocols, games, etc.) that may be difficult or unnatural to express with temporal operators only; a small number of applications are presented in Section[4]. Agents are described in mcmas by means of the dedicated programming language ISPL (Interpreted Systems Programming Language). The approach is symbolic and uses ordered binary decision diagrams (obdds), thereby extending standard techniques for temporal logic to other modalities distinctive of agents. mcmas and all the examples presented in this paper are available for download [14] under the terms of the GPL license.

Book ChapterDOI
25 Mar 2006
TL;DR: Mcmas as discussed by the authors is a model checker for multi-agent systems that allows automatic verification of specifications that use epistemic, correctness, and cooperation modalities, in addition to the standard temporal modalities.
Abstract: This paper presents mcmas, a model checker for Multi-Agent Systems (MAS). Differently from traditional model checkers, mcmas permits the automatic verification of specifications that use epistemic, correctness, and cooperation modalities, in addition to the standard temporal modalities. These additional modalities are used to capture properties of various scenarios (including communication and security protocols, games, etc.) that may be difficult or unnatural to express with temporal operators only; a small number of applications are presented in Section[4]. Agents are described in mcmas by means of the dedicated programming language ISPL (Interpreted Systems Programming Language). The approach is symbolic and uses ordered binary decision diagrams (obdds), thereby extending standard techniques for temporal logic to other modalities distinctive of agents. mcmas and all the examples presented in this paper are available for download [14] under the terms of the GPL license.

Journal ArticleDOI
21 Aug 2006
TL;DR: In this paper, the authors discuss the issues involved in adapting Sliding Autonomy concepts to coordinated multiagent teams, where remote human operators have the ability to join, or leave, the team at will to assist the autonomous agents with their tasks (or aspects of their tasks).
Abstract: Recent research in human-robot interaction has investigated the concept of Sliding, or Adjustable, Autonomy, a mode of operation bridging the gap between explicit teleoperation and complete robot autonomy. This work has largely been in single-agent domains-involving only one human and one robot-and has not examined the issues that arise in multiagent domains. We discuss the issues involved in adapting Sliding Autonomy concepts to coordinated multiagent teams. In our approach, remote human operators have the ability to join, or leave, the team at will to assist the autonomous agents with their tasks (or aspects of their tasks) while not disrupting the team's coordination. Agents model their own and the human operator's performance on subtasks to enable them to determine when to request help from the operator. To validate our approach, we present the results of two experiments. The first evaluates the human/multirobot team's performance under four different collaboration strategies including complete teleoperation, pure autonomy, and two distinct versions of Sliding Autonomy. The second experiment compares a variety of user interface configurations to investigate how quickly a human operator can attain situational awareness when asked to help. The results of these studies support our belief that by incorporating a remote human operator into multiagent teams, the team as a whole becomes more robust and efficient

Posted Content
TL;DR: It is argued that agent-based participatory simulations are also a significant improvement over the MAS/RPG approach, opening new perspectives and solving some of the problems generated by the joint use of role-playing games and multi-agent systems.
Abstract: In 2001, Olivier Barreteau proposed to jointly use multi-agent systems and role-playing games for purposes of research, training and negotiation support in the field of renewable resource management. This joint use was later labeled the "MAS/RPG methodology" and this approach is one of the foundation stones of the ComMod movement. In this article, we present an alternative method called "agent-based participatory simulations". These simulations are multi-agent systems where human participants control some of the agents. The experiments we conducted prove that it is possible to successfully merge multi-agent systems and role-playing games. We argue that agent-based participatory simulations are also a significant improvement over the MAS/RPG approach, opening new perspectives and solving some of the problems generated by the joint use of role-playing games and multi-agent systems. The advantages are at least threefold. Because all interactions are computer mediated, they can be recorded and this record can be processed and used to improve the understanding of participants and organizers alike. Because of the merge, agent-based participatory simulations decrease the distance between the agent-based model and the behavior of participants. Agent-based participatory simulations allow for computer-based improvements such as the introduction of eliciting assistant agents with learning capabilities.

Proceedings ArticleDOI
14 Jun 2006
TL;DR: It is shown that it is always possible to design a control law as the gradient of a suitably-defined navigation function whose minimum corresponds to the desired configuration.
Abstract: We propose a decentralized cooperative controller for a group of mobile agents. The control design is based on the navigation function formalism. The aim of the group control law is to generate a pattern or formation in a given workspace while avoiding obstacles and collisions. The desired goal is specified in terms of distances among the agents. We show that it is always possible to design a control law as the gradient of a suitably-defined navigation function whose minimum corresponds to the desired configuration. Furthermore in certain cases, such as when the topology of the interconnection is an acyclic graph, this minimum is unique. Some simulations are shown to test the strategy.

Journal ArticleDOI
Jun Ota1
TL;DR: This study investigates the current trends and the potentials for multi-agent robot systems, such as multi-robot motion-planning algorithms and exploration algorithms of multiple robots.

Book ChapterDOI
TL;DR: The abstract model and architecture of CArtAgO, a framework for developing artifact-based working environments for multiagent systems (MAS), and a first Java-based prototype technology is discussed.
Abstract: This paper describes CArtAgO, a framework for developing artifact-based working environments for multiagent systems (MAS) The framework is based on the notion of artifact, as a basic abstraction to model and engineer objects, resources and tools designed to be used and manipulated by agents at run-time to support their working activities, in particular the cooperative ones CArtAgO enables MAS engineers to design and develop suitable artifacts, and to extend existing agent platforms with the possibility to create artifact-based working environments, programming agents to exploit them In this paper, first the abstract model and architecture of CArtAgO is described, then a first Java-based prototype technology is discussed

Journal ArticleDOI
TL;DR: This paper tries to answer the question by proposing a new notion called ‘Soft Control’, which keeps the local rule of the existing agents in the system, and shows the feasibility of soft control by a case study.
Abstract: This paper asks a new question: how can we control the collective behavior of self-organized multi-agent systems? We try to answer the question by proposing a new notion called ‘Soft Control’, which keeps the local rule of the existing agents in the system. We show the feasibility of soft control by a case study. Consider the simple but typical distributed multi-agent model proposed by Vicsek et al. for flocking of birds: each agent moves with the same speed but with different headings which are updated using a local rule based on the average of its own heading and the headings of its neighbors. Most studies of this model are about the self-organized collective behavior, such as synchronization of headings. We want to intervene in the collective behavior (headings) of the group by soft control. A specified method is to add a special agent, called a ‘Shill’, which can be controlled by us but is treated as an ordinary agent by other agents. We construct a control law for the shill so that it can synchronize the whole group to an objective heading. This control law is proved to be effective analytically and numerically. Note that soft control} is different from the approach of distributed control}. It is a natural way to intervene in the distributed systems. It may bring out many interesting issues and challenges on the control of complex systems.

Journal ArticleDOI
TL;DR: It is shown how evolutionary dynamics from Evolutionary Game Theory can help the developer of a MAS in good choices of parameter settings of the used RL algorithms and how the improved results for MAS RL in COIN, and a developed extension, are predicted by the ED.
Abstract: In this paper, we investigate Reinforcement learning (RL) in multi-agent systems (MAS) from an evolutionary dynamical perspective. Typical for a MAS is that the environment is not stationary and the Markov property is not valid. This requires agents to be adaptive. RL is a natural approach to model the learning of individual agents. These Learning algorithms are however known to be sensitive to the correct choice of parameter settings for single agent systems. This issue is more prevalent in the MAS case due to the changing interactions amongst the agents. It is largely an open question for a developer of MAS of how to design the individual agents such that, through learning, the agents as a collective arrive at good solutions. We will show that modeling RL in MAS, by taking an evolutionary game theoretic point of view, is a new and potentially successful way to guide learning agents to the most suitable solution for their task at hand. We show how evolutionary dynamics (ED) from Evolutionary Game Theory can help the developer of a MAS in good choices of parameter settings of the used RL algorithms. The ED essentially predict the equilibriums outcomes of the MAS where the agents use individual RL algorithms. More specifically, we show how the ED predict the learning trajectories of Q-Learners for iterated games. Moreover, we apply our results to (an extension of) the COllective INtelligence framework (COIN). COIN is a proved engineering approach for learning of cooperative tasks in MASs. The utilities of the agents are re-engineered to contribute to the global utility. We show how the improved results for MAS RL in COIN, and a developed extension, are predicted by the ED.

Proceedings ArticleDOI
08 May 2006
TL;DR: Certified Reputation works by allowing agents to actively provide third-party references about their previous performance as a means of building up the trust in them of their potential interaction partners, and helps agents pick better interaction partners more quickly than models that do not incorporate this form of trust.
Abstract: Current computational trust models are usually built either on an agent's direct experience of an interaction partner (interaction trust) or reports provided by third parties about their experiences with a partner (witness reputation). However, both of these approaches have their limitations. Models using direct experience often result in poor performance until an agent has had a sufficient number of interactions to build up a reliable picture of a particular partner and witness reports rely on self-interested agents being willing to freely share their experience. To this end, this paper presents Certified Reputation (CR), a novel model of trust that can overcome these limitations. Specifically, CR works by allowing agents to actively provide third-party references about their previous performance as a means of building up the trust in them of their potential interaction partners. By so doing, trust relationships can quickly be established with very little cost to the involved parties. Here we empirically evaluate CR and show that it helps agents pick better interaction partners more quickly than models that do not incorporate this form of trust.

Proceedings ArticleDOI
14 Jun 2006
TL;DR: This paper establishes an existence theorem for the connectivity maintenance problem by introducing a novel state-dependent graph, called the double-integrator disk graph, and designs a distributed "flow-control" algorithm to compute optimal connectivity-maintaining controls.
Abstract: In this paper we consider ad-hoc networks of robotic agents with double integrator dynamics. For such networks, the connectivity maintenance problems are: (i) do there exist control inputs for each agent to maintain network connectivity, and (ii) given desired controls for each agent, can one compute the closest connectivity-maintaining controls in a distributed fashion. The proposed solution is based on three contributions. First, we define and characterize admissible sets for double integrators to remain inside disks. Second, we establish an existence theorem for the connectivity maintenance problem by introducing a novel state-dependent graph, called the double-integrator disk graph. Finally, we design a distributed "flow-control" algorithm to compute optimal connectivity-maintaining controls.