Showing papers on "Multi-agent system published in 2010"

Constrained Consensus and Optimization in Multi-Agent Networks

Abstract: We present distributed algorithms that can be used by multiple agents to align their estimates with a particular value over a network with time-varying connectivity. Our framework is general in that this value can represent a consensus value among multiple agents or an optimal solution of an optimization problem, where the global objective function is a combination of local agent objective functions. Our main focus is on constrained problems where the estimates of each agent are restricted to lie in different convex sets. To highlight the effects of constraints, we first consider a constrained consensus problem and present a distributed "projected consensus algorithm" in which agents combine their local averaging operation with projection on their individual constraint sets. This algorithm can be viewed as a version of an alternating projection method with weights that are varying over time and across agents. We establish convergence and convergence rate results for the projected consensus algorithm. We next study a constrained optimization problem for optimizing the sum of local objective functions of the agents subject to the intersection of their local constraint sets. We present a distributed "projected subgradient algorithm" which involves each agent performing a local averaging operation, taking a subgradient step to minimize its own objective function, and projecting on its constraint set. We show that, with an appropriately selected stepsize rule, the agent estimates generated by this algorithm converge to the same optimal solution for the cases when the weights are constant and equal, and when the weights are time-varying but all agents have the same constraint set.

Population Games and Evolutionary Dynamics

Abstract: This text offers a systematic, rigorous, and unified presentation of evolutionary game theory, covering the core developments of the theory from its inception in biology in the 1970s through recent advances. Evolutionary game theory, which studies the behavior of large populations of strategically interacting agents, is used by economists to make predictions in settings where traditional assumptions about agents' rationality and knowledge may not be justified. Recently, computer scientists, transportation scientists, engineers, and control theorists have also turned to evolutionary game theory, seeking tools for modeling dynamics in multiagent systems. Population Games and Evolutionary Dynamics provides a point of entry into the field for researchers and students in all of these disciplines. The text first considers population games, which provide a simple, powerful model for studying strategic interactions among large numbers of anonymous agents. It then studies the dynamics of behavior in these games. By introducing a general model of myopic strategy revision by individual agents, the text provides foundations for two distinct approaches to aggregate behavior dynamics: the deterministic approach, based on differential equations, and the stochastic approach, based on Markov processes. Key results on local stability, global convergence, stochastic stability, and nonconvergence are developed in detail. Ten substantial appendixes present the mathematical tools needed to work in evolutionary game theory, offering a practical introduction to the methods of dynamic modeling. Accompanying the text are more than 200 color illustrations of the mathematics and theoretical results; many were created using the Dynamo software suite, which is freely available on the author's Web site. Readers are encouraged to use Dynamo to run quick numerical experiments and to create publishable figures for their own research.

Finite-Time Consensus Problems for Networks of Dynamic Agents

Abstract: In this note, we discuss finite-time state consensus problems for multi-agent systems and present one framework for constructing effective distributed protocols, which are continuous state feedbacks. By employing the theory of finite-time stability, we investigate both the bidirectional interaction case and the unidirectional interaction case, and prove that if the sum of time intervals, in which the interaction topology is connected, is sufficiently large, the proposed protocols will solve the finite-time consensus problems.

A Review of the Applications of Agent Technology in Traffic and Transportation Systems

Abstract: The agent computing paradigm is rapidly emerging as one of the powerful technologies for the development of large-scale distributed systems to deal with the uncertainty in a dynamic environment. The domain of traffic and transportation systems is well suited for an agent-based approach because transportation systems are usually geographically distributed in dynamic changing environments. Our literature survey shows that the techniques and methods resulting from the field of agent and multiagent systems have been applied to many aspects of traffic and transportation systems, including modeling and simulation, dynamic routing and congestion management, and intelligent traffic control. This paper examines an agent-based approach and its applications in different modes of transportation, including roadway, railway, and air transportation. This paper also addresses some critical issues in developing agent-based traffic control and management systems, such as interoperability, flexibility, and extendibility. Finally, several future research directions toward the successful deployment of agent technology in traffic and transportation systems are discussed.

An Introduction to Multi-Agent Systems

Abstract: Multi-agent systems is a subfield of Distributed Artificial Intelligence that has experienced rapid growth because of the flexibility and the intelligence available solve distributed problems. In this chapter, a brief survey of multi-agent systems has been presented. These encompass different attributes such as architecture, communication, coordination strategies, decision making and learning abilities. The goal of this chapter is to provide a quick reference to assist in the design of multi-agent systems and to highlight the merit and demerits of the existing methods.

Reinforcement learning-based multi-agent system for network traffic signal control

Abstract: A challenging application of artificial intelligence systems involves the scheduling of traffic signals in multi-intersection vehicular networks. This paper introduces a novel use of a multi-agent system and reinforcement learning (RL) framework to obtain an efficient traffic signal control policy. The latter is aimed at minimising the average delay, congestion and likelihood of intersection cross-blocking. A five-intersection traffic network has been studied in which each intersection is governed by an autonomous intelligent agent. Two types of agents, a central agent and an outbound agent, were employed. The outbound agents schedule traffic signals by following the longest-queue-first (LQF) algorithm, which has been proved to guarantee stability and fairness, and collaborate with the central agent by providing it local traffic statistics. The central agent learns a value function driven by its local and neighbours' traffic conditions. The novel methodology proposed here utilises the Q-Learning algorithm with a feedforward neural network for value function approximation. Experimental results clearly demonstrate the advantages of multi-agent RL-based control over LQF governed isolated single-intersection control, thus paving the way for efficient distributed traffic signal control in complex settings.

A Distributed Control Approach to A Robust Output Regulation Problem for Multi-Agent Linear Systems

Abstract: In this note, the robust output regulation problem of a multi-agent system is considered. An internal model based distributed control scheme is adopted to achieve the objectives of asymptotic tracking and disturbance rejection in an uncertain multi-agent system where both the reference inputs and disturbances are generated by an exosystem.

Ad hoc autonomous agent teams: collaboration without pre-coordination

Abstract: As autonomous agents proliferate in the real world, both in software and robotic settings, they will increasingly need to band together for cooperative activities with previously unfamiliar teammates. In such ad hoc team settings, team strategies cannot be developed a priori. Rather, an agent must be prepared to cooperate with many types of teammates: it must collaborate without pre-coordination. This paper challenges the AI community to develop theory and to implement prototypes of ad hoc team agents. It defines the concept of ad hoc team agents, specifies an evaluation paradigm, and provides examples of possible theoretical and empirical approaches to challenge. The goal is to encourage progress towards this ambitious, newly realistic, and increasingly important research goal.

Indexability of Restless Bandit Problems and Optimality of Whittle Index for Dynamic Multichannel Access

Abstract: In this paper, we consider a class of restless multiarmed bandit processes (RMABs) that arises in dynamic multichannel access, user/server scheduling, and optimal activation in multiagent systems. For this class of RMABs, we establish the indexability and obtain Whittle index in closed form for both discounted and average reward criteria. These results lead to a direct implementation of Whittle index policy with remarkably low complexity. When arms are stochastically identical, we show that Whittle index policy is optimal under certain conditions. Furthermore, it has a semiuniversal structure that obviates the need to know the Markov transition probabilities. The optimality and the semiuniversal structure result from the equivalence between Whittle index policy and the myopic policy established in this work. For nonidentical arms, we develop efficient algorithms for computing a performance upper bound given by Lagrangian relaxation. The tightness of the upper bound and the near-optimal performance of Whittle index policy are illustrated with simulation examples.

Neural-Network-Based Adaptive Leader-Following Control for Multiagent Systems With Uncertainties

Abstract: A neural-network-based adaptive approach is proposed for the leader-following control of multiagent systems. The neural network is used to approximate the agent's uncertain dynamics, and the approximation error and external disturbances are counteracted by employing the robust signal. When there is no control input constraint, it can be proved that all the following agents can track the leader's time-varying state with the tracking error as small as desired. Compared with the related work in the literature, the uncertainty in the agent's dynamics is taken into account; the leader's state could be time-varying; and the proposed algorithm for each following agent is only dependent on the information of its neighbor agents. Finally, the satisfactory performance of the proposed method is illustrated by simulation examples.

Distributed Coordination of Networked Fractional-Order Systems

Abstract: This paper studies the distributed coordination of networked fractional-order systems over a directed interaction graph. A general fractional-order coordination model is introduced by summarizing three different cases: 1) fractional-order agent dynamics with integer-order coordination algorithms; 2) fractional-order agent dynamics with fractional-order coordination algorithms; and 3) integer-order agent dynamics with fractional-order coordination algorithms. We show sufficient conditions on the interaction graph and the fractional order such that coordination can be achieved using the general model. The coordination equilibrium is also explicitly given. In addition, we characterize the relationship between the number of agents and the fractional order to ensure coordination. Furthermore, we compare the convergence speed of coordination for fractional-order systems with that for integer-order systems. It is shown that the convergence speed of the fractional-order coordination algorithms can be improved by varying the fractional orders with time. Finally, simulation results are presented as a proof of concept.

An agent-based approach to model land-use change at a regional scale

Abstract: Land-use/cover change (LUCC) is a complex process that includes actors and factors at different social and spatial levels. A common approach to analyse and simulate LUCC as the result of individual decisions is agent-based modelling (ABM). However, ABM is often applied to simulate processes at local scales, while its application in regional studies is limited. This paper describes first a conceptual framework for ABM to analyse and explore regional LUCC processes. Second, the conceptual framework is represented by combining different concepts including agent typologies, farm trajectories and probabilistic decision-making processes. Finally, the framework is illustrated through a case study in the Netherlands, where processes of farm cessation, farm expansion and farm diversification are shaping the structure of the landscape. The framework is a generic, straightforward approach to analyse and explore regional LUCC with an explicit link to empirical approaches for parameterization of ABM.

Agent-based modeling in ecological economics.

Abstract: Interconnected social and environmental systems are the domain of ecological economics, and models can be used to explore feedbacks and adaptations inherent in these systems. Agent-based modeling (ABM) represents autonomous entities, each with dynamic behavior and heterogeneous characteristics. Agents interact with each other and their environment, resulting in emergent outcomes at the macroscale that can be used to quantitatively analyze complex systems. ABM is contributing to research questions in ecological economics in the areas of natural resource management and land-use change, urban systems modeling, market dynamics, changes in consumer attitudes, innovation, and diffusion of technology and management practices, commons dilemmas and self-governance, and psychological aspects to human decision making and behavior change. Frontiers for ABM research in ecological economics involve advancing the empirical calibration and validation of models through mixed methods, including surveys, interviews, participatory modeling, and, notably, experimental economics to test specific decision-making hypotheses. Linking ABM with other modeling techniques at the level of emergent properties will further advance efforts to understand dynamics of social-environmental systems.

Distributed Q-Learning for Aggregated Interference Control in Cognitive Radio Networks

Abstract: This paper deals with the problem of aggregated interference generated by multiple cognitive radios (CRs) at the receivers of primary (licensed) users. In particular, we consider a secondary CR system based on the IEEE 802.22 standard for wireless regional area networks (WRANs), and we model it as a multiagent system where the multiple agents are the different secondary base stations in charge of controlling the secondary cells. We propose a form of real-time multiagent reinforcement learning, which is known as decentralized Q-learning, to manage the aggregated interference generated by multiple WRAN systems. We consider both situations of complete and partial information about the environment. By directly interacting with the surrounding environment in a distributed fashion, the multiagent system is able to learn, in the first case, an efficient policy to solve the problem and, in the second case, a reasonably good suboptimal policy. Computational and memory requirement considerations are also presented, discussing two different options for uploading and processing the learning information. Simulation results, which are presented for both the upstream and downstream cases, reveal that the proposed approach is able to fulfill the primary-user interference constraints, without introducing signaling overhead in the system.

ASPECS: an agent-oriented software process for engineering complex systems

Abstract: Holonic multiagent systems (HMAS) offer a promising software engineering approach for developing complex open software systems. However the process of building Multi-Agent Systems (MAS) and HMAS is mostly different from the process of building more traditional software systems as it introduces new design and development challenges. This paper introduces an agent-oriented software process for engineering complex systems called ASPECS. ASPECS is based on a holonic organisational metamodel and provides a step-by-step guide from requirements to code allowing the modelling of a system at different levels of details using a set of refinement methods. This paper details the entire ASPECS development process and provides a set of methodological guidelines for each process activity. A complete case study is also used to illustrate the design process and the associated notations. ASPECS uses UML as a modelling language. Because of the specific needs of agents and holonic organisational design, the UML semantics and notation are used as reference points, but they have been extended by introducing new specific profiles.

A Class of Bounded Distributed Control Strategies for Connectivity Preservation in Multi-Agent Systems

Abstract: This technical note proposes a general class of distributed potential-based control laws with the connectivity preserving property for single-integrator agents. The potential functions are designed in such a way that when an edge in the information flow graph is about to lose connectivity, the gradient of the potential function lies in the direction of that edge, aiming to shrink it. The results are developed for a static information flow graph first, and then are extended to the case of dynamic edge addition. Connectivity preservation for problems involving static leaders is covered as well. The potential functions are chosen to be smooth, resulting in bounded control inputs. Other constraints may also be imposed on the potential functions to satisfy various design criteria such as consensus, containment, and formation convergence. The effectiveness of the proposed control strategy is illustrated by simulation for examples of consensus and containment.

Instrumenting multi-agent organisations with organisational artifacts and agents

Abstract: The social and organisational aspects of agency have led to a good amount of theoretical work in terms of formal models and theories However, the conception and engineering of proper organisational infrastructures embodying such models and theories are still an open issue The introduction of normative concerns with requirements of openness and adaptation stresses this issue The corresponding mechanisms for the current infrastructures appear to be not appropriate for managing distributed and open normative organisations There is still the need of proper abstractions and tools to facilitate application agents taking part in the monitoring of the organisation on one hand, and in the adaptation and definition of the organisation in which they are situated on the other hand In this paper we present and discuss ORA4MAS (Organisational Artifacts for Multi-Agent Systems), a proposed approach aiming at these issues Based on the Agents and Artifacts meta-model (A&A), it introduces organisational artifacts as first class entities to instrument the organisation for supporting agents activities within it

Urban traffic signal control using reinforcement learning agents

Abstract: This study presents a distributed multi-agent-based traffic signal control for optimising green timing in an urban arterial road network to reduce the total travel time and delay experienced by vehicles. The proposed multi-agent architecture uses traffic data collected by sensors at each intersection, stored historical traffic patterns and data communicated from agents in adjacent intersections to compute green time for a phase. The parameters like weights, threshold values used in computing the green time is fine tuned by online reinforcement learning with an objective to reduce overall delay. PARAMICS software was used as a platform to simulate 29 signalised intersection at Central Business District of Singapore and test the performance of proposed multi-agent traffic signal control for different traffic scenarios. The proposed multi-agent reinforcement learning (RLA) signal control showed significant improvement in mean time delay and speed in comparison to other traffic control system like hierarchical multi-agent system (HMS), cooperative ensemble (CE) and actuated control.

Stochastic Consensus Seeking With Noisy and Directed Inter-Agent Communication: Fixed and Randomly Varying Topologies

Abstract: We consider consensus seeking of networked agents on directed graphs where each agent has only noisy measurements of its neighbors' states. Stochastic approximation type algorithms are employed so that the individual states converge both in mean square and almost surely to the same limit. We further generalize the algorithm to networks with random link failures and prove convergence results.

Continuous-Time Average-Preserving Opinion Dynamics with Opinion-Dependent Communications

Abstract: We study a simple continuous-time multiagent system related to Krause's model of opinion dynamics: each agent holds a real value, and this value is continuously attracted by every other value differing from it by less than 1, with an intensity proportional to the difference. We prove convergence to a set of clusters, with the agents in each cluster sharing a common value, and provide a lower bound on the distance between clusters at a stable equilibrium, under a suitable notion of multiagent system stability. To better understand the behavior of the system for a large number of agents, we introduce a variant involving a continuum of agents. We prove, under some conditions, the existence of a solution to the system dynamics, convergence to clusters, and a nontrivial lower bound on the distance between clusters. Finally, we establish that the continuum model accurately represents the asymptotic behavior of a system with a finite but large number of agents.

Decentralized Coordination in RoboCup Rescue

Abstract: Emergency responders are faced with a number of significant challenges when managing major disasters. First, the number of rescue tasks posed is usually larger than the number of responders (or agents) and the resources available to them. Second, each task is likely to require a different level of effort in order to be completed by its deadline. Third, new tasks may continually appear or disappear from the environment, thus requiring the responders to quickly recompute their allocation of resources. Fourth, forming teams or coalitions of multiple agents from different agencies is vital since no single agency will have all the resources needed to save victims, unblock roads and extinguish the fires which might erupt in the disaster space. Given this, coalitions have to be efficiently selected and scheduled to work across the disaster space so as to maximize the number of lives and the portion of the infrastructure saved. In particular, it is important that the selection of such coalitions should be performed in a decentralized fashion in order to avoid a single point of failure in the system. Moreover, it is critical that responders communicate only locally given they are likely to have limited battery power or minimal access to long-range communication devices. Against this background, we provide a novel decentralized solution to the coalition formation process that pervades disaster management. More specifically, we model the emergency management scenario defined in the RoboCup Rescue disaster simulation platform as a coalition formation with spatial and temporal constraints (CFST) problem where agents form coalitions to complete tasks, each with different demands. To design a decentralized algorithm for CFST, we formulate it as a distributed constraint optimization problem and show how to solve it using the state-of-the-art Max-Sum algorithm that provides a completely decentralized message-passing solution. We then provide a novel algorithm (F-Max-Sum) that avoids sending redundant messages and efficiently adapts to changes in the environment. In empirical evaluations, our algorithm is shown to generate better solutions than other decentralized algorithms used for this problem.

An agent-based approach to care in independent living

Abstract: This paper presents a multi-agent system for the care of elderly people living at home on their own, with the aim to prolong their independence. The system is composed of seven groups of agents providing a reliable, robust and flexible monitoring by sensing the user in the environment, reconstructing the position and posture to create the physical awareness of the user in the environment, reacting to critical situations, calling for help in the case of an emergency, and issuing warnings if unusual behavior is detected. The system has been tested during several on-line demonstrations.

Controllability analysis of multi-agent systems using relaxed equitable partitions

Abstract: This paper investigates how to make decentralised networks, amenable to external control, i.e., how to ensure that they are appropriately organised so that they can be effectively 'reprogrammed'. In particular, we study networked systems whose interaction dynamics are given by a nearest-neighbour averaging rule, with one leader node providing the control input to the entire system. The main result is a necessary and sufficient condition for the controllability of such systems in terms of the graph topology. In particular, we give a graph theoretic interpretation of the controllability properties through the so-called relaxed equitable partition.

Asynchronous Distributed Optimization With Event-Driven Communication

Abstract: We consider problems where multiple agents cooperate to control their individual state so as to optimize a common objective while communicating with each other to exchange state information. Since communication costs can be significant, especially when the agents are wireless devices with limited energy, we seek conditions under which communication of state information among nodes can be restricted while still ensuring that the optimization process converges. We propose an asynchronous (event-driven) optimization scheme that limits communication to instants when some state estimation error function at a node exceeds a threshold and prove that, under certain conditions, such convergence is guaranteed when communication delays are negligible. We subsequently extend the analysis to include communication delays as long as they are bounded. We apply this approach to a sensor network coverage control problem where the objective is to maximize the probability of detecting events occurring in a given region and show that the proposed asynchronous approach may significantly reduce communication costs, hence also prolonging the system's lifetime, without any performance degradation.