Showing papers on "Multi-agent system published in 2019"

SoPhie: An Attentive GAN for Predicting Paths Compliant to Social and Physical Constraints

Abstract: This paper addresses the problem of path prediction for multiple interacting agents in a scene, which is a crucial step for many autonomous platforms such as self-driving cars and social robots. We present SoPhie; an interpretable framework based on Generative Adversarial Network (GAN), which leverages two sources of information, the path history of all the agents in a scene, and the scene context information, using images of the scene. To predict a future path for an agent, both physical and social information must be leveraged. Previous work has not been successful to jointly model physical and social interactions. Our approach blends a social attention mechanism with physical attention that helps the model to learn where to look in a large scene and extract the most salient parts of the image relevant to the path. Whereas, the social attention component aggregates information across the different agent interactions and extracts the most important trajectory information from the surrounding neighbors. SoPhie also takes advantage of GAN to generates more realistic samples and to capture the uncertain nature of the future paths by modeling its distribution. All these mechanisms enable our approach to predict socially and physically plausible paths for the agents and to achieve state-of-the-art performance on several different trajectory forecasting benchmarks.

Dynamic Event-Triggered and Self-Triggered Control for Multi-agent Systems

Abstract: We propose two novel dynamic event-triggered control laws to solve the average consensus problem for first-order continuous-time multiagent systems over undirected graphs. Compared with the most existing triggering laws, the proposed laws involve internal dynamic variables, which play an essential role in guaranteeing that the triggering time sequence does not exhibit Zeno behavior. Moreover, some existing triggering laws are special cases of ours. For the proposed self-triggered algorithm, continuous agent listening is avoided as each agent predicts its next triggering time and broadcasts it to its neighbors at the current triggering time. Thus, each agent only needs to sense and broadcast at its triggering times, and to listen to and receive incoming information from its neighbors at their triggering times. It is proved that the proposed triggering laws make the state of each agent converge exponentially to the average of the agents’ initial states if and only if the underlying graph is connected. Numerical simulations are provided to illustrate the effectiveness of the theoretical results.

Output Consensus of Heterogeneous Linear Multi-Agent Systems with Adaptive Event-Triggered Control

Abstract: This paper investigates the output consensus problem for heterogeneous linear multi-agent systems via event-triggered control. By introducing a dynamic compensator for each agent, a fully distributed event-triggered control strategy with an adaptive event-triggering mechanism is proposed. It is shown that under the proposed control strategy, all agents asymptotically achieve output consensus with intermittent communication in a fully distributed manner. Moreover, with the proposed event-triggering mechanism, Zeno behavior is strictly excluded for each agent. Compared with existing mechanisms, the proposed event-triggering mechanism is independent of any global information and avoids the continuous monitoring issue. Finally, a numerical example is provided to illustrate the effectiveness of the proposed event-triggered control strategy.

Distributed Optimization for Multiagent Systems: An Edge-Based Fixed-Time Consensus Approach

Abstract: This paper deals with the problem of distributed optimization for multiagent systems by using an edge-based fixed-time consensus approach. In the case of time-invariant cost functions, a new distributed protocol is proposed to achieve the state agreement in a fixed time while the sum of local convex functions known to individual agents is minimized. In the case of time-varying cost functions, based on the new distributed protocol in the case of time-invariant cost functions, a distributed protocol is provided by taking the Hessian matrix into account. In both cases, stability conditions are derived to ensure that the distributed optimization problem is solved under both fixed and switching communication topologies. A distinctive feature of the results in this paper is that an upper bound of settling time for consensus can be estimated without dependence on initial states of agents, and thus can be made arbitrarily small through adjusting system parameters. Therefore, the results in this paper can be applicable in an unknown environment such as drone rendezvous within a required time for military purpose while optimizing local objectives. Case studies of a power output agreement for battery packages are provided to demonstrate the effectiveness of the theoretical results.

Model Free Adaptive Iterative Learning Consensus Tracking Control for a Class of Nonlinear Multiagent Systems

Abstract: This paper proposes a distributed model free adaptive iterative learning control (MFAILC) method for a class of unknown nonlinear multiagent systems to perform consensus tracking. Here, both fixed and iteration-varying topologies are considered and only a subset of followers can access the desired trajectory in each topology. To design the control protocol, the agent’s dynamic is first transformed into a dynamic linearization model along the iteration axis, and then a distributed MFAILC scheme is constructed to guarantee that all agents can track the desired trajectory. Through rigorous analysis, it is shown that under this novel distributed MFAILC scheme, the tracking errors of all agents are convergent along the iteration axis. The main merit of this design is that consensus tracking task can be achieved only utilizing the input/output data of the multiagent system. Three examples are given to validate the effectiveness of the proposed design.

CoLight: Learning Network-level Cooperation for Traffic Signal Control

Abstract: Cooperation among the traffic signals enables vehicles to move through intersections more quickly. Conventional transportation approaches implement cooperation by pre-calculating the offsets between two intersections. Such pre-calculated offsets are not suitable for dynamic traffic environments. To enable cooperation of traffic signals, in this paper, we propose a model, CoLight, which uses graph attentional networks to facilitate communication. Specifically, for a target intersection in a network, CoLight can not only incorporate the temporal and spatial influences of neighboring intersections to the target intersection, but also build up index-free modeling of neighboring intersections. To the best of our knowledge, we are the first to use graph attentional networks in the setting of reinforcement learning for traffic signal control and to conduct experiments on the large-scale road network with hundreds of traffic signals. In experiments, we demonstrate that by learning the communication, the proposed model can achieve superior performance against the state-of-the-art methods.

PRIMAL: Pathfinding via Reinforcement and Imitation Multi-Agent Learning

Abstract: Multi-agent path finding (MAPF) is an essential component of many large-scale, real-world robot deployments, from aerial swarms to warehouse automation. However, despite the community's continued efforts, most state-of-the-art MAPF planners still rely on centralized planning and scale poorly past a few hundred agents. Such planning approaches are maladapted to real-world deployments, where noise and uncertainty often require paths be recomputed online, which is impossible when planning times are in seconds to minutes. We present PRIMAL, a novel framework for MAPF that combines reinforcement and imitation learning to teach fully decentralized policies, where agents reactively plan paths online in a partially observable world while exhibiting implicit coordination. This framework extends our previous work on distributed learning of collaborative policies by introducing demonstrations of an expert MAPF planner during training, as well as careful reward shaping and environment sampling. Once learned, the resulting policy can be copied onto any number of agents and naturally scales to different team sizes and world dimensions. We present results on randomized worlds with up to 1024 agents and compare success rates against state-of-the-art MAPF planners. Finally, we experimentally validate the learned policies in a hybrid simulation of a factory mockup, involving both real world and simulated robots.

Consensus Control of General Linear Multiagent Systems With Antagonistic Interactions and Communication Noises

Abstract: This technical note considers a consensus problem of high-order multiagent systems with antagonistic interactions and communication noises. The interaction network associated with the multiagent system is modeled by a signed graph (called coopetition network) and the agent dynamics is described by a general linear system. A novel stochastic-approximation based control strategy is designed for each agent by using the relative state information from its neighbors. Additionally, convergence of a consensus error system is analyzed by the stochastic stability theory. Finally, the effectiveness of our results is demonstrated by simulation.

Counterfactual Critic Multi-Agent Training for Scene Graph Generation

Abstract: Scene graphs --- objects as nodes and visual relationships as edges --- describe the whereabouts and interactions of objects in an image for comprehensive scene understanding. To generate coherent scene graphs, almost all existing methods exploit the fruitful visual context by modeling message passing among objects. For example, ``person'' on ``bike'' can help to determine the relationship ``ride'', which in turn contributes to the confidence of the two objects. However, we argue that the visual context is not properly learned by using the prevailing cross-entropy based supervised learning paradigm, which is not sensitive to graph inconsistency: errors at the hub or non-hub nodes should not be penalized equally. To this end, we propose a Counterfactual critic Multi-Agent Training (CMAT) approach. CMAT is a multi-agent policy gradient method that frames objects into cooperative agents, and then directly maximizes a graph-level metric as the reward. In particular, to assign the reward properly to each agent, CMAT uses a counterfactual baseline that disentangles the agent-specific reward by fixing the predictions of other agents. Extensive validations on the challenging Visual Genome benchmark show that CMAT achieves a state-of-the-art performance by significant gains under various settings and metrics.

Fully Distributed Formation-Containment Control of Heterogeneous Linear Multiagent Systems

Abstract: This technical note addresses the distributed time-varying formation-containment control problem for heterogeneous general linear multiagent systems (the virtual leader, multileaders, and followers) based on the output regulation framework from an observer viewpoint under the directed topology, which contains a spanning tree. All agents can have different dynamics and different state dimensions. A new format of time-varying formation (TVF) shape is proposed. The multileaders are required to achieve the TVF with tracking the virtual leader, whose output is only available to a subset of them, and only need to send the information of their designed observers and TVF shapes to their neighboring followers. A new class of distributed adaptive observer-based controllers is designed with the mild assumption that both leaders and followers are introspective (i.e., agents have knowledge of their own outputs). Compared with the existing works, one main contribution is that the controllers are fully distributed with the proposition of TVF shapes. The simulation to multivehicle systems is also provided to verify the effectiveness of theoretical results.

Bearing-Only Formation Tracking Control of Multiagent Systems

Abstract: This paper studies the problem of bearing-only formation control of multiagent systems, where the control of each agent merely relies on the relative bearings to its neighbors. Although this problem has received increasing research attention recently, it is still unsolved to a large extent due to its highly nonlinear dynamics. In particular, the existing control approaches are only able to solve the simplest scenario where the target formation is stationary and each agent is modeled as a single integrator. The main contribution of this paper is to propose new bearing-only formation control laws to 1) track moving target formations and 2) handle a variety of agent models including single-integrator, double-integrator, and unicycle models. These control laws are an important step towards the application of bearing-only formation control in practical tasks. Both numerical simulation and real experimental results are presented to verify the effectiveness of the theoretical results.

Optimal Output Consensus of High-Order Multiagent Systems With Embedded Technique

Abstract: In this paper, we study an optimal output consensus problem for a multiagent network with agents in the form of multi-input multioutput minimum-phase dynamics. Optimal output consensus can be taken as an extended version of the existing output consensus problem for higher-order agents with an optimization requirement, where the output variables of agents are driven to achieve a consensus on the optimal solution of a global cost function. To solve this problem, we first construct an optimal signal generator, and then propose an embedded control scheme by embedding the generator in the feedback loop. We give two kinds of algorithms based on different available information along with both state feedback and output feedback, and prove that these algorithms with the embedded technique can guarantee the solvability of the problem for high-order multiagent systems under standard assumptions.

Bearing Rigidity Theory and Its Applications for Control and Estimation of Network Systems: Life Beyond Distance Rigidity

Abstract: Distributed control and location estimation of multiagent systems have received tremendous research attention in recent years because of their potential across many application domains [1], [2]. The term agent can represent a sensor, autonomous vehicle, or any general dynamical system. Multiagent systems are attractive because of their robustness against system failure, ability to adapt to dynamic and uncertain environments, and economic advantages compared to the implementation of more expensive monolithic systems.

Data-Based Optimal Control of Multiagent Systems: A Reinforcement Learning Design Approach

Abstract: This paper studies an optimal consensus tracking problem of heterogeneous linear multiagent systems. By introducing tracking error dynamics, the optimal tracking problem is reformulated as finding a Nash-equilibrium solution to multiplayer games, which can be done by solving associated coupled Hamilton–Jacobi equations. A data-based error estimator is designed to obtain the data-based control for the multiagent systems. Using the quadratic functional to approximate every agent’s value function, we can obtain the optimal cooperative control by the input–output (I/O) ${Q}$ -learning algorithm with a value iteration technique in the least-square sense. The control law solves the optimal consensus problem for multiagent systems with measured I/O information, and does not rely on the model of multiagent systems. A numerical example is provided to illustrate the effectiveness of the proposed algorithm.

Optimal Containment Control of Unknown Heterogeneous Systems With Active Leaders

Abstract: This brief presents a partially model-free solution to the distributed containment control of multiagent systems using off-policy reinforcement learning (RL). The followers are assumed to be heterogeneous with different dynamics, and the leaders are assumed to be active in the sense that their control inputs can be nonzero. Optimality is explicitly imposed in solving the containment problem to not only drive the agents’ states into a convex hull of the leaders’ states but also minimize their transient responses. Inhomogeneous algebraic Riccati equations (AREs) are derived to solve the optimal containment control with active leaders. The resulting control protocol for each agent depends on its own state and an estimation of an interior point inside the convex hull spanned by the leaders. This estimation is provided by designing a distributed observer for a trajectory inside the convex hull of active leaders. Only the knowledge of the leaders’ dynamics is required by the observer. An off-policy RL algorithm is developed to solve the inhomogeneous AREs online in real time without requiring any knowledge of the followers’ dynamics. Finally, a simulation example is presented to show the effectiveness of the presented algorithm.

Control Barrier Functions for Multi-Agent Systems Under Conflicting Local Signal Temporal Logic Tasks

Abstract: Motivated by the recent interest in cyber-physical and interconnected autonomous systems, we study the problem of dynamically coupled multi-agent systems under conflicting local signal temporal logic (STL) tasks. Each agent is assigned a local STL task regardless of the tasks that the other agents are assigned to. Such a task may be dependent, i.e., the satisfaction of the task may depend on the behavior of more than one agent, so that the satisfaction of the conjunction of all local tasks may be conflicting. We propose a hybrid feedback control strategy using time-varying control barrier functions. Our control strategy finds least violating solutions in the aforementioned conflicting situations based on a suitable robustness notion and by initiating collaboration among agents.

On the Control of Multi-Agent Systems: A Survey

Abstract: Multi-Agent Systems (MAS) use networked multiple autonomous agents to accomplish complex tasks in areas such as space-based applications, smart grids, and machine learning. The overall system goal is achieved using local interactions among the agents. The last two decades have witnessed rapid development of MASs in automatic control. Tracing the roots of such systems back more than 50 years, this monograph provides the reader with an in-depth and comprehensive survey of the research in Multi-Agent Systems. The focus is on the research conducted in the two decades. It introduces the basic concepts and definitions to the reader before going on to describe how MAS has been used in most forms of systems. The monograph offers a concise reference for understanding the use of MASs and the contemporary research issues for further investigation. In addition to covering the basic theory, the authors also cover applications in multi-robot systems, sensor networks, smart grid, machine learning, social networks, and many-core microprocessors. On the Control of Multi-Agent Systems provides researchers and students in systems and control a modern, comprehensive survey of one of the most important current day topics.

Event-Triggered Cooperative Output Regulation of Linear Multi-Agent Systems Under Jointly Connected Topologies

Abstract: This paper addresses the cooperative output regulation problem of linear multi-agent systems under switching communication topologies. A distributed event-triggered control scheme is proposed so that the cooperative output regulation problem is solved with only intermittent communication. The communication topology is not required to be connected at every time instant under the jointly connected assumption. With the proposed triggering mechanism, each agent only transmits the information to its neighbors at its own triggering times or the switching times. By introducing a fixed timer, Zeno behavior is strictly excluded for each agent. The effectiveness of the proposed control scheme is demonstrated by an example.

Self-Triggered Leader-Following Consensus for High-Order Nonlinear Multiagent Systems via Dynamic Output Feedback Control

Abstract: This paper investigates the event-based leader-following consensus problem for high-order nonlinear multiagent systems whose dynamics are in strict feedback forms and satisfy Lipschitz condition. By using self-triggered control scheme and dynamic output feedback control method in combination, a new class of distributed self-triggered consensus protocols is proposed based only on the relative output measurements of neighboring agents. It is noted that the proposed protocols only require the output information of neighboring agents to be shared and the designed self-triggered algorithm can avoid continuous communication among neighboring agents, thus the communication cost is reduced significantly. Sufficient conditions in terms of matrix inequalities are derived to guarantee the exponential leader-following consensus. The effectiveness of the theoretical results is illustrated through a simulation example.

Event Triggered Adaptive Fuzzy Consensus for Interconnected Switched Multiagent Systems

Abstract: This paper concentrates on the problem of event triggered adaptive leader-following control for a class of multiagent systems. The considered systems contain completely unknown interconnections and arbitrary asynchronously switching. To handle these nonlinearties, a new hierarchical barrier Lyapunov method is proposed. Based on this method, a novel adaptive fuzzy control strategy is designed to make the output tracking errors of the multiagent systems converge to a small neighborhood of origin. Stability analysis shows that the proposed method can guarantee the compact set conditions of the fuzzy logic systems during the entire design process. Meanwhile, to reduce the computational burden, novel event triggered mechanisms are presented for both the transmission between two connected agents, and the communication between the sensor and the controller in one agent. An illustrative example is presented to verify the effectiveness of the proposed controller.

Distributed Reinforcement Learning for Multi-robot Decentralized Collective Construction

Abstract: Inspired by recent advances in single agent reinforcement learning, this paper extends the single-agent asynchronous advantage actor-critic (A3C) algorithm to enable multiple agents to learn a homogeneous, distributed policy, where agents work together toward a common goal without explicitly interacting. Our approach relies on centralized policy and critic learning, but decentralized policy execution, in a fully-observable system. We show that the sum of experience of all agents can be leveraged to quickly train a collaborative policy that naturally scales to smaller and larger swarms. We demonstrate the applicability of our method on a multi-robot construction problem, where agents need to arrange simple block elements to build a user-specified structure. We present simulation results where swarms of various sizes successfully construct different test structures without the need for additional training.

Intelligent E-commerce logistics platform using hybrid agent based approach

Abstract: This paper proposes a hybrid agent-based approach for the scheduling and synchronization of e-commerce logistics parks (EcLP). This is accomplished by integrating intelligent distribution centers within the e-commerce environment. The proposed platform has been developed based on an agent technology, which not only serves for decentralization and synchronization purposes but also it has been optimized for the transportation and logistics of the overall system. Moreover, mobile agent-based communication mechanisms between the hardware agents and the software agents were developed, and the proposed hybrid agent-based platform was implemented and tested based on a case study. Following this, the results were compared to a conventional system based on four main indicators.