scispace - formally typeset
Search or ask a question

Showing papers on "Multi-agent system published in 2017"


Journal ArticleDOI
05 Apr 2017-PLOS ONE
TL;DR: The present work shows that Deep Q-Networks can become a useful tool for studying decentralized learning of multiagent systems coping with high-dimensional environments and describes the progression from competitive to collaborative behavior when the incentive to cooperate is increased.
Abstract: Evolution of cooperation and competition can appear when multiple adaptive agents share a biological, social, or technological niche. In the present work we study how cooperation and competition emerge between autonomous agents that learn by reinforcement while using only their raw visual input as the state representation. In particular, we extend the Deep Q-Learning framework to multiagent environments to investigate the interaction between two learning agents in the well-known video game Pong. By manipulating the classical rewarding scheme of Pong we show how competitive and collaborative behaviors emerge. We also describe the progression from competitive to collaborative behavior when the incentive to cooperate is increased. Finally we show how learning by playing against another adaptive agent, instead of against a hard-wired algorithm, results in more robust strategies. The present work shows that Deep Q-Networks can become a useful tool for studying decentralized learning of multiagent systems coping with high-dimensional environments.

577 citations


Journal ArticleDOI
TL;DR: An event-triggered formation protocol is delicately proposed by using only locally triggered sampled data in a distributed manner and the state formation control problem is cast into an asymptotic stability problem of a reduced-order closed-loop system.
Abstract: This paper addresses the distributed formation control problem of a networked multi-agent system (MAS) subject to limited communication resources. First, a dynamic event-triggered communication mechanism (DECM) is developed to schedule inter-agent communication such that some unnecessary data exchanges among agents can be reduced so as to achieve better resource efficiency. Different from most of the existing event-triggered communication mechanisms, wherein threshold parameters are fixed all the time, the threshold parameter in the developed event triggering condition is dynamically adjustable in accordance with a dynamic rule. It is numerically shown that the proposed DECM can achieve a better tradeoff between reducing inter-agent communication frequency and preserving an expected formation than some existing ones. Second, an event-triggered formation protocol is delicately proposed by using only locally triggered sampled data in a distributed manner. Based on the formation protocol, it is shown that the state formation control problem is cast into an asymptotic stability problem of a reduced-order closed-loop system. Then, criteria for designing desired formation protocol and communication mechanism are derived. Finally, the effectiveness and advantages of the proposed approach are demonstrated through a comparative study in multirobot formation control.

448 citations


Proceedings ArticleDOI
01 May 2017
TL;DR: In this article, a value network is proposed to estimate the estimated time to the goal given an agent's joint configuration (positions and velocities) with its neighbors, and the value network not only admits efficient (i.e., realtime implementable) queries for finding a collision-free velocity vector, but also considers the uncertainty in the other agents' motion.
Abstract: Finding feasible, collision-free paths for multiagent systems can be challenging, particularly in non-communicating scenarios where each agent's intent (e.g. goal) is unobservable to the others. In particular, finding time efficient paths often requires anticipating interaction with neighboring agents, the process of which can be computationally prohibitive. This work presents a decentralized multiagent collision avoidance algorithm based on a novel application of deep reinforcement learning, which effectively offloads the online computation (for predicting interaction patterns) to an offline learning procedure. Specifically, the proposed approach develops a value network that encodes the estimated time to the goal given an agent's joint configuration (positions and velocities) with its neighbors. Use of the value network not only admits efficient (i.e., real-time implementable) queries for finding a collision-free velocity vector, but also considers the uncertainty in the other agents' motion. Simulation results show more than 26% improvement in paths quality (i.e., time to reach the goal) when compared with optimal reciprocal collision avoidance (ORCA), a state-of-the-art collision avoidance strategy.

328 citations


Proceedings ArticleDOI
20 Mar 2017
TL;DR: This work poses a cooperative ‘image guessing’ game between two agents who communicate in natural language dialog so that Q-BOT can select an unseen image from a lineup of images and shows the emergence of grounded language and communication among ‘visual’ dialog agents with no human supervision.
Abstract: We introduce the first goal-driven training for visual question answering and dialog agents. Specifically, we pose a cooperative ‘image guessing’ game between two agents – Q-BOT and A-BOT– who communicate in natural language dialog so that Q-BOT can select an unseen image from a lineup of images. We use deep reinforcement learning (RL) to learn the policies of these agents end-to-end – from pixels to multi-agent multi-round dialog to game reward.,,We demonstrate two experimental results.,,First, as a ‘sanity check’ demonstration of pure RL (from scratch), we show results on a synthetic world, where the agents communicate in ungrounded vocabularies, i.e., symbols with no pre-specified meanings (X, Y, Z). We find that two bots invent their own communication protocol and start using certain symbols to ask/answer about certain visual attributes (shape/color/style). Thus, we demonstrate the emergence of grounded language and communication among ‘visual’ dialog agents with no human supervision.,,Second, we conduct large-scale real-image experiments on the VisDial dataset [5], where we pretrain on dialog data with supervised learning (SL) and show that the RL finetuned agents significantly outperform supervised pretraining. Interestingly, the RL Q-BOT learns to ask questions that A-BOT is good at, ultimately resulting in more informative dialog and a better team.

297 citations


Journal ArticleDOI
TL;DR: A data-based adaptive dynamic programming method is presented using the current and past system data rather than the accurate system models also instead of the traditional identification scheme which would cause the approximation residual errors.
Abstract: This paper investigates the optimal consensus control problem for discrete-time multi-agent systems with completely unknown dynamics by utilizing a data-driven reinforcement learning method. It is known that the optimal consensus control for multi-agent systems relies on the solution of the coupled Hamilton–Jacobi–Bellman equation, which is generally impossible to be solved analytically. Even worse, most real-world systems are too complicated to obtain accurate mathematical models. To overcome these deficiencies, a data-based adaptive dynamic programming method is presented using the current and past system data rather than the accurate system models also instead of the traditional identification scheme which would cause the approximation residual errors. First, we establish a discounted performance index and formulate the optimal consensus problem via Bellman optimality principle. Then, we introduce the policy iteration algorithm which motivates this paper. To implement the proposed online action-dependent heuristic dynamic programming method, two neural networks (NNs), 1) critic NN and 2) actor NN, are employed to approximate the iterative performance index functions and control policies, respectively, in real time. Finally, two simulation examples are provided to demonstrate the effectiveness of the proposed method.

287 citations


Journal ArticleDOI
TL;DR: This paper investigates the problem of network-based leader-following consensus of nonlinear multi-agent systems via distributed impulsive control by taking network-induced delays into account and derives a general consensus criterion.

270 citations


Journal ArticleDOI
TL;DR: This paper addresses the output consensus problem of heterogeneous linear multi-agent systems by introducing a fixed timer into both event- and self-triggered control schemes, so that Zeno behavior can be ruled out for each agent.
Abstract: This paper addresses the output consensus problem of heterogeneous linear multi-agent systems. We first propose a novel distributed event-triggered control scheme. It is shown that, with the proposed control scheme, the output consensus problem can be solved if two matrix equations are satisfied. Then, we further propose a novel self-triggered control scheme, with which continuous monitoring is avoided. By introducing a fixed timer into both event- and self-triggered control schemes, Zeno behavior can be ruled out for each agent. The effectiveness of the event- and self-triggered control schemes is illustrated by an example.

260 citations


Journal ArticleDOI
TL;DR: This paper investigates distributed adaptive consensus tracking control without such requirements for nonlinear high-order multi-agent systems subjected to mismatched unknown parameters and uncertain external disturbances by introducing compensating terms in a smooth function form of consensus errors and certain positive integrable functions in each step of virtual control design.

248 citations


Journal ArticleDOI
TL;DR: It is shown that the leader-following consensus problem with stochastic sampling can be transferred into a master-slave synchronization problem with only one master system and two slave systems.
Abstract: This paper is concerned with sampled-data leader-following consensus of a group of agents with nonlinear characteristic. A distributed consensus protocol with probabilistic sampling in two sampling periods is proposed. First, a general consensus criterion is derived for multiagent systems under a directed graph. A number of results in several special cases without transmittal delays or with the deterministic sampling are obtained. Second, a dimension-reduced condition is obtained for multiagent systems under an undirected graph. It is shown that the leader-following consensus problem with stochastic sampling can be transferred into a master–slave synchronization problem with only one master system and two slave systems. The problem solving is independent of the number of agents, which greatly facilitates its application to large-scale networked agents. Third, the network design issue is further addressed, demonstrating the positive and active roles of the network structure in reaching consensus. Finally, two examples are given to verify the theoretical results.

247 citations


Journal ArticleDOI
TL;DR: It is proved that all agents with any initial state can reach output consensus at an optimal solution to the given constrained optimization problem, provided that the graph describing the communication links among agents is undirected and connected.
Abstract: This technical note presents a continuous-time multi-agent system for distributed optimization with an additive objective function composed of individual objective functions subject to bound, equality, and inequality constraints. Each individual objective function is assumed to be convex in the region defined by its local bound constraints only without the need to be globally convex. All agents in the system communicate using a proportional-integral protocol with their output information instead of state information to reduce communication bandwidth. It is proved that all agents with any initial state can reach output consensus at an optimal solution to the given constrained optimization problem, provided that the graph describing the communication links among agents is undirected and connected. It is further proved that the system with only integral protocol is also convergent to the unique optimal solution if each individual objective function is strictly convex. Simulation results are presented to substantiate the theoretical results.

236 citations


Journal ArticleDOI
TL;DR: It is shown that most human decision sub-models are not explicitly based on a specific theory and if so they are mostly based on economic theories, such as the rational actor, and mainly ignoring other relevant disciplines.
Abstract: Recent reviews stated that the complex and context-dependent nature of human decision-making resulted in ad-hoc representations of human decision in agent-based land use change models (LUCC ABMs) and that these representations are often not explicitly grounded in theory. However, a systematic survey on the characteristics (e.g. uncertainty, adaptation, learning, interactions and heterogeneities of agents) of representing human decision-making in LUCC ABMs is missing. Therefore, the aim of this study is to inform this debate by reviewing 134 LUCC ABM papers. We show that most human decision sub-models are not explicitly based on a specific theory and if so they are mostly based on economic theories, such as the rational actor, and mainly ignoring other relevant disciplines. Consolidating and enlarging the theoretical basis for modelling human decision-making may be achieved by using a structural framework for modellers, re-using published decision models, learning from other disciplines and fostering collaboration with social scientists. Review on human decisions in agent-based models of land use change.Most human decision models are not based on theory.Used theories are mainly from economics and not from psychology.A framework to guide modellers how to implement human decision is needed.

Journal ArticleDOI
TL;DR: A delicate convex optimization algorithm in terms of recursive linear matrix inequalities is proposed to design desired consensus protocol and event-based mechanism for networked multi-agent systems subject to limited communication resources and unknown-but-bounded process and measurement noise.
Abstract: This paper addresses the problem of leader-following consensus for networked multi-agent systems subject to limited communication resources and unknown-but-bounded process and measurement noise. First, a new distributed event-based communication mechanism on the basis of a time-varying threshold parameter is developed to schedule transmission of each sensor's measurement through a communication network so as to alleviate consecutive occupancy of communication resources. Second, a novel concept of set-membership leader-following consensus is put forward, through which the true states of all followers are guaranteed to always reside in a bounding ellipsoidal set of the leader's state. Third, in the case that full information of followers’ states are not measurable, a distributed observer-based consensus protocol is presented to provide a set-membership estimation of each follower's state. Then, based on a recursive computation of confidence state estimation ellipsoids and leader state ellipsoid, a delicate convex optimization algorithm in terms of recursive linear matrix inequalities is proposed to design desired consensus protocol and event-based mechanism. Finally, an illustrative example is given to show the effectiveness and advantage of the developed approach.

Journal ArticleDOI
TL;DR: In this article, the authors argue that a new generation of holonic energy systems is required to orchestrate the interplay between these dense, diverse and distributed energy components, which promotes the systemic features of autonomy, belonging, connectivity, diversity and emergence, and balances global and local system objectives.
Abstract: The energy landscape is experiencing accelerating change; centralized energy systems are being decarbonized, and transitioning towards distributed energy systems, facilitated by advances in power system management and information and communication technologies. This paper elaborates on these generations of energy systems by critically reviewing relevant authoritative literature. This includes a discussion of modern concepts such as ‘smart grid’, ‘microgrid’, ‘virtual power plant’ and ‘multi-energy system’, and the relationships between them, as well as the trends towards distributed intelligence and interoperability. Each of these emerging urban energy concepts holds merit when applied within a centralized grid paradigm, but very little research applies these approaches within the emerging energy landscape typified by a high penetration of distributed energy resources, prosumers (consumers and producers), interoperability, and big data. Given the ongoing boom in these fields, this will lead to new challenges and opportunities as the status-quo of energy systems changes dramatically. We argue that a new generation of holonic energy systems is required to orchestrate the interplay between these dense, diverse and distributed energy components. The paper therefore contributes a description of holonic energy systems and the implicit research required towards sustainability and resilience in the imminent energy landscape. This promotes the systemic features of autonomy, belonging, connectivity, diversity and emergence, and balances global and local system objectives, through adaptive control topologies and demand responsive energy management. Future research avenues are identified to support this transition regarding interoperability, secure distributed control and a system of systems approach.

Journal ArticleDOI
TL;DR: The aim of the proposed problem is to design time-varying output-feedback controllers such that, at each time step, the mean-square consensus index of the closed-loop multi-agent system satisfies the pre-specified upper bound constraints subject to certain triggering mechanism.
Abstract: In this technical note, the consensus control problem is investigated for a class of discrete time-varying stochastic multi-agent system subject to sensor saturations. An event-based mechanism is adopted where each agent updates the control input signal only when the pre-specified triggering condition is violated. To reflect the time-varying manner and characterize the transient consensus behavior, a new index for mean-square consensus is put forward to quantify the deviation level from individual agent to the average value of all agents’ states. For a fixed network topology, the aim of the proposed problem is to design time-varying output-feedback controllers such that, at each time step, the mean-square consensus index of the closed-loop multi-agent system satisfies the pre-specified upper bound constraints subject to certain triggering mechanism. Both the existence conditions and the explicit expression of the desired controllers are established by resorting to the solutions to a set of recursive matrix inequalities. An illustrative simulation example is utilized to demonstrate the usefulness of the proposed algorithms.

Journal ArticleDOI
TL;DR: This note proposes an adaptive method to relax such a requirement to allow non-identical control directions, under the condition that some control directions are known.
Abstract: Existing Nussbaum function based results on consensus of multi-agent systems require that the unknown control directions of all the agents should be the same. This note proposes an adaptive method to relax such a requirement to allow non-identical control directions, under the condition that some control directions are known. Technically, a novel idea is proposed to construct a new Nussbaum function, from which a conditional inequality is developed to handle time-varying input gains. Then, the inequality is integrated with adaptive control technique such that the proposed Nussbaum function for each agent is adaptively updated. Moreover, in addition to parametric uncertainties, each agent has non-parametric bounded modelling errors which may include external disturbances and approximation errors of static input nonlinearities. Even in the presence of such uncertainties, the proposed control scheme is still able to ensure the states of all the agents asymptotically reach perfect consensus. Finally, simulation study is performed to show the effectiveness of the proposed approach.

Journal ArticleDOI
TL;DR: A robust decentralized control law of minimal complexity is proposed that achieves prescribed, arbitrarily fast and accurate synchronization of the following agents with the leader.
Abstract: In this paper, we consider the synchronization control problem for uncertain high-order nonlinear multi-agent systems in a leader-follower scheme, under a directed communication protocol. A robust decentralized control law of minimal complexity is proposed that achieves prescribed, arbitrarily fast and accurate synchronization of the following agents with the leader. The control protocol is decentralized in the sense that the control signal of each agent is calculated based solely on local relative state information from its neighborhood set. Additionally, no information regarding the agents' dynamic model is employed in the design procedure. Moreover, provided that the communication graph is connected and contrary to the related works on multi-agent systems, the controller-imposed transient and steady state performance bounds are fully decoupled from: 1) the underlying graph topology, 2) the control gains selection, and 3) the agents' model uncertainties, and are solely prescribed by certain designer-specified performance functions. Extensive simulation results clarify and verify the approach.

Journal ArticleDOI
TL;DR: A review of state-of-the-art applications and trends in multi-agent system and smart microgrids and several combinatorial optimization problems opened to be improved and discussed along the next coming years are presented.

Journal ArticleDOI
TL;DR: A novel distributed event-triggered control scheme is developed to solve the cooperative output regulation problem of heterogeneous MASs and an internal reference model for each agent is proposed, such that continuous monitoring of measurement errors can be avoided.
Abstract: In this paper, we consider the cooperative output regulation problem of heterogeneous linear multi-agent systems (MASs) by event-triggered control. We first develop an event-triggering mechanism for leader-following consensus of homogeneous MASs. Then by proposing an internal reference model for each agent, a novel distributed event-triggered control scheme is developed to solve the cooperative output regulation problem of heterogeneous MASs. Furthermore, a novel self-triggered control scheme is also proposed, such that continuous monitoring of measurement errors can be avoided. The feasibility of both proposed control schemes is studied by excluding Zeno behavior for each agent. An example is finally provided to demonstrate the effectiveness of the control schemes.

Journal ArticleDOI
TL;DR: The proposed algorithm is extended to solve the adaptive finite-time bipartite consensus tracking problem for leader–follower case by designing distributed finite- time estimator.

Journal ArticleDOI
TL;DR: The proposed algorithm for formation of multiple linear second-order agents with collision avoidance and obstacle avoidance with recursive feasibility of the resulting optimization problem is guaranteed and closed-loop stability of the whole system is ensured.
Abstract: The paper is concerned with the problem of distributed model predictive control (DMPC) for formation of multiple linear second-order agents with collision avoidance and obstacle avoidance. All the agents are permitted to implement optimization simultaneously at each time step. The assumed input trajectory and state trajectory are introduced to obtain a computationally tractable optimization problem in a distributed manner. As a result, a compatibility constraint is required to ensure the consistency between each agent׳s real operation and its plan and to establish the agreement among agents. The terminal ingredients are tailored by making use of the specific form of the system model and the control objective. The terminal set is ensured to be positively invariant with the designed terminal controller. The collision avoidance constraint and the obstacle avoidance constraint are satisfied for any state in the terminal set. The weighted matrix of the terminal cost is determined by solving a Lyapunov equation. Moreover, recursive feasibility of the resulting optimization problem is guaranteed and closed-loop stability of the whole system is ensured. Finally, a numerical example is given to illustrate the effectiveness of the proposed algorithm.

Journal ArticleDOI
TL;DR: A decentralized controller that uses event-triggered communication scheduling is developed for the leader-follower consensus problem under fixed and switching communication topologies and analysis is provided to show that Zeno behavior is avoided by developing a positive constant lower bound on the minimum inter-event interval.
Abstract: A decentralized controller that uses event-triggered communication scheduling is developed for the leader-follower consensus problem under fixed and switching communication topologies. To eliminate continuous interagent communication, state estimates of neighboring agents are designed for control feedback and are updated via communication to reset growing estimate errors. The communication times are based on an event-triggered approach and are adapted based on the trade-off between the control system performance and the desire to minimize the amount of communication. An important aspect of the developed event trigger strategy is that communication is not required to determine when a state update is needed. Since the control strategy produces switched dynamics, analysis is provided to show that Zeno behavior is avoided by developing a positive constant lower bound on the minimum inter-event interval. A Lyapunov-based convergence analysis is also provided to indicate bounded convergence of the developed control methodology.

Journal ArticleDOI
TL;DR: This paper develops an off-policy reinforcement learning (RL) algorithm to solve optimal synchronization of multiagent systems by using the framework of graphical games and shows that the optimal distributed policies found by the proposed algorithm satisfy the global Nash equilibrium and synchronize all agents to the leader.
Abstract: This paper develops an off-policy reinforcement learning (RL) algorithm to solve optimal synchronization of multiagent systems. This is accomplished by using the framework of graphical games. In contrast to traditional control protocols, which require complete knowledge of agent dynamics, the proposed off-policy RL algorithm is a model-free approach, in that it solves the optimal synchronization problem without knowing any knowledge of the agent dynamics. A prescribed control policy, called behavior policy, is applied to each agent to generate and collect data for learning. An off-policy Bellman equation is derived for each agent to learn the value function for the policy under evaluation, called target policy, and find an improved policy, simultaneously. Actor and critic neural networks along with least-square approach are employed to approximate target control policies and value functions using the data generated by applying prescribed behavior policies. Finally, an off-policy RL algorithm is presented that is implemented in real time and gives the approximate optimal control policy for each agent using only measured data. It is shown that the optimal distributed policies found by the proposed algorithm satisfy the global Nash equilibrium and synchronize all agents to the leader. Simulation results illustrate the effectiveness of the proposed method.

Journal ArticleDOI
TL;DR: A class of novel event-triggered dynamic encoding and decoding algorithms is proposed, based on which a kind of consensus protocol is presented and it is shown that the asymptotic convergence rate is related to the scale of the network, the number of quantization levels, the system parameter, and the network structure.
Abstract: Communication data rates and energy constraints are two important factors that have to be considered in the coordination control of multiagent networks. Although some encoder–decoder-based consensus protocols are available, there still exists a fundamental theoretical problem: how can we further reduce the update rate of control input for each agent without the changing consensus performance? In this paper, we consider the problem of average consensus over directed and time-varying digital networks of discrete-time first-order multiagent systems with limited communication data transmission rates. Each agent has a real-valued state but can only exchange binary symbolic sequence with its neighbors due to bandwidth constraints. A class of novel event-triggered dynamic encoding and decoding algorithms is proposed, based on which a kind of consensus protocol is presented. Moreover, we develop a scheme to select the numbers of time-varying quantization levels for each connected communication channel in the time-varying directed topologies at each time step. The analytical relation among system and network parameters is characterized explicitly. It is shown that the asymptotic convergence rate is related to the scale of the network, the number of quantization levels, the system parameter, and the network structure. It is also found that under the designed event-triggered protocol, for a directed and time-varying digital network, which uniformly contains a spanning tree over a time interval, the average consensus can be achieved with an exponential convergence rate based on merely 1-b information exchange between each pair of adjacent agents at each time step.

Journal ArticleDOI
TL;DR: In this article, a multi-agent system consisting of $N$ agents is considered and the problem of steering each agent from its initial position to a desired goal while avoiding collisions with obstacles and other agents is studied.
Abstract: A multi-agent system consisting of $N$ agents is considered. The problem of steering each agent from its initial position to a desired goal while avoiding collisions with obstacles and other agents is studied. This problem, referred to as the multi-agent collision avoidance problem , is formulated as a differential game. Dynamic feedback strategies that approximate the feedback Nash equilibrium solutions of the differential game are constructed and it is shown that, provided certain assumptions are satisfied, these guarantee that the agents reach their targets while avoiding collisions.

Journal ArticleDOI
TL;DR: It is shown that the algorithm is robust to arbitrarily bounded communication delays and arbitrarily switching communication graphs provided that the union of the graphs has directed spanning trees among each certain time interval.
Abstract: In this technical note, a distributed velocity-constrained consensus problem is studied for discrete-time multi-agent systems, where each agent's velocity is constrained to lie in a nonconvex set. A distributed constrained control algorithm is proposed to enable all agents to converge to a common point using only local information. The gains of the algorithm for all agents need not to be the same or predesigned and can be adjusted by each agent itself based on its own and neighbors' information. It is shown that the algorithm is robust to arbitrarily bounded communication delays and arbitrarily switching communication graphs provided that the union of the graphs has directed spanning trees among each certain time interval. The analysis approach is based on multiple novel model transformations, proper control parameter selections, boundedness analysis of state-dependent stochastic matrices1, exploitation of the convexity of stochastic matrices, and the joint connectivity of the communication graphs. Numerical examples are included to illustrate the theoretical results.

Journal ArticleDOI
TL;DR: In this paper, an event-triggered consensus protocol is proposed to avoid the need for continuous communication between agents and provide a decentralized method for transmission of information in the presence of time-varying communication delays, where each agent decides its own broadcasting time instants based on local information.
Abstract: Multi-agent systems' cooperation to achieve global goals is usually limited by sensing, actuation, and communication issues. At the local level, continuous measurement and actuation is only approximated by the use of digital mechanisms that measure and process information in order to compute and update new control input values at discrete time instants. Interaction with other agents takes place, in general, through a digital communication channel with limited bandwidth where transmission of continuous-time signals is not possible. This technical note considers the problem of consensus (or synchronization of state trajectories) of multi-agent systems that are described by general linear dynamics and are connected using undirected graphs. The proposed event-triggered consensus protocol not only avoids the need for continuous communication between agents but also provides a decentralized method for transmission of information in the presence of time-varying communication delays, where each agent decides its own broadcasting time instants based only on local information. This method gives more flexibility for scheduling information broadcasting compared to periodic and sampled-data implementations.

Journal ArticleDOI
TL;DR: The proposed algorithm is superior over consensus algorithms in terms of convergence speed and utilizes reduced communication infrastructure compared to centralized controllers, and can be deployed in real-world microgrids and offer superior decision making on optimal microgrid control.
Abstract: This paper proposes a multiagent-based optimal microgrid control scheme using a fully distributed diffusion strategy. A two-level cooperative optimization multiagent system is adapted for distributed energy resources economic dispatch. The lower level implements an adaptive droop scheme based on online no-load frequency adjustments. The upper level implements distributed communication using diffusion between neighboring agents for optimal microgrid management. The proposed control scheme enables peer-to-peer communication among the agents without the necessity of a centralized controller, and simultaneously performs resource optimization while regulating the system frequency. The results are compared with centralized and consensus-based optimization algorithms. We have concluded that the proposed algorithm is superior over consensus algorithms in terms of convergence speed and utilizes reduced communication infrastructure compared to centralized controllers. Simulation demonstrations were conducted along with experimental results from a hardware-based microgrid using an industrial multiagent framework. The simulation and experimental results show that the proposed method and the agent framework can be deployed in real-world microgrids and offer superior decision making on optimal microgrid control.

Journal ArticleDOI
TL;DR: Through nonsmooth Lyapunov analysis, it is shown that uniformly bounded consensus results are derived and the bound of the consensus error is adjustable by choosing suitable design parameters.
Abstract: In this paper, we propose a new distributed event-trigger consensus protocol for linear multiagent systems with external disturbances. Two consensus problems are considered: one is a leader–follower case and the other is a nonleader case. Different from the existing results, our proposed scheme enables each agent to decide when to transmit its state signals to its neighbors such that continuous communication between neighboring agents is avoided. Clearly, this can largely decrease the communication burden of the whole communication network. Besides, since the control signal for each agent is discontinuous because of the event-triggering mechanism, the existence of a solution for the closed-loop system in the classical sense may not be guaranteed. To solve this problem, we employ a nonsmooth analysis technique including differential inclusion and Filippov solution. Through nonsmooth Lyapunov analysis, it is shown that uniformly bounded consensus results are derived and the bound of the consensus error is adjustable by choosing suitable design parameters.

Journal ArticleDOI
TL;DR: It is proved that as the parameter characterizing this robustness becomes infinite, the two Nash equilibria become identical and equivalent to that of the risk-neutral case, as in the one-agent risk-sensitive and robust control theory.
Abstract: This paper considers two classes of large population stochastic differential games connected to optimal and robust decentralized control of large-scale multiagent systems. The first problem ( P1 ) is one where each agent minimizes an exponentiated cost function, capturing risk-sensitive behavior, whereas in the second problem ( P2 ) each agent minimizes a worst-case risk-neutral cost function, where the “worst case” stems from the presence of an adversary entering each agent’s dynamics characterized by a stochastic differential equation. In both problems, the individual agents are coupled through the mean field term included in each agent’s cost function, which captures the average or mass behavior of the agents. We solve both P1 and P2 via mean field game theory. Specifically, we first solve a generic risk-sensitive optimal control problem and a generic stochastic zero-sum differential game, where the corresponding optimal controllers are applied by each agent to construct the mean field systems of P1 and P2 . We then characterize an approximated mass behavior effect on an individual agent via a fixed-point analysis of the mean field system. For each problem, P1 and P2 , we show that the approximated mass behavior is in fact the best estimate of the actual mass behavior in various senses as the population size, $N$ , goes to infinity. Moreover, we show that for finite $N$ , there exist $\epsilon$ - Nash equilibria for both P1 and P2 , where the corresponding individual Nash strategies are decentralized in terms of local state information and the approximated mass behavior. We also show that $\epsilon$ can be taken to be arbitrarily small when $N$ is sufficiently large. We show that the $\epsilon$ - Nash equilibria of P1 and P2 are partially equivalent in the sense that the individual Nash strategies share identical control laws, but the approximated mass behaviors for P1 and P2 are different, since in P2 , the mass behavior is also affected by the associated worst-case disturbance. Finally, we prove that the Nash equilibria for P1 and P2 both feature robustness, and as the parameter characterizing this robustness becomes infinite, the two Nash equilibria become identical and equivalent to that of the risk-neutral case, as in the one-agent risk-sensitive and robust control theory.

Journal ArticleDOI
TL;DR: This paper proves in the sense of Lyapunov that, if the dwell time is larger than a positive threshold, the global leader-follower consensus for the closed-loop linear multiagent systems with input saturation under the derived topology containing a directed spanning tree can be achieved.
Abstract: This paper proposes a global cooperative control framework to address leader-follower consensus of constraints subsystems of industrial plants, in which each subsystem is modeled as an agent and all the subsystems and networks of information flow construct a multiagent system. The focus of this paper is to solve the global leader-follower consensus for multiagent systems with input saturation via low-high gain feedback approach and parametric algebraic Riccati equation approach, in which the feedback gain design is distributed and decoupled from network topologies. By introducing appropriate assumptions, a class of low-high gain feedback protocol is designed based on the states of local neighbors to reach the global stability. It is proved in the sense of Lyapunov that, if the dwell time is larger than a positive threshold, the global leader-follower consensus for the closed-loop linear multiagent systems with input saturation under the derived topology containing a directed spanning tree can be achieved. The results are further extended to leader-follower consensus for nonlinear multiagent systems with the design of nonlinear low-high gain feedback protocol. As industrial applications of the proposed low-high gain scheduling approaches, the controller design of vibration in mechanical systems and satellite formation systems are revisited. Numerical simulations with cooperative control of industries subsystems show the effectiveness of the proposed approach.