scispace - formally typeset
Search or ask a question

Showing papers in "arXiv: Systems and Control in 2018"


Journal ArticleDOI
TL;DR: The critical topics of DSSE, including mathematical problem formulation, application of pseudo-measurements, metering instrument placement, network topology issues, impacts of renewable penetration, and cyber-security are discussed.
Abstract: This paper presents a review of the literature on State Estimation (SE) in power systems While covering some works related to SE in transmission systems, the main focus of this paper is Distribution System State Estimation (DSSE) The paper discusses a few critical topics of DSSE, including mathematical problem formulation, application of pseudo-measurements, metering instrument placement, network topology issues, impacts of renewable penetration, and cyber-security Both conventional and modern data-driven and probabilistic techniques have been reviewed This paper can provide researchers and utility engineers with insights into the technical achievements, barriers, and future research directions of DSSE

246 citations


Posted Content
TL;DR: Verisig as discussed by the authors is a hybrid system approach to verify safety properties of closed-loop systems using neural networks as controllers, which can be solved using state-of-the-art reachability tools.
Abstract: This paper presents Verisig, a hybrid system approach to verifying safety properties of closed-loop systems using neural networks as controllers. Although techniques exist for verifying input/output properties of the neural network itself, these methods cannot be used to verify properties of the closed-loop system (since they work with piecewise-linear constraints that do not capture non-linear plant dynamics). To overcome this challenge, we focus on sigmoid-based networks and exploit the fact that the sigmoid is the solution to a quadratic differential equation, which allows us to transform the neural network into an equivalent hybrid system. By composing the network's hybrid system with the plant's, we transform the problem into a hybrid system verification problem which can be solved using state-of-the-art reachability tools. We show that reachability is decidable for networks with one hidden layer and decidable for general networks if Schanuel's conjecture is true. We evaluate the applicability and scalability of Verisig in two case studies, one from reinforcement learning and one in which the neural network is used to approximate a model predictive controller.

138 citations


Posted Content
TL;DR: In this paper, the authors proposed a methodology based on sensitivity analysis to assess the impact of P2P transactions on the network and to guarantee an exchange of energy that does not violate network constraints.
Abstract: The increasing uptake of distributed energy resources (DERs) in distribution systems and the rapid advance of technology have established new scenarios in the operation of low-voltage networks. In particular, recent trends in cryptocurrencies and blockchain have led to a proliferation of peer-to-peer (P2P) energy trading schemes, which allow the exchange of energy between the neighbors without any intervention of a conventional intermediary in the transactions. Nevertheless, far too little attention has been paid to the technical constraints of the network under this scenario. A major challenge to implementing P2P energy trading is that of ensuring that network constraints are not violated during the energy exchange. This paper proposes a methodology based on sensitivity analysis to assess the impact of P2P transactions on the network and to guarantee an exchange of energy that does not violate network constraints. The proposed method is tested on a typical UK low-voltage network. The results show that our method ensures that energy is exchanged between users under the P2P scheme without violating the network constraints, and that users can still capture the economic benefits of the P2P architecture.

132 citations


Posted Content
TL;DR: A control and planning architecture for CAVs is introduced, and the state of the art on each functional block therein is surveyed; the main focus is on techniques to improve energy efficiency.
Abstract: Autonomous driving technology pledges safety, convenience, and energy efficiency. Challenges include the unknown intentions of other road users: communication between vehicles and with the road infrastructure is a possible approach to enhance awareness and enable cooperation. Connected and automated vehicles (CAVs) have the potential to disrupt mobility, extending what is possible with driving automation and connectivity alone. Applications include real-time control and planning with increased awareness, routing with micro-scale traffic information, coordinated platooning using traffic signals information, eco-mobility on demand with guaranteed parking. This paper introduces a control and planning architecture for CAVs, and surveys the state of the art on each functional block therein; the main focus is on techniques to improve energy efficiency. We provide an overview of existing algorithms and their mutual interactions, we present promising optimization-based approaches to CAVs control and identify future challenges.

128 citations


Journal ArticleDOI
TL;DR: Guaranteed-cost consensus for high-order nonlinear multi-agent networks with switching topologies is investigated by constructing a time-varying nonsingular matrix with a specific structure and an approach to minimize the guaranteed cost is given in terms of linear matrix inequalities.
Abstract: Guaranteed-cost consensus for high-order nonlinear multi-agent networks with switching topologies is investigated. By constructing a time-varying nonsingular matrix with a specific structure, the whole dynamics of multi-agent networks is decomposed into the consensus and disagreement parts with nonlinear terms, which is the key challenge to be dealt with. An explicit expression of the consensus dynamics, which contains the nonlinear term, is given and its initial state is determined. Furthermore, by the structure property of the time-varying nonsingular transformation matrix and the Lipschitz condition, the impacts of the nonlinear term on the disagreement dynamics are linearized and the gain matrix of the consensus protocol is determined on the basis of the Riccati equation. Moreover, an approach to minimize the guaranteed cost is given in terms of linear matrix inequalities. Finally, the numerical simulation is shown to demonstrate the effectiveness of theoretical results.

98 citations


Posted Content
TL;DR: In this paper, the progress achieved by researchers worldwide regarding different aspects of Cooperative Adaptive Cruise Control (CACC) systems is reviewed, which explain how this system works from a higher level.
Abstract: Connected and automated vehicles (CAVs) have the potential to address the safety, mobility and sustainability issues of our current transportation systems. Cooperative adaptive cruise control (CACC), for example, is one promising technology to allow CAVs to be driven in a cooperative manner and introduces system-wide benefits. In this paper, we review the progress achieved by researchers worldwide regarding different aspects of CACC systems. Literature of CACC system architectures are reviewed, which explain how this system works from a higher level. Different control methodologies and their related issues are reviewed to introduce CACC systems from a lower level. Applications of CACC technology are demonstrated with detailed literature, which draw an overall landscape of CACC, point out current opportunities and challenges, and anticipate its development in the near future.

89 citations


Posted Content
TL;DR: In this article, a model predictive safety certification (MPSC) scheme for polytopic linear systems with additive disturbances is proposed, which verifies safety of a proposed learning-based input and modifies it as little as necessary to keep the system within a given set of constraints.
Abstract: While it has been repeatedly shown that learning-based controllers can provide superior performance, they often lack of safety guarantees. This paper aims at addressing this problem by introducing a model predictive safety certification (MPSC) scheme for polytopic linear systems with additive disturbances. The scheme verifies safety of a proposed learning-based input and modifies it as little as necessary in order to keep the system within a given set of constraints. Safety is thereby related to the existence of a model predictive controller (MPC) providing a feasible trajectory towards a safe target set. A robust MPC formulation accounts for the fact that the model is generally uncertain in the context of learning, which allows proving constraint satisfaction at all times under the proposed MPSC strategy. The MPSC scheme can be used in order to expand any potentially conservative set of safe states for learning and we prove an iterative technique for enlarging the safe set. Finally, a practical data-based design procedure for MPSC is proposed using scenario optimization.

74 citations


Posted Content
TL;DR: In this article, the authors presented a method to learn accurate safety certificates for nonlinear, closed-loop dynamical systems by constructing a neural network Lyapunov function and a training algorithm that adapts it to the shape of the largest safe region in the state space.
Abstract: Learning algorithms have shown considerable prowess in simulation by allowing robots to adapt to uncertain environments and improve their performance. However, such algorithms are rarely used in practice on safety-critical systems, since the learned policy typically does not yield any safety guarantees. That is, the required exploration may cause physical harm to the robot or its environment. In this paper, we present a method to learn accurate safety certificates for nonlinear, closed-loop dynamical systems. Specifically, we construct a neural network Lyapunov function and a training algorithm that adapts it to the shape of the largest safe region in the state space. The algorithm relies only on knowledge of inputs and outputs of the dynamics, rather than on any specific model structure. We demonstrate our method by learning the safe region of attraction for a simulated inverted pendulum. Furthermore, we discuss how our method can be used in safe learning algorithms together with statistical models of dynamical systems.

70 citations


Posted Content
TL;DR: This paper studies the scheduling of sensor transmissions to estimate the states of multiple remote, dynamic processes using a Deep Q-Network, a recent deep reinforcement learning algorithm that is at once scalable and model-free.
Abstract: In many Cyber-Physical Systems, we encounter the problem of remote state estimation of geographically distributed and remote physical processes This paper studies the scheduling of sensor transmissions to estimate the states of multiple remote, dynamic processes Information from the different sensors have to be transmitted to a central gateway over a wireless network for monitoring purposes, where typically fewer wireless channels are available than there are processes to be monitored For effective estimation at the gateway, the sensors need to be scheduled appropriately, ie, at each time instant one needs to decide which sensors have network access and which ones do not To address this scheduling problem, we formulate an associated Markov decision process (MDP) This MDP is then solved using a Deep Q-Network, a recent deep reinforcement learning algorithm that is at once scalable and model-free We compare our scheduling algorithm to popular scheduling algorithms such as round-robin and reduced-waiting-time, among others Our algorithm is shown to significantly outperform these algorithms for many example scenarios

63 citations


Posted Content
TL;DR: In this paper, the authors derived finite time error bounds for estimating general linear time-invariant (LTI) systems from a single observed trajectory using the method of least squares, and provided the first analysis of the general case when eigenvalues of the LTI system are arbitrarily distributed in three regimes: stable, marginally stable and explosive.
Abstract: We derive finite time error bounds for estimating general linear time-invariant (LTI) systems from a single observed trajectory using the method of least squares. We provide the first analysis of the general case when eigenvalues of the LTI system are arbitrarily distributed in three regimes: stable, marginally stable, and explosive. Our analysis yields sharp upper bounds for each of these cases separately. We observe that although the underlying process behaves quite differently in each of these three regimes, the systematic analysis of a self--normalized martingale difference term helps bound identification error up to logarithmic factors of the lower bound. On the other hand, we demonstrate that the least squares solution may be statistically inconsistent under certain conditions even when the signal-to-noise ratio is high.

62 citations


Journal ArticleDOI
TL;DR: A linear dynamic time-invariant model is identified to describe the relationship between the reference signal and the output of the system and the power spectrum of the unmodeled disturbances are identified to generate uncertainty bounds on the estimated model.
Abstract: This article addresses the following problems: 1) First, a nonlinearity analysis is made looking for the presence of nonlinearities in an early phase of the identification process. The level and the nature of the nonlinearities should be retrieved without a significant increase in the amount of measured data. 2) Next it is studied if it is safe to use a linear system identification approach, even if the presence of nonlinear distortions is detected. The properties of the linear system identification approach under these conditions are studied, and the reliability of the uncertainty bounds is checked. 3) Eventually, tools are provided to check how much can be gained if a nonlinear model were identified instead of a linear model. Addressing these three questions forms the outline of this article. The possibilities and pitfalls of using a linear identification framework in the presence of nonlinear distortions will be discussed and illustrated on lab-scale and industrial examples. In this article, the focus is on nonparametric and parametric black box identification methods, however the results might also be useful for physical modeling methods. Knowing the actual nonlinear distortion level can help to choose the required level of detail that is needed in the physical model. This will strongly influence the modeling effort. Also, in this case, significant time can be saved if it is known from experiments that the system behaves almost linearly. The converse is also true. If the experiments show that some (sub-)systems are highly nonlinear, it helps to focus the physical modeling effort on these critical elements.

Posted Content
TL;DR: The Deep Centralized Multi-agent Actor Critic (DCMAC) is developed, an off-policy actor-critic DRL approach, providing efficient life-cycle policies for large multi-component systems operating in high-dimensional spaces.
Abstract: Decision-making for engineering systems can be efficiently formulated as a Markov Decision Process (MDP) or a Partially Observable MDP (POMDP). Typical MDP and POMDP solution procedures utilize offline knowledge about the environment and provide detailed policies for relatively small systems with tractable state and action spaces. However, in large multi-component systems the sizes of these spaces easily explode, as system states and actions scale exponentially with the number of components, whereas environment dynamics are difficult to be described in explicit forms for the entire system and may only be accessible through numerical simulators. In this work, to address these issues, an integrated Deep Reinforcement Learning (DRL) framework is introduced. The Deep Centralized Multi-agent Actor Critic (DCMAC) is developed, an off-policy actor-critic DRL approach, providing efficient life-cycle policies for large multi-component systems operating in high-dimensional spaces. Apart from deep function approximations that parametrize large state spaces, DCMAC also adopts a factorized representation of the system actions, being able to designate individualized component- and subsystem-level decisions, while maintaining a centralized value function for the entire system. DCMAC compares well against Deep Q-Network (DQN) solutions and exact policies, where applicable, and outperforms optimized baselines that are based on time-based, condition-based and periodic policies.

Posted Content
TL;DR: This paper proposes a faulted line localization method based on a convolutional neural network (CNN) classifier using bus voltages, based on features with physical interpretations that improve the robustness of the location performance.
Abstract: Diverse fault types, fast re-closures, and complicated transient states after a fault event make real-time fault location in power grids challenging. Existing localization techniques in this area rely on simplistic assumptions, such as static loads, or require much higher sampling rates or total measurement availability. This paper proposes a faulted line localization method based on a Convolutional Neural Network (CNN) classifier using bus voltages. Unlike prior data-driven methods, the proposed classifier is based on features with physical interpretations that improve the robustness of the location performance. The accuracy of our CNN based localization tool is demonstrably superior to other machine learning classifiers in the literature. To further improve the location performance, a joint phasor measurement units (PMU) placement strategy is proposed and validated against other methods. A significant aspect of our methodology is that under very low observability (7% of buses), the algorithm is still able to localize the faulted line to a small neighborhood with high probability. The performance of our scheme is validated through simulations of faults of various types in the IEEE 39-bus and 68-bus power systems under varying uncertain conditions, system observability, and measurement quality.

Posted Content
TL;DR: In this article, a distributed guaranteed-cost synchronization problem for general high-order linear multiagent networks is addressed, which can realize the tradeoff design between the energy consumption and the synchronization regulation performance with the given cost budget.
Abstract: The current paper addresses the distributed guaranteed-cost synchronization problems for general high-order linear multiagent networks. Existing works on the guaranteed-cost synchronization usually require all state information of neighboring agents and cannot give the cost budget previously. For both leaderless and leader-following interaction topologies, the current paper firstly proposes a dynamic output feedback synchronization protocol with guaranteed-cost constraints, which can realize the tradeoff design between the energy consumption and the synchronization regulation performance with the given cost budget. Then, according to different structure features of interaction topologies, leaderless and leader-following guaranteed-cost synchronization analysis and design criteria are presented, respectively, and an algorithm is proposed to deal with the impacts of nonlinear terms by using both synchronization analysis and design criteria. Especially, an explicit expression of the synchronization function is shown for leaderless cases, which is independent of protocol states and the given cost budget. Finally, numerical examples are presented to demonstrate theoretical results.

Posted Content
TL;DR: In this article, an adversarial deep reinforcement learning (RL) algorithm is proposed to maximize the robustness of AV dynamics control to cyber-physical (CP) attacks by adversaries that seek to take control of the AVs by manipulating their data.
Abstract: To operate effectively in tomorrow's smart cities, autonomous vehicles (AVs) must rely on intra-vehicle sensors such as camera and radar as well as inter-vehicle communication. Such dependence on sensors and communication links exposes AVs to cyber-physical (CP) attacks by adversaries that seek to take control of the AVs by manipulating their data. Thus, to ensure safe and optimal AV dynamics control, the data processing functions at AVs must be robust to such CP attacks. To this end, in this paper, the state estimation process for monitoring AV dynamics, in presence of CP attacks, is analyzed and a novel adversarial deep reinforcement learning (RL) algorithm is proposed to maximize the robustness of AV dynamics control to CP attacks. The attacker's action and the AV's reaction to CP attacks are studied in a game-theoretic framework. In the formulated game, the attacker seeks to inject faulty data to AV sensor readings so as to manipulate the inter-vehicle optimal safe spacing and potentially increase the risk of AV accidents or reduce the vehicle flow on the roads. Meanwhile, the AV, acting as a defender, seeks to minimize the deviations of spacing so as to ensure robustness to the attacker's actions. Since the AV has no information about the attacker's action and due to the infinite possibilities for data value manipulations, the outcome of the players' past interactions are fed to long-short term memory (LSTM) blocks. Each player's LSTM block learns the expected spacing deviation resulting from its own action and feeds it to its RL algorithm. Then, the the attacker's RL algorithm chooses the action which maximizes the spacing deviation, while the AV's RL algorithm tries to find the optimal action that minimizes such deviation.

Proceedings ArticleDOI
TL;DR: A stochastic model predictive control algorithm for linear discrete-time systems affected by possibly unbounded additive disturbances and subject to probabilistic constraints is proposed, for which an asymptotic average performance bound is derived.
Abstract: In this paper we propose a stochastic model predictive control (MPC) algorithm for linear discrete-time systems affected by possibly unbounded additive disturbances and subject to probabilistic constraints. Constraints are treated in analogy to robust MPC using a constraint tightening based on the concept of probabilistic reachable sets, which is shown to provide closed-loop fulfillment of chance constraints under a unimodality assumption on the disturbance distribution. A control scheme reverting to a backup solution from a previous time step in case of infeasibility is proposed, for which an asymptotic average performance bound is derived. Two examples illustrate the approach, highlighting closed-loop chance constraint satisfaction and the benefits of the proposed controller in the presence of unmodeled disturbances.

Posted Content
TL;DR: Methods for estimating the reachable set and verifying safety properties of dynamical systems under control of neural network-based controllers that may be implemented in embedded software are developed.
Abstract: Autonomous cyber-physical systems (CPS) rely on the correct operation of numerous components, with state-of-the-art methods relying on machine learning (ML) and artificial intelligence (AI) components in various stages of sensing and control. This paper develops methods for estimating the reachable set and verifying safety properties of dynamical systems under control of neural network-based controllers that may be implemented in embedded software. The neural network controllers we consider are feedforward neural networks called multilayer perceptrons (MLP) with general activation functions. As such feedforward networks are memoryless, they may be abstractly represented as mathematical functions, and the reachability analysis of the network amounts to range (image) estimation of this function provided a set of inputs. By discretizing the input set of the MLP into a finite number of hyper-rectangular cells, our approach develops a linear programming (LP) based algorithm for over-approximating the output set of the MLP with its input set as a union of hyper-rectangular cells. Combining the over-approximation for the output set of an MLP based controller and reachable set computation routines for ordinary difference/differential equation (ODE) models, an algorithm is developed to estimate the reachable set of the closed-loop system. Finally, safety verification for neural network control systems can be performed by checking the existence of intersections between the estimated reachable set and unsafe regions. The approach is implemented in a computational software prototype and evaluated on numerical examples.

Posted Content
TL;DR: By regulating the partial gradients of policies, strong guarantees of robust stability can be obtained based on a proposed semidefinite programming feasibility problem and the method is able to certify a large set of stabilizing controllers by exploiting problem-specific structures.
Abstract: We investigate the important problem of certifying stability of reinforcement learning policies when interconnected with nonlinear dynamical systems. We show that by regulating the input-output gradients of policies, strong guarantees of robust stability can be obtained based on a proposed semidefinite programming feasibility problem. The method is able to certify a large set of stabilizing controllers by exploiting problem-specific structures; furthermore, we analyze and establish its (non)conservatism. Empirical evaluations on two decentralized control tasks, namely multi-flight formation and power system frequency regulation, demonstrate that the reinforcement learning agents can have high performance within the stability-certified parameter space, and also exhibit stable learning behaviors in the long run.

Journal ArticleDOI
TL;DR: In this article, a greedy algorithm is developed to solve the problem of identifying the most vulnerable phasor measurement units in the network and a computationally efficient alternating minimization algorithm for joint state estimation and attack reconstruction is presented.
Abstract: Sensors such as phasor measurement units (PMUs) endowed with GPS receivers are ubiquitously installed providing real-time grid visibility. A number of PMUs can cooperatively enable state estimation routines. However, GPS spoofing attacks can notably alter the PMU measurements, mislead the network operator, and drastically impact subsequent corrective control actions. Leveraging a novel measurement model that explicitly accounts for the GPS spoofing attacks, this paper formulates an optimization problem to identify the most vulnerable PMUs in the network. A greedy algorithm is developed to solve the aforementioned problem. Furthermore, the paper develops a computationally efficient alternating minimization algorithm for joint state estimation and attack reconstruction. Numerical tests on IEEE benchmark networks validate the developed methods.

Journal ArticleDOI
TL;DR: This approach is able to synthesize relatively simple polynomial control Lyapunov-like functions, and in that process replace the MPC using a guaranteed and computationally less expensive controller.
Abstract: We present a technique for learning control Lyapunov-like functions, which are used in turn to synthesize controllers for nonlinear dynamical systems that can stabilize the system, or satisfy specifications such as remaining inside a safe set, or eventually reaching a target set while remaining inside a safe set. The learning framework uses a demonstrator that implements a black-box, untrusted strategy presumed to solve the problem of interest, a learner that poses finitely many queries to the demonstrator to infer a candidate function, and a verifier that checks whether the current candidate is a valid control Lyapunov function. The overall learning framework is iterative, eliminating a set of candidates on each iteration using the counterexamples discovered by the verifier and the demonstrations over these counterexamples. We prove its convergence using ellipsoidal approximation techniques from convex optimization. We also implement this scheme using nonlinear MPC controllers to serve as demonstrators for a set of state and trajectory stabilization problems for nonlinear dynamical systems. We show how the verifier can be constructed efficiently using convex relaxations of the verification problem for polynomial systems to semi-definite programming (SDP) problem instances. Our approach is able to synthesize relatively simple polynomial control Lyapunov functions, and in that process replace the MPC using a guaranteed and computationally less expensive controller.

Posted Content
TL;DR: The results show that the coordination between AMoD fleets and public transit can yield significant benefits compared to an AMiD system operating in isolation.
Abstract: In this paper we study models and coordination policies for intermodal Autonomous Mobility-on-Demand (AMoD), wherein a fleet of self-driving vehicles provides on-demand mobility jointly with public transit. Specifically, we first present a network flow model for intermodal AMoD, where we capture the coupling between AMoD and public transit and the goal is to maximize social welfare. Second, leveraging such a model, we design a pricing and tolling scheme that allows to achieve the social optimum under the assumption of a perfect market with selfish agents. Finally, we present a real-world case study for New York City. Our results show that the coordination between AMoD fleets and public transit can yield significant benefits compared to an AMoD system operating in isolation.

Journal ArticleDOI
TL;DR: Wang et al. as discussed by the authors developed a novel data-driven robust model predictive control (DDRMPC) approach for automatic control of irrigation systems, which integrates mechanistic models, which describe dynamics in soil moisture variations, and data driven models which characterize uncertainty in forecast errors of evapotranspiration and precipitation, into a holistic systems control framework.
Abstract: We develop a novel data-driven robust model predictive control (DDRMPC) approach for automatic control of irrigation systems. The fundamental idea is to integrate both mechanistic models, which describe dynamics in soil moisture variations, and data-driven models, which characterize uncertainty in forecast errors of evapotranspiration and precipitation, into a holistic systems control framework. To better capture the support of uncertainty distribution, we take a new learning-based approach by constructing uncertainty sets from historical data. For evapotranspiration forecast error, the support vector clustering-based uncertainty set is adopted, which can be conveniently built from historical data. As for precipitation forecast errors, we analyze the dependence of their distribution on forecast values, and further design a tailored uncertainty set based on the properties of this type of uncertainty. In this way, the overall uncertainty distribution can be elaborately described, which finally contributes to rational and efficient control decisions. To assure the quality of data-driven uncertainty sets, a training-calibration scheme is used to provide theoretical performance guarantees. A generalized affine decision rule is adopted to obtain tractable approximations of optimal control problems, thereby ensuring the practicability of DDRMPC. Case studies using real data show that, DDRMPC can reliably maintain soil moisture above the safety level and avoid crop devastation. The proposed DDRMPC approach leads to a 40% reduction of total water consumption compared to the fine-tuned open-loop control strategy. In comparison with the carefully tuned rule-based control and certainty equivalent model predictive control, the proposed DDRMPC approach can significantly reduce the total water consumption and improve the control performance.

Journal ArticleDOI
TL;DR: In this article, a detection algorithm for sensor attacks and a resilient state estimation scheme for a class of uniformly observable nonlinear systems is presented, where an adversary is supposed to corrupt a subset of sensors with the possibly unbounded signals, while the system has sensor redundancy.
Abstract: This paper presents a detection algorithm for sensor attacks and a resilient state estimation scheme for a class of uniformly observable nonlinear systems. An adversary is supposed to corrupt a subset of sensors with the possibly unbounded signals, while the system has sensor redundancy. We design an individual high-gain observer for each measurement output so that only the observable portion of the system state is obtained. Then, a nonlinear error correcting problem is solved by collecting all the information from those partial observers and exploiting redundancy. A computationally efficient, on-line monitoring scheme is presented for attack detection. Based on the attack detection scheme, an algorithm for resilient state estimation is provided. The simulation results demonstrate the effectiveness of the proposed algorithm.

Book ChapterDOI
TL;DR: The class of MDP models previously coined linearly solvable MDPs, that describe optimal dynamics of the probability distribution of an ensemble of many cycling devices, are utilized and developed.
Abstract: A Markov decision process (MDP) framework is adopted to represent ensemble control of devices with cyclic energy consumption patterns, e.g., thermostatically controlled loads. Specifically we utilize and develop the class of MDP models previously coined linearly solvable MDPs, that describe optimal dynamics of the probability distribution of an ensemble of many cycling devices. Two principally different settings are discussed. First, we consider optimal strategy of the ensemble aggregator balancing between minimization of the cost of operations and minimization of the ensemble welfare penalty, where the latter is represented as a KL-divergence between actual and normal probability distributions of the ensemble. Then, second, we shift to the demand response setting modeling the aggregator’s task to minimize the welfare penalty under the condition that the aggregated consumption matches the targeted time-varying consumption requested by the system operator. We discuss a modification of both settings aimed at encouraging or constraining the transitions between different states. The dynamic programming feature of the resulting modified MDPs is always preserved; however, “linear solvability” is lost fully or partially, depending on the type of modification. We also conducted some (limited in scope) numerical experimentation using the formulations of the first setting. We conclude by discussing future generalizations and applications.

Posted Content
TL;DR: The estimated output reachable set can be estimated iteratively for a given finite-time interval and the safety verification for piecewise linear systems with neural network controllers can be performed by checking the existence of intersections of unsafe regions and output reach set.
Abstract: In this work, the reachable set estimation and safety verification problems for a class of piecewise linear systems equipped with neural network controllers are addressed. The neural network is considered to consist of Rectified Linear Unit (ReLU) activation functions. A layer-by-layer approach is developed for the output reachable set computation of ReLU neural networks. The computation is formulated in the form of a set of manipulations for a union of polytopes. Based on the output reachable set for neural network controllers, the output reachable set for a piecewise linear feedback control system can be estimated iteratively for a given finite-time interval. With the estimated output reachable set, the safety verification for piecewise linear systems with neural network controllers can be performed by checking the existence of intersections of unsafe regions and output reach set. A numerical example is presented to illustrate the effectiveness of our approach.

Journal ArticleDOI
TL;DR: In this article, the authors investigate the influence of the network topology on collective response in a leader-follower linear consensus model of distributed decision-making and study the collective capacity of the system to follow a dynamic driving signal (the leader) for a range of topologies and system sizes.
Abstract: Natural, social, and artificial multi-agent systems usually operate in dynamic environments, where the ability to respond to changing circumstances is a crucial feature. An effective collective response requires suitable information transfer among agents, and thus is critically dependent on the agents' interaction network. In order to investigate the influence of the network topology on collective response, we consider an archetypal model of distributed decision-making---the leader-follower linear consensus---and study the collective capacity of the system to follow a dynamic driving signal (the "leader") for a range of topologies and system sizes. The analysis reveals a nontrivial relationship between optimal topology and frequency of the driving signal. Interestingly, the response is optimal when each individual interacts with a certain number of agents which decreases monotonically with the frequency and, for large enough systems, is independent of the size of the system. This phenomenology is investigated in experiments of collective motion using a swarm of land robots. The emergent collective response to both a slow- and a fast-changing leader is measured and analyzed for a range of interaction topologies. These results have far-reaching practical implications for the design and understanding of distributed systems, since they highlight that a dynamic rewiring of the interaction network is paramount to the effective collective operations of multi-agent systems at different time-scales.

Posted Content
TL;DR: A state-of-the art, fully operational test-bed showing the potential for merged IoT, 5G, and cloud, and an evaluation of the system while operating a mission-critical application and an outlook on a novel research direction are provided.
Abstract: With the emergence of industrial IoT and cloud computing, and the advent of 5G and edge clouds, there are ambitious expectations on elasticity, economies of scale, and fast time to market for demanding use cases in the next generation of ICT networks. Responsiveness and reliability of wireless communication links and services in the cloud are set to improve significantly as the concept of edge clouds is becoming more prevalent. To enable industrial uptake we must provide cloud capacity in the networks but also a sufficient level of simplicity and self-sustainability in the software platforms. In this paper, we present a research test-bed built to study mission-critical control over the distributed edge cloud. We evaluate system properties using a conventional control application in the form of a Model Predictive Controller. Our cloud platform provides the means to continuously operate our mission-critical application while seamlessly relocating computations across geographically dispersed compute nodes. Through our use of 5G wireless radio, we allow for mobility and reliably provide compute resources with low latency, at the edge. The primary contribution of this paper is a state-of-the art, fully operational test-bed showing the potential for merged IoT, 5G, and cloud. We also provide an evaluation of the system while operating a mission-critical application and provide an outlook on a novel research direction.

Journal ArticleDOI
TL;DR: In this paper, the authors investigate a remote estimation problem in which a transmitter observes a Markov source and chooses the power level to transmit it over a time-varying packet-drop channel.
Abstract: We investigate a remote estimation problem in which a transmitter observes a Markov source and chooses the power level to transmit it over a time-varying packet-drop channel. The channel is modeled as a channel with Markovian state where the packet drop probability depends on the channel state and the transmit power. A receiver observes the channel output and the channel state and estimates the source realization. The receiver also feeds back the channel state and an acknowledgment for successful reception to the transmitter. We consider two models for the source---finite state Markov chains and first-order autoregressive processes. For the first model, using ideas from team theory, we establish the structure of optimal transmission and estimation strategies and identify a dynamic program to determine optimal strategies with that structure. For the second model, we assume that the noise process has unimodal and symmetric distribution. Using ideas from majorization theory, we show that the optimal transmission strategy is symmetric and monotonic and the optimal estimation strategy is like Kalman filter. Consequently, when there are a finite number of power levels, the optimal transmission strategy may be described using thresholds that depend on the channel state. Finally, we propose a simulation based approach (Renewal Monte Carlo) to compute the optimal thresholds and optimal performance and elucidate the algorithm with an example.

Posted Content
TL;DR: A simulation-based approach for generating barrier certificate functions for safety verification of cyber-physical systems (CPS) that contain neural network-based controllers that are controlled by a neural network to follow a given path is presented.
Abstract: We present a simulation-based approach for generating barrier certificate functions for safety verification of cyber-physical systems (CPS) that contain neural network-based controllers. A linear programming solver is utilized to find a candidate generator function from a set of simulation traces obtained by randomly selecting initial states for the CPS model. A level set of the generator function is then selected to act as a barrier certificate for the system, meaning it demonstrates that no unsafe system states are reachable from a given set of initial states. The barrier certificate properties are verified with an SMT solver. This approach is demonstrated on a case study in which a Dubins car model of an autonomous vehicle is controlled by a neural network to follow a given path.

Posted Content
TL;DR: This article proposes a computationally efficient algorithm that is provably robust to Byzantine agents, and establishes that good agents learn the true parameter asymptotically in almost sure sense.
Abstract: This work considers resilient, cooperative state estimation in unreliable multi-agent networks. A network of agents aims to collaboratively estimate the value of an unknown vector parameter, while an {\em unknown} subset of agents suffer Byzantine faults. Faulty agents malfunction arbitrarily and may send out {\em highly unstructured} messages to other agents in the network. As opposed to fault-free networks, reaching agreement in the presence of Byzantine faults is far from trivial. In this paper, we propose a computationally-efficient algorithm that is provably robust to Byzantine faults. At each iteration of the algorithm, a good agent (1) performs a gradient descent update based on noisy local measurements, (2) exchanges its update with other agents in its neighborhood, and (3) robustly aggregates the received messages using coordinate-wise trimmed means. Under mild technical assumptions, we establish that good agents learn the true parameter asymptotically in almost sure sense. We further complement our analysis by proving (high probability) {\em finite-time} convergence rate, encapsulating network characteristics.