Showing papers on "Discrete time and continuous time published in 2017"

Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations

Abstract: We introduce physics informed neural networks -- neural networks that are trained to solve supervised learning tasks while respecting any given law of physics described by general nonlinear partial differential equations. In this second part of our two-part treatise, we focus on the problem of data-driven discovery of partial differential equations. Depending on whether the available data is scattered in space-time or arranged in fixed temporal snapshots, we introduce two main classes of algorithms, namely continuous time and discrete time models. The effectiveness of our approach is demonstrated using a wide range of benchmark problems in mathematical physics, including conservation laws, incompressible fluid flow, and the propagation of nonlinear shallow-water waves.

Data-Driven Optimal Consensus Control for Discrete-Time Multi-Agent Systems With Unknown Dynamics Using Reinforcement Learning Method

Abstract: This paper investigates the optimal consensus control problem for discrete-time multi-agent systems with completely unknown dynamics by utilizing a data-driven reinforcement learning method. It is known that the optimal consensus control for multi-agent systems relies on the solution of the coupled Hamilton–Jacobi–Bellman equation, which is generally impossible to be solved analytically. Even worse, most real-world systems are too complicated to obtain accurate mathematical models. To overcome these deficiencies, a data-based adaptive dynamic programming method is presented using the current and past system data rather than the accurate system models also instead of the traditional identification scheme which would cause the approximation residual errors. First, we establish a discounted performance index and formulate the optimal consensus problem via Bellman optimality principle. Then, we introduce the policy iteration algorithm which motivates this paper. To implement the proposed online action-dependent heuristic dynamic programming method, two neural networks (NNs), 1) critic NN and 2) actor NN, are employed to approximate the iterative performance index functions and control policies, respectively, in real time. Finally, two simulation examples are provided to demonstrate the effectiveness of the proposed method.

Statistical clustering of temporal networks through a dynamic stochastic block model

Abstract: Statistical node clustering in discrete time dynamic networks is an emerging field that raises many challenges. Here, we explore statistical properties and frequentist inference in a model that combines a stochastic block model (SBM) for its static part with independent Markov chains for the evolution of the nodes groups through time. We model binary data as well as weighted dynamic random graphs (with discrete or continuous edges values). Our approach, motivated by the importance of controlling for label switching issues across the different time steps, focuses on detecting groups characterized by a stable within group connectivity behavior. We study identifiability of the model parameters , propose an inference procedure based on a variational expectation maximization algorithm as well as a model selection criterion to select for the number of groups. We carefully discuss our initialization strategy which plays an important role in the method and compare our procedure with existing ones on synthetic datasets. We also illustrate our approach on dynamic contact networks, one of encounters among high school students and two others on animal interactions. An implementation of the method is available as a R package called dynsbm.

Event-Triggered State Estimation for Discrete-Time Multidelayed Neural Networks With Stochastic Parameters and Incomplete Measurements

Abstract: In this paper, the event-triggered state estimation problem is investigated for a class of discrete-time multidelayed neural networks with stochastic parameters and incomplete measurements. In order to cater for more realistic transmission process of the neural signals, we make the first attempt to introduce a set of stochastic variables to characterize the random fluctuations of system parameters. In the addressed neural network model, the delays among the interconnections are allowed to be different, which are more general than those in the existing literature. The incomplete information under consideration includes randomly occurring sensor saturations and quantizations. For the purpose of energy saving, an event-triggered state estimator is constructed and a sufficient condition is given under which the estimation error dynamics is exponentially ultimately bounded in the mean square. It is worth noting that the ultimate boundedness of the error dynamics is explicitly estimated. The characterization of the desired estimator gain is designed in terms of the solution to a certain matrix inequality. Finally, a numerical simulation example is presented to illustrate the effectiveness of the proposed event-triggered state estimation scheme.

State Estimation for Discrete-Time Dynamical Networks With Time-Varying Delays and Stochastic Disturbances Under the Round-Robin Protocol

Abstract: This paper is concerned with the state estimation problem for a class of nonlinear dynamical networks with time-varying delays subject to the round-robin protocol. The communication between the state estimator and the nodes of the dynamical networks is implemented through a shared constrained network, in which only one node is allowed to send data at each time instant. The round-robin protocol is utilized to orchestrate the transmission order of nodes. By using a switch-based approach, the dynamics of the estimation error is modeled by a periodic parameter-switching system with time-varying delays. The purpose of the problem addressed is to design an estimator, such that the estimation error is exponentially ultimately bounded with a certain asymptotic upper bound in mean square subject to the process noise and exogenous disturbance. Furthermore, such a bound is subsequently minimized by the designed estimator parameters. A novel Lyapunov-like functional is employed to deal with the dynamics analysis issue of the estimation error. Sufficient conditions are established to guarantee the ultimate boundedness of the estimation error in mean square by applying the stochastic analysis approach. Then, the desired estimator gains are characterized by solving a convex problem. Finally, a numerical example is given to illustrate the effectiveness of the estimator design scheme.

Adaptive Fault-Tolerant Tracking Control for MIMO Discrete-Time Systems via Reinforcement Learning Algorithm With Less Learning Parameters

Abstract: This paper is concerned with a reinforcement learning-based adaptive tracking control technique to tolerate faults for a class of unknown multiple-input multiple-output nonlinear discrete-time systems with less learning parameters. Not only abrupt faults are considered, but also incipient faults are taken into account. Based on the approximation ability of neural networks, action network and critic network are proposed to approximate the optimal signal and to generate the novel cost function, respectively. The remarkable feature of the proposed method is that it can reduce the cost in the procedure of tolerating fault and can decrease the number of learning parameters and thus reduce the computational burden. Stability analysis is given to ensure the uniform boundedness of adaptive control signals and tracking errors. Finally, three simulations are used to show the effectiveness of the present strategy.

Continuous time structural equation modeling with R package ctsem

Abstract: We introduce ctsem, an R package for continuous time structural equation modeling of panel (N > 1) and time series (N = 1) data, using full information maximum likelihood. Most dynamic models (e.g., cross-lagged panel models) in the social and behavioural sciences are discrete time models. An assumption of discrete time models is that time intervals between measurements are equal, and that all subjects were assessed at the same intervals. Violations of this assumption are often ignored due to the difficulty of accounting for varying time intervals, therefore parameter estimates can be biased and the time course of effects becomes ambiguous. By using stochastic differential equations to estimate an underlying continuous process, continuous time models allow for any pattern of measurement occasions. By interfacing to OpenMx, ctsem combines the flexible specification of structural equation models with the enhanced data gathering opportunities and improved estimation of continuous time models. ctsem can estimate relationships over time for multiple latent processes, measured by multiple noisy indicators with varying time intervals between observations. Within and between effects are estimated simultaneously by modeling both observed covariates and unobserved heterogeneity. Exogenous shocks with different shapes, group differences, higher order diffusion effects and oscillating processes can all be simply modeled. We first introduce and define continuous time models, then show how to specify and estimate a range of continuous time models using ctsem.

Stochastic Actor-Oriented Models for Network Dynamics

Abstract: This article discusses the stochastic actor-oriented model for analyzing panel data of networks. The model is defined as a continuous-time Markov chain, observed at two or more discrete time moments. It can be regarded as a generalized linear model with a large amount of missing data. Several estimation methods are discussed. After presenting the model for evolution of networks, attention is given to coevolution models. These use the same approach of a continuous-time Markov chain observed at a small number of time points, but now with an extended state space. The state space can be, for example, the combination of a network and nodal variables, or a combination of several networks. This leads to models for the dynamics of multivariate networks. The article emphasizes the approach to modeling and algorithmic issues for estimation; some attention is given to comparison with other models.

Adaptive Event-Triggered Control Based on Heuristic Dynamic Programming for Nonlinear Discrete-Time Systems

Abstract: This paper presents the design of a novel adaptive event-triggered control method based on the heuristic dynamic programming (HDP) technique for nonlinear discrete-time systems with unknown system dynamics. In the proposed method, the control law is only updated when the event-triggered condition is violated. Compared with the periodic updates in the traditional adaptive dynamic programming (ADP) control, the proposed method can reduce the computation and transmission cost. An actor–critic framework is used to learn the optimal event-triggered control law and the value function. Furthermore, a model network is designed to estimate the system state vector. The main contribution of this paper is to design a new trigger threshold for discrete-time systems. A detailed Lyapunov stability analysis shows that our proposed event-triggered controller can asymptotically stabilize the discrete-time systems. Finally, we test our method on two different discrete-time systems, and the simulation results are included.

Sliding Mode Control of Discrete-Time Switched Systems with Repeated Scalar Nonlinearities

Abstract: This note studies the design of sliding mode control (SMC) for discrete-time hybrid stochastic switched systems with repeated scalar nonlinearities. The weighed $\mathcal{H}_{\infty}$ gain performance is considered for the system dynamics to optimize its transient state performance. First, sufficient conditions are given to guarantee the corresponding system is exponentially stable while achieving a desired weighed $\mathcal{H}_{\infty}$ performance. A new switching surface function is constructed by the average dwell time technique and the positive diagonally dominant Lyapunov functional method to further reduce the conservativeness induced by the repeated scalar nonlinearity. Then, the corresponding sliding mode dynamics are obtained and the solvability condition for the desired switching surface function is derived. Furthermore, the synthesis of the proposed SMC law is proposed to force the resulting closed-loop system trajectories onto the pre-specified sliding mode region with a desired level of accuracy. Finally, the feasibility and the effectiveness of the presented new design techniques are illustrated by examples and simulations.

Distributed Velocity-Constrained Consensus of Discrete-Time Multi-Agent Systems With Nonconvex Constraints, Switching Topologies, and Delays

Abstract: In this technical note, a distributed velocity-constrained consensus problem is studied for discrete-time multi-agent systems, where each agent's velocity is constrained to lie in a nonconvex set. A distributed constrained control algorithm is proposed to enable all agents to converge to a common point using only local information. The gains of the algorithm for all agents need not to be the same or predesigned and can be adjusted by each agent itself based on its own and neighbors' information. It is shown that the algorithm is robust to arbitrarily bounded communication delays and arbitrarily switching communication graphs provided that the union of the graphs has directed spanning trees among each certain time interval. The analysis approach is based on multiple novel model transformations, proper control parameter selections, boundedness analysis of state-dependent stochastic matrices1, exploitation of the convexity of stochastic matrices, and the joint connectivity of the communication graphs. Numerical examples are included to illustrate the theoretical results.

Periodic Event-Triggered Synchronization of Linear Multi-Agent Systems With Communication Delays

Abstract: Multi-agent systems' cooperation to achieve global goals is usually limited by sensing, actuation, and communication issues. At the local level, continuous measurement and actuation is only approximated by the use of digital mechanisms that measure and process information in order to compute and update new control input values at discrete time instants. Interaction with other agents takes place, in general, through a digital communication channel with limited bandwidth where transmission of continuous-time signals is not possible. This technical note considers the problem of consensus (or synchronization of state trajectories) of multi-agent systems that are described by general linear dynamics and are connected using undirected graphs. The proposed event-triggered consensus protocol not only avoids the need for continuous communication between agents but also provides a decentralized method for transmission of information in the presence of time-varying communication delays, where each agent decides its own broadcasting time instants based only on local information. This method gives more flexibility for scheduling information broadcasting compared to periodic and sampled-data implementations.

Event‐triggered optimal tracking control of nonlinear systems

Abstract: Summary We propose a novel event-triggered optimal tracking control algorithm for nonlinear systems with an infinite horizon discounted cost. The problem is formulated by appropriately augmenting the system and the reference dynamics and then using ideas from reinforcement learning to provide a solution. Namely, a critic network is used to estimate the optimal cost while an actor network is used to approximate the optimal event-triggered controller. Because the actor network updates only when an event occurs, we shall use a zero-order hold along with appropriate tuning laws to encounter for this behavior. Because we have dynamics that evolve in continuous and discrete time, we write the closed-loop system as an impulsive model and prove asymptotic stability of the equilibrium point and Zeno behavior exclusion. Simulation results of a helicopter, a one-link rigid robot under gravitation field, and a controlled Van-der-Pol oscillator are presented to show the efficacy of the proposed approach. Copyright © 2016 John Wiley & Sons, Ltd.

Filtering for Discrete-Time Switched Fuzzy Systems With Quantization

Abstract: The paper is concerned with the $H_\infty$ and $l_2$ – $l_\infty$ filtering design problems for discrete-time nonlinear switched systems with quantized measurements using the Takagi–Sugeno (T–S) fuzzy model. The systems under consideration inherently combine features of the switched hybrid systems and the T–S fuzzy systems. The sector bound approach is employed to deal with quantization effects. Based on the fuzzy-basis-dependent Lyapunov function, sufficient conditions are established such that the filtering error system is stochastically stable and a prescribed noise attenuation level in an $H_\infty$ or $l_2$ – $l_\infty$ sense is achieved. Both numerical and practical examples are provided to show the feasibility and efficiency of the design schemes.

Novel Discrete-Time Zhang Neural Network for Time-Varying Matrix Inversion

Abstract: In the previous work, Zhang et al. developed a special type of recurrent neural networks called Zhang neural network (ZNN) with continuous-time and discrete-time forms for time-varying matrix inversion. In this paper, a novel discrete-time ZNN (DTZNN) model for time-varying matrix inversion is proposed and investigated. Specifically, a new numerical difference rule based on Taylor series expansion is established in this paper for first-order derivative approximation. Then, by exploiting this Taylor-type difference rule, the novel DTZNN model, which is a five-step iteration algorithm, is thus proposed for time-varying matrix inversion. Theoretical results are also presented for the proposed DTZNN model to show its excellent computational property. Comparative numerical results with three illustrative examples further substantiate the efficacy and superiority of the proposed DTZNN model for time-varying matrix inversion compared with previous DTZNN models.

p th moment exponential stabilisation of hybrid stochastic differential equations by feedback controls based on discrete-time state observations with a time delay

Abstract: The authors are concerned with the stability of hybrid stochastic differential equations by feedback controls based on discrete-time state observations. Under some reasonable conditions, they establish an upper bound on the duration τ between two consecutive state observations. Moreover, we can design the discrete-time state feedback control to stabilise the given hybrid stochastic differential equations in the sense of p th moment exponential stability by developing a new theory. In comparison to the results given in the previous literature, this study has two new characteristics: (i) the stability criterion concerns p th moment exponential stability, which is different from the existing works; (ii) discrete-time state observations depend on time delays.

Unbiased Minimum Variance Fault and State Estimation for Linear Discrete Time-Varying Two-Dimensional Systems

Abstract: The fault and state estimation problem is addressed for a class of linear discrete time-varying two-dimensional systems subject to state and measurement noises. Two estimators are proposed to compute the estimation of the system state and/or fault recursively, both of which are unbiased with minimum variance. Through formulating the estimation problem as the solvability problem of the corresponding matrix equations of estimator gains and system constraint, the necessary and sufficient condition of the existence and the solution for the proposed estimators are given. An example is used to demonstrate the effectiveness of the proposed estimators.

State Estimation of Discrete-Time Switched Neural Networks With Multiple Communication Channels

Abstract: In this paper, the state estimation problem for a class of discrete-time switched neural networks with modal persistent dwell time (MPDT) switching and mixed time delays is investigated. The considered switching law, not only generalizes the commonly studied dwell-time (DT) and average DT (ADT) switchings, but also further attaches mode-dependency to the persistent DT (PDT) switching that is shown to be more general. Multiple communication channels, which include one primary channel and multiredundant channels, are considered to coexist for the state estimation of underlying switched neural networks. The desired mode-dependent filters are designed such that the resulting filtering error system is exponentially mean-square stable with a guaranteed nonweighted generalized $ {\mathcal {H}}_{2}$ performance index. It is verified that better filtering performance index can be achieved as the number of channels to be used increases. The potential and effectiveness of the developed theoretical results are demonstrated via a numerical example.

State of Charge Estimation of Lithium-Ion Batteries Using a Discrete-Time Nonlinear Observer

Abstract: In this paper, a novel approach using the discrete-time nonlinear observer (DNLO) is proposed for state-of-charge (SOC) estimation of lithium-ion batteries (LIBs). To design the DNLO for SOC estimation, the state equations based on a second-order resistor–capacitor equivalent circuit model are derived to simulate the dynamic behaviors of an LIB. Considering the hysteresis characteristic of the battery, the model parameters depend on the SOC and the direction of battery current simultaneously, and then, the exponential-function fitting method is adopted to identify the offline results of the parameters. The ninth-order polynomial function is adopted to represent the highly nonlinear relationship between the open-circuit voltage and the SOC. The Lyapunov stability theory is used to prove the convergence of the proposed DNLO. The performance of the proposed method is also verified by the experiments based on the hybrid pulse power characteristic test, which indicates that compared with the extended Kalman filter (EKF) and the discrete-time sliding mode observer (SMO) algorithms, the proposed observer has better performance in reducing the computation cost, improving the estimation accuracy and enhancing the convergence capability.

Discrete-Time Nonzero-Sum Games for Multiplayer Using Policy-Iteration-Based Adaptive Dynamic Programming Algorithms

Abstract: In this paper, we investigate the nonzero-sum games for a class of discrete-time (DT) nonlinear systems by using a novel policy iteration (PI) adaptive dynamic programming (ADP) method. The main idea of our proposed PI scheme is to utilize the iterative ADP algorithm to obtain the iterative control policies, which not only ensure the system to achieve stability but also minimize the performance index function for each player. This paper integrates game theory, optimal control theory, and reinforcement learning technique to formulate and handle the DT nonzero-sum games for multiplayer. First, we design three actor-critic algorithms, an offline one and two online ones, for the PI scheme. Subsequently, neural networks are employed to implement these algorithms and the corresponding stability analysis is also provided via the Lyapunov theory. Finally, a numerical simulation example is presented to demonstrate the effectiveness of our proposed approach.

Stability Analysis of Discrete-Time Infinite-Horizon Optimal Control With Discounted Cost

Abstract: We analyze the stability of general nonlinear discrete-time systems controlled by an optimal sequence of inputs that minimizes an infinite-horizon discounted cost. First, assumptions related to the controllability of the system and its detectability with respect to the stage cost are made. Uniform semiglobal and practical stability of the closed-loop system is then established, where the adjustable parameter is the discount factor. Stronger stability properties are thereupon guaranteed by gradually strengthening the assumptions. Next, we show that the Lyapunov function used to prove stability is continuous under additional conditions, implying that stability has a certain amount of nominal robustness. The presented approach is flexible and we show that robust stability can still be guaranteed when the sequence of inputs applied to the system is no longer optimal but near-optimal. We also analyze stability for cost functions in which the importance of the stage cost increases with time, opposite to discounting. Finally, we exploit stability to derive new relationships between the optimal value functions of the discounted and undiscounted problems, when the latter is well-defined.

Event-Triggered Nonsynchronized $\mathcal {H}_{\infty }$ Filtering for Discrete-Time T–S Fuzzy Systems Based on Piecewise Lyapunov Functions

Abstract: This paper is concerned with the design of ${\mathcal {H}_{\infty }}$ event-triggered filter for a class of Takagi–Sugeno fuzzy systems. Based on the proposed communication strategy, only the measured outputs of the physical plant that violate a predefined triggering condition will win the right for transmission in the shared communication channel. Considering that the implementation of the filter may not be synchronized with the plant trajectories due to the asynchronous premise variables in network environment, a novel observer-based piecewise fuzzy filter is proposed. By adopting the idea of input delay method, the filtering error dynamics is reformulated as a new event-triggered piecewise fuzzy system. By applying a piecewise Lyapunov–Krasovskii functional and some techniques on matrix convexification, a method of event-triggered ${\mathcal {H}_{\infty }}$ piecewise filter design is developed for the filtering error system concerned to be asymptotically stable with a given disturbance attenuation level and reduced transmission rate. Moreover, a co-design algorithm to derive the filter gains and the event triggering parameters is proposed. Illustrative examples are finally given to show the effectiveness of the developed method.

General criteria for the study of quasi-stationarity

Abstract: For Markov processes with absorption, we provide general criteria ensuring the existence and the exponential non-uniform convergence in total variation norm to a quasi-stationary distribution. We also characterize a subset of its domain of attraction by an integrability condition, prove the existence of a right eigenvector for the semigroup of the process and the existence and exponential ergodicity of the Q-process. These results are applied to one-dimensional and multi-dimensional diffusion processes, to pure jump continuous time processes, to reducible processes with several communication classes, to perturbed dynamical systems and discrete time processes evolving in discrete state spaces.

Fault Estimation Observer Design for Discrete-Time Takagi–Sugeno Fuzzy Systems Based on Homogenous Polynomially Parameter-Dependent Lyapunov Functions

Abstract: This paper investigates the problem of robust fault estimation (FE) observer design for discrete-time Takagi–Sugeno fuzzy systems via homogenous polynomially parameter-dependent Lyapunov functions. First, a novel framework of the fuzzy FE observer is established with the help of a maximum–minimum-priority-based switching mechanism. Then, for every activated switching case, a targeted result is achieved by the aid of exploring an important property of improved homogenous polynomials. Since the helpful information of the underlying system can be duly updated and effectively utilized at every sampled point, the conservatism of previous results is availably reduced. Furthermore, the proposed result is further improved by eliminating those redundant terms of the introduced matrix-valued variables. Simulation results based on a discrete-time nonlinear truck-trailer model are provided to show the advantages of the theoretic result that is developed in this paper.

Security-guaranteed filtering for discrete-time stochastic delayed systems with randomly occurring sensor saturations and deception attacks

Abstract: Summary In this paper, the security-guaranteed filtering problem is studied for a class of nonlinear stochastic discrete time-delay systems with randomly occurring sensor saturations (ROSSs) and randomly occurring deception attacks (RODAs). The nonlinearities in systems satisfy the sector-bounded conditions, and the time-varying delays are unknown with given lower and upper bounds. A novel measurement output model is proposed to reflect both the ROSSs and the RODAs. A new definition is put forward on the security level with respect to the noise intensity, the energy bound of the false signals, the energy of the initial system state, and the desired security degree. We aim at designing a filter such that, in the presence of ROSSs and RODAs, the filtering error dynamics achieves the prescribed level of security. By using the stochastic analysis techniques, a sufficient condition is first derived under which the filtering error system is guaranteed to have the desired security level, and then, the filter gain is designed by solving a linear matrix inequality with nonlinear constraints. Finally, a numerical example is provided to demonstrate the feasibility of the proposed filtering scheme. Copyright © 2016 John Wiley & Sons, Ltd.

A distributed model predictive control based load frequency control scheme for multi-area interconnected power system using discrete-time Laguerre functions.

Abstract: This paper proposes a distributed model predictive control based load frequency control (MPC-LFC) scheme to improve control performances in the frequency regulation of power system. In order to reduce the computational burden in the rolling optimization with a sufficiently large prediction horizon, the orthonormal Laguerre functions are utilized to approximate the predicted control trajectory. The closed-loop stability of the proposed MPC scheme is achieved by adding a terminal equality constraint to the online quadratic optimization and taking the cost function as the Lyapunov function. Furthermore, the treatments of some typical constraints in load frequency control have been studied based on the specific Laguerre-based formulations. Simulations have been conducted in two different interconnected power systems to validate the effectiveness of the proposed distributed MPC-LFC as well as its superiority over the comparative methods.

The Cooperative Output Regulation Problem of Discrete-Time Linear Multi-Agent Systems by the Adaptive Distributed Observer

Abstract: In this paper, we first present an adaptive distributed observer for a discrete-time leader system. This adaptive distributed observer will provide, to each follower, not only the estimation of the leader's signal, but also the estimation of the leader's system matrix. Then, based on the estimation of the matrix S, we devise a discrete adaptive algorithm to calculate the solution to the regulator equations associated with each follower, and obtain an estimated feedforward control gain. Finally, we solve the cooperative output regulation problem for discrete-time linear multi-agent systems by both state feedback and output feedback adaptive distributed control laws utilizing the adaptive distributed observer.