Continuous reinforcement learning to robust fault tolerant control for a class of unknown nonlinear systems
01 Dec 2015-Vol. 37, pp 702-714
TL;DR: Two strategies to design robust adaptive fault tolerant control (FTC) systems for a class of unknown n-order nonlinear systems in presence of actuator and sensor faults versus bounded unknown external disturbances are proposed.
Abstract: Proposing two robust adaptive FTC systems based on machine learning approachesPresenting adaptation laws in the sense of the proposed Lyapunov functionUsing an intelligent observer for unknown nonlinear systems in presence of faultsAdapting the critic and actor of continuous RL based on the Lyapunov function This paper proposes two strategies to design robust adaptive fault tolerant control (FTC) systems for a class of unknown n-order nonlinear systems in presence of actuator and sensor faults versus bounded unknown external disturbances It is based on machine learning approaches which are continuous reinforcement learning (RL) and neural networks (NNs) In the first FTC strategy, an intelligent observer is designed for unknown nonlinear systems when faults occur or not In the second strategy, a robust reinforcement learning FTC is proposed through combining reinforcement learning to treat the unknown nonlinear faulty system and nonlinear control theory to guarantee the stability and robustness of the system Critic and actor of continuous RL are adopted based on the behavior of the defined Lyapunov function In both strategies, to generate the residual a Gaussian radial basis function is used for an online estimation of the unknown dynamic function of the normal system The adaptation law of the online estimator is derived in the sense of Lyapunov function which is defined based on adjustable parameters of the estimator and switching surfaces containing dynamic errors and residuals Simulation results demonstrate the validity and feasibility of proposed FTC systems
Citations
More filters
01 Jan 2005
TL;DR: The paper reviews the development of fault-tolerant control systems with highlights to several important issues from a historical perspective and some critical issues in this area are discussed as open problems for future research/development in this emerging field.
Abstract: This paper presents an introductory overview on the development of fault-tolerant control systems. For this reason, the paper is written in a tutorial fashion to summarize some of the important results in this subject area deliberately without going into details in any of them. However, key references are provided from which interested readers can obtain more detailed information on a particular subject. It is necessary to mention that, throughout this paper, no efforts were made to provide an exhaustive coverage on the subject matter. In fact, it is far from it. The paper merely represents the view and experience of its author. It can very well be that some important issues or topics were left out unintentionally. If that is the case, the author sincerely apologizes in advance.After a brief account of fault-tolerant control systems, particularly on the original motivations, and the concept of redundancies, the paper reviews the development of fault-tolerant control systems with highlights to several important issues from a historical perspective. The general approaches to fault-tolerant control has been divided into passive, active, and hybrid approaches. The analysis techniques for active fault-tolerant control systems are also discussed. Practical applications of faulttolerant control are highlighted from a practical and industrial perspective. Finally, some critical issues in this area are discussed as open problems for future research/development in this emerging field.
99 citations
TL;DR: A comprehensive survey of the state-of-the-art work on fault tolerance methods proposed for cloud computing is presented and current issues and challenges in cloud fault tolerance are discussed to identify promising areas for future research.
Abstract: This paper presents a comprehensive survey of the state-of-the-art work on fault tolerance methods proposed for cloud computing. The survey classifies fault-tolerance methods into three categories: 1) ReActive Methods (RAMs); 2) PRoactive Methods (PRMs); and 3) ReSilient Methods (RSMs). RAMs allow the system to enter into a fault status and then try to recover the system. PRMs tend to prevent the system from entering a fault status by implementing mechanisms that enable them to avoid errors before they affect the system. On the other hand, recently emerging RSMs aim to minimize the amount of time it takes for a system to recover from a fault. Machine Learning and Artificial Intelligence have played an active role in RSM domain in such a way that the recovery time is mapped to a function to be optimized (i.e., by converging the recovery time to a fraction of milliseconds). As the system learns to deal with new faults, the recovery time will become shorter. In addition, current issues and challenges in cloud fault tolerance are also discussed to identify promising areas for future research.
71 citations
Cites background or methods from "Continuous reinforcement learning t..."
...Farivar and Ahmadabadi [19] propose two strategies that can be used to design a robust and adaptive fault tolerant control (FTC) system....
[...]
...A number of fault tolerance studies have already applied Machine Learning in various ways to introduce intelligence and resiliency [17], [18], [19], [20], [21], [22],...
[...]
...addition of the ability learn through interaction with the environment and to adapt the fault tolerance [19], [20], [21], [22], [23], [24], [25]....
[...]
...Reinforcement Learning appears to be the most common technique used in the FT domain [19], [20], [21], [22], [23], [24], [25], [86]....
[...]
...the most common technique used in the FT domain [19], [20], [21], [22], [23], [24], [25], [39], [86]....
[...]
01 May 2020
TL;DR: The result shows that the generic fault-tolerant control strategy via reinforcement learning can effectively tolerate different types of attacks/faults and maintain the vehicle's position, outperforming the other two methods.
Abstract: In this paper, we present a generic fault-tolerant control (FTC) strategy via reinforcement learning (RL). We demonstrate the effectiveness of this method on quadcopter unmanned aerial vehicles (UAVs). The fault-tolerant control policy is trained to handle actuator and sensor fault/attack. Unlike traditional FTC, this policy does not require fault detection and diagnosis (FDD) nor tailoring the controller for specific attack scenarios. Instead, the policy is running simultaneously alongside the stabilizing controller without the need for on- detection activation. The effectiveness of the policy is compared with traditional active and passive FTC strategies against actuator and sensor faults. We compare their performance in position control tasks via simulation and experiments on quadcopters. The result shows that the strategy can effectively tolerate different types of attacks/faults and maintain the vehicle's position, outperforming the other two methods.
48 citations
Cites methods from "Continuous reinforcement learning t..."
...proposed methods using learning based observer and RL based controller to treat bounded additive sensor and actuator fault [21]....
[...]
TL;DR: In this paper, an efficient method based on axial MFL level contours is devised to estimate the length of the defect using a Gaussian radial basis function neural network (NN).
Abstract: Magnetic flux leakage (MFL) method is the most widely used non-destructive testing techniques for detection and characterization of defects in product transmission pipelines. The maximum safe operating pressure is a crucial parameter in practice, which is predicted with respect to the length, width, and depth of defects. In a previous work, an algorithm is devised to detect and estimate the width of the defect based on MFL signals. In this paper, an efficient method based on axial MFL level contours is devised to estimate the length of the defect. This method uses the patterns of signal level contours in the region corresponding to the defect’s area. In addition, a Gaussian radial basis function neural network (NN) is trained to approximate the depth of the defect. The NN is fed with the estimated length, width, and signal peak-to-peak values, and the output of the network is the estimated depth. The proposed detection and estimation method is applied to MFL measurements to detect and determine the sizing of metal loss defects. The efficacy and accuracy of the proposed methods are examined through a rich set of sample defects that contains simulated defects, designed defects that are carved on a real pipe by milling, and actual pigging data validated by several dig up verifications. Obtained results confirm the effectiveness and accuracy of the proposed depth and length estimation method along with the previously devised detection and width estimation methods, in characterization of metal loss defects.
41 citations
Cites background from "Continuous reinforcement learning t..."
...The network output can perform the mapping according to [30]–[32]...
[...]
References
More filters
Book•
01 Jan 1991
TL;DR: Covers in a progressive fashion a number of analysis tools and design techniques directly applicable to nonlinear control problems in high performance systems (in aerospace, robotics and automotive areas).
Abstract: Covers in a progressive fashion a number of analysis tools and design techniques directly applicable to nonlinear control problems in high performance systems (in aerospace, robotics and automotive areas).
15,545 citations
Additional excerpts
...The switching surface is defined as [18]:...
[...]
Book•
01 Mar 1998
TL;DR: In Reinforcement Learning, Richard Sutton and Andrew Barto provide a clear and simple account of the key ideas and algorithms of reinforcement learning.
Abstract: From the Publisher:
In Reinforcement Learning, Richard Sutton and Andrew Barto provide a clear and simple account of the key ideas and algorithms of reinforcement learning. Their discussion ranges from the history of the field's intellectual foundations to the most recent developments and applications. The only necessary mathematical background is familiarity with elementary concepts of probability.
7,016 citations
Book•
22 Dec 2012
TL;DR: In this paper, model-based analysis and design methods for fault diagnosis and fault-tolerant control are presented, where the propagation of the fault through the process, test fault detectability and reveal redundancies that can be used to ensure fault tolerance.
Abstract: This book presents model-based analysis and design methods for fault diagnosis and fault-tolerant control. Architectural and structural models are used to analyse the propagation of the fault through the process, test fault detectability and reveal redundancies that can be used to ensure fault tolerance. Case studies demonstrate the methods presented. The second edition includes new material on reconfigurable control, diagnosis of nonlinear systems, and remote diagnosis, plus new examples and updated bibliography.
2,677 citations
"Continuous reinforcement learning t..." refers background in this paper
...On the other hand, the consistency of the system with the normal model can be checked at every time t by determining the difference f N R y y , which is called a residual [6]....
[...]
TL;DR: A bibliographical review on reconfigurable fault-tolerant control systems (FTCS) is presented, with emphasis on the reconfiguring/restructurable controller design techniques.
Abstract: In this paper, a bibliographical review on reconfigurable (active) fault-tolerant control systems (FTCS) is presented. The existing approaches to fault detection and diagnosis (FDD) and fault-tolerant control (FTC) in a general framework of active fault-tolerant control systems (AFTCS) are considered and classified according to different criteria such as design methodologies and applications. A comparison of different approaches is briefly carried out. Focuses in the field on the current research are also addressed with emphasis on the practical application of the techniques. In total, 376 references in the open literature, dating back to 1971, are compiled to provide an overall picture of historical, current, and future developments in this area.
2,455 citations
"Continuous reinforcement learning t..." refers background in this paper
...Page 2 of 28 Ac ce pt ed M an us cr ip t There are survey papers [1-5] published for explaining the FTC problem, introducing the solving approaches, and describing the research improvements in this domain....
[...]
...According to [1-5], FTC systems can be classified into passive and active FTC systems....
[...]