Learning and tuning fuzzy logic controllers through reinforcements
TL;DR: The generalized approximate-reasoning-based intelligent control (GARIC) architecture learns and tunes a fuzzy logic controller even when only weak reinforcement is available; introduces a new conjunction operator in computing the rule strengths of fuzzy control rules; and learns to produce real-valued control actions.
Abstract: A method for learning and tuning a fuzzy logic controller based on reinforcements from a dynamic system is presented. It is shown that: the generalized approximate-reasoning-based intelligent control (GARIC) architecture learns and tunes a fuzzy logic controller even when only weak reinforcement, such as a binary failure signal, is available; introduces a new conjunction operator in computing the rule strengths of fuzzy control rules; introduces a new localized mean of maximum (LMOM) method in combining the conclusions of several firing control rules; and learns to produce real-valued control actions. Learning is achieved by integrating fuzzy inference into a feedforward network, which can then adaptively improve performance by using gradient descent methods. The GARIC architecture is applied to a cart-pole balancing system and demonstrates significant improvements in terms of the speed of learning and robustness to changes in the dynamic system's parameters over previous schemes for cart-pole balancing. >
Citations
More filters
01 Mar 1995
TL;DR: The essential part of neuro-fuzzy synergisms comes from a common framework called adaptive networks, which unifies both neural networks and fuzzy models, which possess certain advantages over neural networks.
Abstract: Fundamental and advanced developments in neuro-fuzzy synergisms for modeling and control are reviewed. The essential part of neuro-fuzzy synergisms comes from a common framework called adaptive networks, which unifies both neural networks and fuzzy models. The fuzzy models under the framework of adaptive networks is called adaptive-network-based fuzzy inference system (ANFIS), which possess certain advantages over neural networks. We introduce the design methods for ANFIS in both modeling and control applications. Current problems and future directions for neuro-fuzzy approaches are also addressed. >
2,260 citations
01 Mar 1995
TL;DR: After synthesizing a FLS, it is demonstrated that it can be expressed mathematically as a linear combination of fuzzy basis functions, and is a nonlinear universal function approximator, a property that it shares with feedforward neural networks.
Abstract: A fuzzy logic system (FLS) is unique in that it is able to simultaneously handle numerical data and linguistic knowledge. It is a nonlinear mapping of an input data (feature) vector into a scalar output, i.e., it maps numbers into numbers. Fuzzy set theory and fuzzy logic establish the specifics of the nonlinear mapping. This tutorial paper provides a guided tour through those aspects of fuzzy sets and fuzzy logic that are necessary to synthesize an FLS. It does this by starting with crisp set theory and dual logic and demonstrating how both can be extended to their fuzzy counterparts. Because engineering systems are, for the most part, causal, we impose causality as a constraint on the development of the FLS. After synthesizing a FLS, we demonstrate that it can be expressed mathematically as a linear combination of fuzzy basis functions, and is a nonlinear universal function approximator, a property that it shares with feedforward neural networks. The fuzzy basis function expansion is very powerful because its basis functions can be derived from either numerical data or linguistic knowledge, both of which can be cast into the forms of IF-THEN rules. >
2,024 citations
Cites background from "Learning and tuning fuzzy logic con..."
...Parameters of a FLS can also be trained on-line using reinforcement learning [ 4 ]....
[...]
TL;DR: A survey on recent developments (or state of the art) of analysis and design of model based fuzzy control systems based on the so-called Takagi-Sugeno fuzzy models or fuzzy dynamic models.
Abstract: Fuzzy logic control was originally introduced and developed as a model free control design approach. However, it unfortunately suffers from criticism of lacking of systematic stability analysis and controller design though it has a great success in industry applications. In the past ten years or so, prevailing research efforts on fuzzy logic control have been devoted to model-based fuzzy control systems that guarantee not only stability but also performance of closed-loop fuzzy control systems. This paper presents a survey on recent developments (or state of the art) of analysis and design of model based fuzzy control systems. Attention will be focused on stability analysis and controller design based on the so-called Takagi-Sugeno fuzzy models or fuzzy dynamic models. Perspectives of model based fuzzy control in future are also discussed
1,575 citations
Additional excerpts
...works, whose connection weights correspond to the parameters of fuzzy reasoning [38], [123], [124], [135], [187], [220], [231], [232], [264]....
[...]
[...]
TL;DR: This chapter discusses Reinforcement Learning with Self-Modifying Policies J. Schmidhuber, et al., and theoretical Models of Learning to Learn J. Baxter, a first step towards Continual Learning.
Abstract: Preface. Part I: Overview Articles. 1. Learning to Learn: Introduction and Overview S. Thrun, L. Pratt. 2. A Survey of Connectionist Network Reuse Through Transfer L. Pratt, B. Jennings. 3. Transfer in Cognition A. Robins. Part II: Prediction. 4. Theoretical Models of Learning to Learn J. Baxter. 5. Multitask Learning R. Caruana. 6. Making a Low-Dimensional Representation Suitable for Diverse Tasks N. Intrator, S. Edelman. 7. The Canonical Distortion Measure for Vector Quantization and Function Approximation J. Baxter. 8. Lifelong Learning Algorithms S. Thrun. Part III: Relatedness. 9. The Parallel Transfer of Task Knowledge Using Dynamic Learning Rates Based on a Measure of Relatedness D.L. Silver, R.E. Mercer. 10. Clustering Learning Tasks and the Selective Cross-Task Transfer of Knowledge S. Thrun, J. O'Sullivan. Part IV: Control. 11. CHILD: A First Step Towards Continual Learning M.B. Ring. 12. Reinforcement Learning with Self-Modifying Policies J. Schmidhuber, et al. 13. Creating Advice-Taking Reinforcement Learners R. Maclin, J.W. Shavlik. Contributing Authors. Index.
1,382 citations
TL;DR: A linear transformation for each input variable can be incorporated into the network so that much fewer rules are needed or higher accuracy can be achieved.
Abstract: A self-constructing neural fuzzy inference network (SONFIN) with online learning ability is proposed in this paper. The SONFIN is inherently a modified Takagi-Sugeno-Kang (TSK)-type fuzzy rule-based model possessing neural network learning ability. There are no rules initially in the SONFIN. They are created and adapted as online learning proceeds via simultaneous structure and parameter identification. In the structure identification of the precondition part, the input space is partitioned in a flexible way according to an aligned clustering-based algorithm. As to the structure identification of the consequent part, only a singleton value selected by a clustering method is assigned to each rule initially. Afterwards, some additional significant terms selected via a projection-based correlation measure for each rule will be added to the consequent part incrementally as learning proceeds. The combined precondition and consequent structure identification scheme can set up an economic and dynamically growing network, a main feature of the SONFIN. In the parameter identification, the consequent parameters are tuned optimally by either least mean squares or recursive least squares algorithms and the precondition parameters are tuned by a backpropagation algorithm. To enhance the knowledge representation ability of the SONFIN, a linear transformation for each input variable can be incorporated into the network so that much fewer rules are needed or higher accuracy can be achieved.
910 citations
Cites methods from "Learning and tuning fuzzy logic con..."
...Only the center of each Gaussian membership function is delivered to the next layer for the local mean of maximum (LMOM) defuzzification operation [23] and the width is used for output clustering only....
[...]
References
More filters
01 Jan 1988
TL;DR: This chapter contains sections titled: The Problem, The Generalized Delta Rule, Simulation Results, Some Further Generalizations, Conclusion.
Abstract: This chapter contains sections titled: The Problem, The Generalized Delta Rule, Simulation Results, Some Further Generalizations, Conclusion
17,604 citations
"Learning and tuning fuzzy logic con..." refers background or methods in this paper
...This structure is shown in Fig. 4. The learning algorithm is composed of Sutton’s AHC algorithm [26] for the output unit and the error back-propagation algorithm [ 24 ] for the hidden units....
[...]
...The weight update function for the hidden layer is based on a modified version of the error back-propagation algorithm [ 24 ]....
[...]
...[ 24 ] or some sort of line-search technique to determine the optimal step size at each point 1151....
[...]
Book•
03 Jan 1986
TL;DR: In this paper, the problem of the generalized delta rule is discussed and the Generalized Delta Rule is applied to the simulation results of simulation results in terms of the generalized delta rule.
Abstract: This chapter contains sections titled: The Problem, The Generalized Delta Rule, Simulation Results, Some Further Generalizations, Conclusion
13,579 citations
Book•
01 Jan 1985TL;DR: This chapter discusses Adaptive Arrays and Adaptive Beamforming, as well as other Adaptive Algorithms and Structures, and discusses the Z-Transform in Adaptive Signal Processing.
Abstract: GENERAL INTRODUCTION. Adaptive Systems. The Adaptive Linear Combiner. THEORY OF ADAPTATION WITH STATIONARY SIGNALS. Properties of the Quadratic Performance Surface. Searching the Performance Surface. Gradient Estimation and Its Effects on Adaptation. ADAPTIVE ALGORITHMS AND STRUCTURES. The LMS Algorithm. The Z-Transform in Adaptive Signal Processing. Other Adaptive Algorithms and Structures. Adaptive Lattice Filters. APPLICATIONS. Adaptive Modeling and System Identification. Inverse Adaptive Modeling, Deconvolution, and Equalization. Adaptive Control Systems. Adaptive Interference Cancelling. Introduction to Adaptive Arrays and Adaptive Beamforming. Analysis of Adaptive Beamformers.
5,645 citations
TL;DR: This article introduces a class of incremental learning procedures specialized for prediction – that is, for using past experience with an incompletely known system to predict its future behavior – and proves their convergence and optimality for special cases and relation to supervised-learning methods.
Abstract: This article introduces a class of incremental learning procedures specialized for prediction – that is, for using past experience with an incompletely known system to predict its future behavior. Whereas conventional prediction-learning methods assign credit by means of the difference between predicted and actual outcomes, the new methods assign credit by means of the difference between temporally successive predictions. Although such temporal-difference methods have been used in Samuel's checker player, Holland's bucket brigade, and the author's Adaptive Heuristic Critic, they have remained poorly understood. Here we prove their convergence and optimality for special cases and relate them to supervised-learning methods. For most real-world prediction problems, temporal-difference methods require less memory and less peak computation than conventional methods and they produce more accurate predictions. We argue that most problems to which supervised learning is currently applied are really prediction problems of the sort to which temporal-difference methods can be applied to advantage.
4,803 citations
TL;DR: This work proposes a network architecture which uses a single internal layer of locally-tuned processing units to learn both classification tasks and real-valued function approximations (Moody and Darken 1988).
Abstract: We propose a network architecture which uses a single internal layer of locally-tuned processing units to learn both classification tasks and real-valued function approximations (Moody and Darken 1988). We consider training such networks in a completely supervised manner, but abandon this approach in favor of a more computationally efficient hybrid learning method which combines self-organized and supervised learning. Our networks learn faster than backpropagation for two reasons: the local representations ensure that only a few units respond to any given input, thus reducing computational overhead, and the hybrid learning rules are linear rather than nonlinear, thus leading to faster convergence. Unlike many existing methods for data analysis, our network architecture and learning rules are truly adaptive and are thus appropriate for real-time use.
4,406 citations
"Learning and tuning fuzzy logic con..." refers methods in this paper
...The center and spreads may be considered as weights on the input links, analogous to the approach taken with radial-basis-function units in neural networks [ 20 ]....
[...]