Reinforcement learning, one of the most active research areas in artificial intelligence, is a computational approach to learning whereby an agent tries to maximize the total amount of reward it receives when interacting with a complex, uncertain environment. In Reinforcement Learning, Richard Sutton and Andrew Barto provide a clear and simple account of the key ideas and algorithms of reinforcement learning. Their discussion ranges from the history of the field's intellectual foundations to the most recent developments and applications. The only necessary mathematical background is familiarity with elementary concepts of probability. The book is divided into three parts. Part I defines the reinforcement learning problem in terms of Markov decision processes. Part II provides basic solution methods: dynamic programming, Monte Carlo methods, and temporal-difference learning. Part III presents a unified view of the solution methods and incorporates artificial neural networks, eligibility traces, and planning; the two final chapters present case studies and consider the future of reinforcement learning.

/pdf/reinforcement-learning-an-introduction-rzxgej9p17.pdf

Reinforcement Learning: An Introduction

The architecture and learning procedure underlying ANFIS (adaptive-network-based fuzzy inference system) is presented, which is a fuzzy inference system implemented in the framework of adaptive networks. By using a hybrid learning procedure, the proposed ANFIS can construct an input-output mapping based on both human knowledge (in the form of fuzzy if-then rules) and stipulated input-output data pairs. In the simulation, the ANFIS architecture is employed to model nonlinear functions, identify nonlinear components on-line in a control system, and predict a chaotic time series, all yielding remarkable results. Comparisons with artificial neural networks and earlier work on fuzzy modeling are listed and discussed. Other extensions of the proposed ANFIS and promising applications to automatic control and signal processing are also suggested. >

ANFIS: adaptive-network-based fuzzy inference system

https://hal.archives-ouvertes.fr/hal-00417283/document

Independent component analysis, a new concept?

Fundamentals Of Statistical Signal Processing

In 1974 an article appeared in Science magazine with the dry-sounding title “Judgment Under Uncertainty: Heuristics and Biases” by a pair of psychologists who were not well known outside their discipline of decision theory. In it Amos Tversky and Daniel Kahneman introduced the world to Prospect Theory, which mapped out how humans actually behave when faced with decisions about gains and losses, in contrast to how economists assumed that people behave. Prospect Theory turned Economics on its head by demonstrating through a series of ingenious experiments that people are much more concerned with losses than they are with gains, and that framing a choice from one perspective or the other will result in decisions that are exactly the opposite of each other, even if the outcomes are monetarily the same. Prospect Theory led cognitive psychology in a new direction that began to uncover other human biases in thinking that are probably not learned but are part of our brain’s wiring.

Thinking fast and slow.

A general method is developed to generate fuzzy rules from numerical data. The method consists of five steps: divide the input and output spaces of the given numerical data into fuzzy regions; generate fuzzy rules from the given data; assign a degree of each of the generated rules for the purpose of resolving conflicts among the generated rules; create a combined fuzzy rule base based on both the generated rules and linguistic rules of human experts; and determine a mapping from input space to output space based on the combined fuzzy rule base using a defuzzifying procedure. The mapping is proved to be capable of approximating any real continuous function on a compact set to arbitrary accuracy. Applications to truck backer-upper control and time series prediction problems are presented. >

Generating fuzzy rules by learning from examples

Fuzzy systems are represented as series expansions of fuzzy basis functions which are algebraic superpositions of fuzzy membership functions. Using the Stone-Weierstrass theorem, it is proved that linear combinations of the fuzzy basis functions are capable of uniformly approximating any real continuous function on a compact set to arbitrary accuracy. Based on the fuzzy basis function representations, an orthogonal least-squares (OLS) learning algorithm is developed for designing fuzzy systems based on given input-output pairs; then, the OLS algorithm is used to select significant fuzzy basis functions which are used to construct the final fuzzy system. The fuzzy basis function expansion is used to approximate a controller for the nonlinear ball and beam system, and the simulation results show that the control performance is improved by incorporating some common-sense fuzzy control rules. >

Fuzzy basis functions, universal approximation, and orthogonal least-squares learning

(NOTE: Each chapter concludes with Exercises.) I: PRELIMINARIES. 1. Introduction. Rule-Based FLSs. A New Direction for FLSs. New Concepts and Their Historical Background. Fundamental Design Requirement. The Flow of Uncertainties. Existing Literature on Type-2 Fuzzy Sets. Coverage. Applicability Outside of Rule-Based FLSs. Computation. Supplementary Material: Short Primers on Fuzzy Sets and Fuzzy Logic. Primer on Fuzzy Sets. Primer on FL. Remarks. 2. Sources of Uncertainty. Uncertainties in a FLS. Words Mean Different Things to Different People. 3. Membership Functions and Uncertainty. Introduction. Type-1 Membership Functions. Type-2 Membership Functions. Returning to Linguistic Labels. Multivariable Membership Functions. Computation. 4. Case Studies. Introduction. Forecasting of Time-Series. Knowledge Mining Using Surveys. II: TYPE-1 FUZZY LOGIC SYSTEMS. 5. Singleton Type-1 Fuzzy Logic Systems: No Uncertainties. Introduction. Rules. Fuzzy Inference Engine. Fuzzification and Its Effect on Inference. Defuzzification. Possibilities. Fuzzy Basis Functions. FLSs Are Universal Approximators. Designing FLSs. Case Study: Forecasting of Time-Series. Case Study: Knowledge Mining Using Surveys. A Final Remark. Computation. 6. Non-Singleton Type-1 Fuzzy Logic Systems. Introduction. Fuzzification and Its Effect on Inference. Possibilities. FBFs. Non-Singleton FLSs Are Universal Approximators. Designing Non-Singleton FLSs. Case Study: Forecasting of Time-Series. A Final Remark. Computation. III: TYPE-2 FUZZY SETS. 7. Operations on and Properties of Type-2 Fuzzy Sets. Introduction. Extension Principle. Operations on General Type-2 Fuzzy Sets. Operations on Interval Type-2 Fuzzy Sets. Summary of Operations. Properties of Type-2 Fuzzy Sets. Computation. 8. Type-2 Relations and Compositions. Introduction. Relations in General. Relations and Compositions on the Same Product Space. Relations and Compositions on Different Product Spaces. Composition of a Set with a Relation. Cartesian Product of Fuzzy Sets. Implications. 9. Centroid of a Type-2 Fuzzy Set: Type-Reduction. Introduction. General Results for the Centroid. Generalized Centroid for Interval Type-2 Fuzzy Sets. Centroid of an Interval Type-2 Fuzzy Set. Type-Reduction: General Results. Type-Reduction: Interval Sets. Concluding Remark. Computation. IV: TYPE-2 FUZZY LOGIC SYSTEMS. 10. Singleton Type-2 Fuzzy Logic Systems. Introduction. Rules. Fuzzy Inference Engine. Fuzzification and Its Effect on Inference. Type-Reduction. Defuzzification. Possibilities. FBFs: The Lack Thereof. Interval Type-2 FLSs. Designing Interval Singleton Type-2 FLSs. Case Study: Forecasting of Time-Series. Case Study: Knowledge Mining Using Surveys. Computation. 11. Type-1 Non-Singleton Type-2 Fuzzy Logic Systems. Introduction. Fuzzification and Its Effect on Inference. Interval Type-1 Non-Singleton Type-2 FLSs. Designing Interval Type-1 Non-Singleton Type-2 FLSs. Case Study: Forecasting of Time-Series. Final Remark. Computation. 12. Type-2 Non-Singleton Type-2 Fuzzy Logic Systems. Introduction. Fuzzification and Its Effect on Inference. Interval Type-2 Non-Singleton Type-2 FLSs. Designing Interval Type-2 Non-Singleton Type-2 FLSs. Case Study: Forecasting of Time-Series. Computation. 13. TSK Fuzzy Logic Systems. Introduction. Type-1 TSK FLSs. Type-2 TSK FLSs. Example: Forecasting of Compressed Video Traffic. Final Remark. Computation. 14. Epilogue. Introduction. Type-2 Versus Type-1 FLSs. Appropriate Applications for a Type-2 FLS. Rule-Based Classification of Video Traffic. Equalization of Time-Varying Non-linear Digital Communication Channels. Overcoming CCI and ISI for Digital Communication Channels. Connection Admission Control for ATM Networks. Potential Application Areas for a Type-2 FLS. A. Join, Meet, and Negation Operations For Non-Interval Type-2 Fuzzy Sets. Introduction. Join Under Minimum or Product t-Norms. Meet Under Minimum t-Norm. Meet Under Product t-Norm. Negation. Computation. B. Properties of Type-1 and Type-2 Fuzzy Sets. Introduction. Type-1 Fuzzy Sets. Type-2 Fuzzy Sets. C. Computation. Type-1 FLSs. General Type-2 FLSs. Interval Type-2 FLSs. References. Index.

Uncertain Rule-Based Fuzzy Logic Systems: Introduction and New Directions

Type-2 fuzzy sets let us model and minimize the effects of uncertainties in rule-base fuzzy logic systems. However, they are difficult to understand for a variety of reasons which we enunciate. In this paper, we strive to overcome the difficulties by: (1) establishing a small set of terms that let us easily communicate about type-2 fuzzy sets and also let us define such sets very precisely, (2) presenting a new representation for type-2 fuzzy sets, and (3) using this new representation to derive formulas for union, intersection and complement of type-2 fuzzy sets without having to use the Extension Principle.

/pdf/type-2-fuzzy-sets-made-simple-3iyg95vkei.pdf

Type-2 fuzzy sets made simple

A fuzzy logic system (FLS) is unique in that it is able to simultaneously handle numerical data and linguistic knowledge. It is a nonlinear mapping of an input data (feature) vector into a scalar output, i.e., it maps numbers into numbers. Fuzzy set theory and fuzzy logic establish the specifics of the nonlinear mapping. This tutorial paper provides a guided tour through those aspects of fuzzy sets and fuzzy logic that are necessary to synthesize an FLS. It does this by starting with crisp set theory and dual logic and demonstrating how both can be extended to their fuzzy counterparts. Because engineering systems are, for the most part, causal, we impose causality as a constraint on the development of the FLS. After synthesizing a FLS, we demonstrate that it can be expressed mathematically as a linear combination of fuzzy basis functions, and is a nonlinear universal function approximator, a property that it shares with feedforward neural networks. The fuzzy basis function expansion is very powerful because its basis functions can be derived from either numerical data or linguistic knowledge, both of which can be cast into the forms of IF-THEN rules. >

Jerry M. Mendel

Papers

Generating fuzzy rules by learning from examples

Fuzzy basis functions, universal approximation, and orthogonal least-squares learning

Uncertain Rule-Based Fuzzy Logic Systems: Introduction and New Directions

Type-2 fuzzy sets made simple

Fuzzy logic systems for engineering: a tutorial