https://www2.econ.iastate.edu/tesfatsi/DeepLearningInNeuralNetworksOverview.JSchmidhuber2015.pdf

Deep learning in neural networks

Machine Learning is the study of methods for programming computers to learn. Computers are applied to a wide range of tasks, and for most of these it is relatively easy for programmers to design and implement the necessary software. However, there are many tasks for which this is difficult or impossible. These can be divided into four general categories. First, there are problems for which there exist no human experts. For example, in modern automated manufacturing facilities, there is a need to predict machine failures before they occur by analyzing sensor readings. Because the machines are new, there are no human experts who can be interviewed by a programmer to provide the knowledge necessary to build a computer system. A machine learning system can study recorded data and subsequent machine failures and learn prediction rules. Second, there are problems where human experts exist, but where they are unable to explain their expertise. This is the case in many perceptual tasks, such as speech recognition, hand-writing recognition, and natural language understanding. Virtually all humans exhibit expert-level abilities on these tasks, but none of them can describe the detailed steps that they follow as they perform them. Fortunately, humans can provide machines with examples of the inputs and correct outputs for these tasks, so machine learning algorithms can learn to map the inputs to the outputs. Third, there are problems where phenomena are changing rapidly. In finance, for example, people would like to predict the future behavior of the stock market, of consumer purchases, or of exchange rates. These behaviors change frequently, so that even if a programmer could construct a good predictive computer program, it would need to be rewritten frequently. A learning program can relieve the programmer of this burden by constantly modifying and tuning a set of learned prediction rules. Fourth, there are applications that need to be customized for each computer user separately. Consider, for example, a program to filter unwanted electronic mail messages. Different users will need different filters. It is unreasonable to expect each user to program his or her own rules, and it is infeasible to provide every user with a software engineer to keep the rules up-to-date. A machine learning system can learn which mail messages the user rejects and maintain the filtering rules automatically. Machine learning addresses many of the same research questions as the fields of statistics, data mining, and psychology, but with differences of emphasis. Statistics focuses on understanding the phenomena that have generated the data, often with the goal of testing different hypotheses about those phenomena. Data mining seeks to find patterns in the data that are understandable by people. Psychological studies of human learning aspire to understand the mechanisms underlying the various learning behaviors exhibited by people (concept learning, skill acquisition, strategy change, etc.).

Machine learning

[신간의 별자리x] 우리/미술, 그리고 ‘슬픔의 박물관’

The purpose of this paper is to provide a comprehensive presentation and interpretation of the Ensemble Kalman Filter (EnKF) and its numerical implementation. The EnKF has a large user group, and numerous publications have discussed applications and theoretical aspects of it. This paper reviews the important results from these studies and also presents new ideas and alternative interpretations which further explain the success of the EnKF. In addition to providing the theoretical framework needed for using the EnKF, there is also a focus on the algorithmic formulation and optimal numerical implementation. A program listing is given for some of the key subroutines. The paper also touches upon specific issues such as the use of nonlinear measurements, in situ profiles of temperature and salinity, and data which are available with high frequency in time. An ensemble based optimal interpolation (EnOI) scheme is presented as a cost-effective approach which may serve as an alternative to the EnKF in some applications. A fairly extensive discussion is devoted to the use of time correlated model errors and the estimation of model bias.

The Ensemble Kalman Filter: Theoretical formulation and practical implementation

Reinforcement learning offers to robotics a framework and set of tools for the design of sophisticated and hard-to-engineer behaviors. Conversely, the challenges of robotic problems provide both inspiration, impact, and validation for developments in reinforcement learning. The relationship between disciplines has sufficient promise to be likened to that between physics and mathematics. In this article, we attempt to strengthen the links between the two research communities by providing a survey of work in reinforcement learning for behavior generation in robots. We highlight both key challenges in robot reinforcement learning as well as notable successes. We discuss how contributions tamed the complexity of the domain and study the role of algorithms, representations, and prior knowledge in achieving these successes. As a result, a particular focus of our paper lies on the choice between model-based and model-free as well as between value-function-based and policy-search methods. By analyzing a simple problem in some detail we demonstrate how reinforcement learning approaches may be profitably applied, and we note throughout open questions and the tremendous potential for future research.

/pdf/reinforcement-learning-in-robotics-a-survey-4w4397qx9i.pdf

Reinforcement learning in robotics: A survey

Deep Neural Networks (DNNs) can be used as function approximators in Reinforcement Learning (RL). One advantage of DNNs is that they can cope with large input dimensions. Instead of relying on feature engineering to lower the input dimension, DNNs can extract the features from raw observations. The drawback of this end-to-end learning is that it usually requires a large amount of data, which for real-world control applications is not always available. In this paper, a new algorithm, Model Learning Deep Deterministic Policy Gradient (ML-DDPG), is proposed that combines RL with state representation learning, i.e., learning a mapping from an input vector to a state before solving the RL task. The ML-DDPG algorithm uses a concept we call predictive priors to learn a model network which is subsequently used to pre-train the first layer of the actor and critic networks. Simulation results show that the ML-DDPG can learn reasonable continuous control policies from high-dimensional observations that contain also task-irrelevant information. Furthermore, in some cases, this approach significantly improves the final performance in comparison to end-to-end learning.

/pdf/learning-state-representation-for-deep-actor-critic-control-2x4k0k3pei.pdf

Learning state representation for deep actor-critic control

A novel framework for fuzzy modeling and model-based control design is described. The fuzzy model is of the Takagi-Sugeno (TS) type with constant consequents. It uses multivariate antecedent membership functions obtained by Delaunay triangulation of their characteristic points. The number and position of these points are determined by an iterative insertion algorithm. Constrained optimization is used to estimate the consequent parameters, where the constraints are based on control-relevant a priori knowledge about the modeled process. Finally, methods for control design through linearization and inversion of this model are developed. The proposed techniques are demonstrated by means of two benchmark examples: identification of the well-known Box-Jenkins gas furnace and inverse model-based control of a pH process. The obtained results are compared with results from the literature.

Fuzzy modeling with multivariate membership functions: gray-box identification and control design

A large class of nonlinear systems can be well approximated by Takagi-Sugeno (TS) fuzzy models with linear or affine consequents. It is well known that the stability of these consequent models does not ensure the stability of the overall fuzzy system. Therefore, several stability conditions have been developed for TS fuzzy systems. We study a special class of nonlinear dynamic systems that can be decomposed into cascaded subsystems, which are represented as TS fuzzy models. We analyze the stability of the overall TS system based on the stability of the subsystems and prove that the stability of the subsystems implies the stability of the overall system. The main benefit of this approach is that it relaxes the conditions imposed when the system is globally analyzed, thereby solving some of the feasibility problems. Another benefit is that by using this approach, the dimension of the associated linear matrix inequality (LMI) problem can be reduced. For naturally distributed applications, such as multiagent systems, the construction and tuning of a centralized observer may not be feasible. Therefore, we also extend the cascaded approach to the observer design and use fuzzy observers to individually estimate the states of these subsystems. A theoretical proof of stability and simulation examples are presented. The results show that the distributed observer achieves the same performance as the centralized one, while leading to increased modularity, reduced complexity, lower computational costs, and easier tuning. Applications of such cascaded systems include multiagent systems, distributed process control, and hierarchical large-scale systems.

/pdf/stability-of-cascaded-fuzzy-systems-and-observers-4b0m9g686r.pdf

Stability of Cascaded Fuzzy Systems and Observers

A virtual sensor for normal acceleration has been developed and implemented in the flight control system of a small commercial aircraft. The inputs of the virtual sensor are the consolidated outputs of dissimilar sensor signals. The virtual sensor is a fuzzy model of the Takagi-Sugeno type and it has been identified from simulated data, using a detailed, realistic Matlab/Simulink/sup TM/ model used by the aircraft manufacturer. This virtual sensor can be applied to identify a failed sensor in the case that only two real sensors are available and even to detect a failure of the last available sensor.

Virtual sensor for fault detection and isolation in flight control systems - fuzzy modeling approach

Preface. Part I: The Position and State of the Art of Fuzzy Systems. 1. Fuzzy Systems in Control Engineering H.B. Verbruggen, P.M. Bruijn. 2. Fuzzy Logic, Control Engineering and Artificial Intelligence D. Dubois, et al. 3. Fuzzy Control Versus Conventional Control K.-E. Arzen, et al. 4. Data-Driven Construction of Transparent Fuzzy Models R. Babuska, M. Setnes. Part II: Design and Analysis Issues. 5. Fuzzy Logic Normal Forms for Control Law Representation I. Perfilieva. 6. Stability Analysis of Fuzzy Control Loops A. Ollero, et al. 7. Performance Criteria: Classical and Fuzzy Design J.M. Sousa, et al. 8. Complexity Reduction Methods for Fuzzy Systems M. Setnes, et al. Part III: Application of Fuzzy Systems. 9. Intelligent Data Analysis and Fuzzy Control H.-J. Zimmermann, et al. 10. Fuzzy Control in Process Industry E.K. Juuso. 11. Fuzzy Logic Applications in Mobile Robotics A. Ollero, et al. 12. Enhancing Flight Control using Fuzzy Logic G. Schram, et al. References. Index.

Robert Babuska

Papers

Learning state representation for deep actor-critic control

Fuzzy modeling with multivariate membership functions: gray-box identification and control design

Stability of Cascaded Fuzzy Systems and Observers

Virtual sensor for fault detection and isolation in flight control systems - fuzzy modeling approach

Fuzzy algorithms for control