Showing papers by "Robert Babuska published in 2017"

Residential Demand Response of Thermostatically Controlled Loads Using Batch Reinforcement Learning

Abstract: Driven by recent advances in batch Reinforcement Learning (RL), this paper contributes to the application of batch RL to demand response. In contrast to conventional model-based approaches, batch RL techniques do not require a system identification step, making them more suitable for a large-scale implementation. This paper extends fitted Q-iteration, a standard batch RL technique, to the situation when a forecast of the exogenous data is provided. In general, batch RL techniques do not rely on expert knowledge about the system dynamics or the solution. However, if some expert knowledge is provided, it can be incorporated by using the proposed policy adjustment method. Finally, we tackle the challenge of finding an open-loop schedule required to participate in the day-ahead market. We propose a model-free Monte Carlo method that uses a metric based on the state-action value function or Q-function and we illustrate this method by finding the day-ahead schedule of a heat-pump thermostat. Our experiments show that batch RL techniques provide a valuable alternative to model-based controllers and that they can be used to construct both closed-loop and open-loop policies.

Railway Track Circuit Fault Diagnosis Using Recurrent Neural Networks

Abstract: Timely detection and identification of faults in railway track circuits are crucial for the safety and availability of railway networks. In this paper, the use of the long-short-term memory (LSTM) recurrent neural network is proposed to accomplish these tasks based on the commonly available measurement signals. By considering the signals from multiple track circuits in a geographic area, faults are diagnosed from their spatial and temporal dependences. A generative model is used to show that the LSTM network can learn these dependences directly from the data. The network correctly classifies 99.7% of the test input sequences, with no false positive fault detections. In addition, the t-Distributed Stochastic Neighbor Embedding (t-SNE) method is used to examine the resulting network, further showing that it has learned the relevant dependences in the data. Finally, we compare our LSTM network with a convolutional network trained on the same task. From this comparison, we conclude that the LSTM network architecture is better suited for the railway track circuit fault detection and identification tasks than the convolutional network.

A big data analysis approach for rail failure risk assessment

Abstract: Railway infrastructure monitoring is a vital task to ensure rail transportation safety A rail failure could result in not only a considerable impact on train delays and maintenance costs, but also on safety of passengers In this article, the aim is to assess the risk of a rail failure by analyzing a type of rail surface defect called squats that are detected automatically among the huge number of records from video cameras We propose an image processing approach for automatic detection of squats, especially severe types that are prone to rail breaks We measure the visual length of the squats and use them to model the failure risk For the assessment of the rail failure risk, we estimate the probability of rail failure based on the growth of squats Moreover, we perform severity and crack growth analyses to consider the impact of rail traffic loads on defects in three different growth scenarios The failure risk estimations are provided for several samples of squats with different crack growth lengths on a busy rail track of the Dutch railway network The results illustrate the practicality and efficiency of the proposed approach

Automated tuning and configuration of path planning algorithms

Abstract: A large number of novel path planning methods for a wide range of problems have been described in literature over the past few decades. These algorithms can often be configured using a set of parameters that greatly influence their performance. In a typical use case, these parameters are only very slightly tuned or even left untouched. Systematic approaches to tune parameters of path planning algorithms have been largely unexplored. At the same time, there is a rising interest in the planning and robotics communities regarding the real world application of these theoretically developed and simulation-tested planning algorithms. In this work, we propose the use of Sequential Model-based Algorithm Configuration (SMAC) tools to address these concerns. We show that it is possible to improve the performance of a planning algorithm for a specific problem without the need of in-depth knowledge of the algorithm itself. We compare five planners that see a lot of practical usage on three typical industrial pick-and-place tasks to demonstrate the effectiveness of the method.

Enhanced Symbolic Regression Through Local Variable Transformations

Abstract: Genetic programming (GP) is a technique widely used in a range of symbolic regression problems, in particular when there is no prior knowledge about the symbolic function sought. In this paper, we present a GP extension introducing a new concept of local transformed variables, based on a locally applied affine transformation of the original variables. This approach facilitates finding accurate parsimonious models. We have evaluated the proposed extension in the context of the Single Node Genetic Programming (SNGP) algorithm on synthetic as well as real-problem datasets. The results confirm our hypothesis that the transformed variables significantly improve the performance of the standard SNGP algorithm.

Model-based real-time control of a magnetic manipulator system

Abstract: Precise magnetic manipulation has numerous applications, ranging from manufacturing to the medical field. Owing to the nonlinear nature of the electromagnetic force, magnetic manipulation requires advanced nonlinear control. In this paper, we design and experimentally evaluate two nonlinear controllers for a magnetic manipulation (Magman) system, which consists of four electromagnetic coils arranged linearly. The current through the coils is controlled in order to accurately position a steel ball, rolling freely in a track above the coils. We benchmark two nonlinear control methods, namely feedback linearization and a constrained state-dependent Riccati equation (SDRE) control. These methods are chosen due to their widespread use in academia as well as industrial applications. On the actual setup, constrained SDRE has performed considerably better in terms of the settling time, overshoot, and the amount of control effort when compared to feedback linearization.