Robert Babuska

Bio: Robert Babuska is an academic researcher from Delft University of Technology. The author has contributed to research in topics: Fuzzy logic & Reinforcement learning. The author has an hindex of 56, co-authored 371 publications receiving 15388 citations. Previous affiliations of Robert Babuska include Carnegie Mellon University & Czech Technical University in Prague.

Optimization-based feedforward control for a Drop-on-Demand inkjet printhead

Abstract: The printing quality delivered by a Drop-on-Demand (DoD) inkjet printhead is limited due to operational issues such as residual oscillations in the ink channel and the cross-talk between the ink channels. The maximal jetting frequency of a DoD inkjet printhead can be increased by quickly damping the residual oscillations and by bringing in this way the ink channel to rest after jetting the ink drop. This paper proposes an optimization-based method to design the input actuation waveform for the piezo actuator in order to improve the damping of the residual oscillations. The narrow-gap model is used to predict the response of the ink channel under the application of the piezo input. Simulation and experimental results are presented to show the applicability of the proposed method.

Visual Navigation in Real-World Indoor Environments Using End-to-End Deep Reinforcement Learning

Abstract: Visual navigation is essential for many applications in robotics, from manipulation, through mobile robotics to automated driving. Deep reinforcement learning (DRL) provides an elegant map-free approach integrating image processing, localization, and planning in one module, which can be trained and therefore optimized for a given environment. However, to date, DRL-based visual navigation was validated exclusively in simulation, where the simulator provides information that is not available in the real world, e.g., the robot's position or image segmentation masks. This precludes the use of the learned policy on a real robot. Therefore, we propose a novel approach that enables a direct deployment of the trained policy on real robots. We have designed visual auxiliary tasks, a tailored reward scheme, and a new powerful simulator to facilitate domain randomization. The policy is fine-tuned on images collected from real-world environments. We have evaluated the method on a mobile robot in a real office environment. The training took ~30 hours on a single GPU. In 30 navigation experiments, the robot reached a 0.3-meter neighborhood of the goal in more than 86.7% of cases. This result makes the proposed method directly applicable to tasks like mobile manipulation.

Takagi-Sugeno fuzzy observer and extended-Kalman filter for adaptive payload estimation

Abstract: In this paper, two nonlinear state estimation methods, Takagi-Sugeno fuzzy observer and extended-Kalman filter are compared in terms of their ability to reliably estimate the velocity and an unknown, variable payload of a nonlinear servo system. Using the system dynamics and a position measurement, the velocity and unknown payload are estimated. In a simulation study, the servo system is excited with a randomly generated step input. In real-time experiments, the estimation is performed under feedback-linearizing control. The performance of the TS fuzzy payload estimator is discussed with respect to the choice of the desired convergence rate. The application results show that the Takagi-Sugeno fuzzy observer provides better performance than the extended-Kalman filter with robust and less parameter dependent structure.

Fuzzy Clustering Algorithms

Abstract: An effective approach to the identification of complex nonlinear systems is to partition the available data into subsets and approximate each subset by a simple model. Fuzzy clustering can be used as a tool to obtain a partitioning of data where the transitions between the subsets are gradual rather than abrupt. This chapter gives an introduction to the basic concepts of fuzzy clustering, and simultaneously serves as a reference to clustering algorithms that can be used to construct fuzzy models from data. The basic notions of clustering and the different types of partitions are defined in Sections 3.1 and 3.2. Section 3.3 presents the basic idea of fuzzy clustering with objective function and the fuzzy c-means algorithm. Sections 3.4 and 3.5 address algorithms which can detect clusters contained in linear subspaces of the data space. These methods include clustering with an adaptive distance measure, clustering with linear prototypes, and fuzzy regression clustering. Section 3.6 presents the approach known as possibilistic clustering. Section 3.7 addresses the determination of an appropriate number of clusters and Section 3.8 deals with pre-processing of the data. The aim of this chapter is to explain clustering at a level necessary to understand the subsequent chapters. For a more detailed treatment of the subject, the reader may refer to the classical monographs by Duda and Hart (1973), Bezdek (1981) and Jain and Dubes (1988). A more recent overview can be found in a collection of Bezdek and Pal (1992), and the monograph by Backer (1995). The notation and terminology in this chapter closely follows Bezdek (1981).

Machine learning

Abstract: Machine Learning is the study of methods for programming computers to learn. Computers are applied to a wide range of tasks, and for most of these it is relatively easy for programmers to design and implement the necessary software. However, there are many tasks for which this is difficult or impossible. These can be divided into four general categories. First, there are problems for which there exist no human experts. For example, in modern automated manufacturing facilities, there is a need to predict machine failures before they occur by analyzing sensor readings. Because the machines are new, there are no human experts who can be interviewed by a programmer to provide the knowledge necessary to build a computer system. A machine learning system can study recorded data and subsequent machine failures and learn prediction rules. Second, there are problems where human experts exist, but where they are unable to explain their expertise. This is the case in many perceptual tasks, such as speech recognition, hand-writing recognition, and natural language understanding. Virtually all humans exhibit expert-level abilities on these tasks, but none of them can describe the detailed steps that they follow as they perform them. Fortunately, humans can provide machines with examples of the inputs and correct outputs for these tasks, so machine learning algorithms can learn to map the inputs to the outputs. Third, there are problems where phenomena are changing rapidly. In finance, for example, people would like to predict the future behavior of the stock market, of consumer purchases, or of exchange rates. These behaviors change frequently, so that even if a programmer could construct a good predictive computer program, it would need to be rewritten frequently. A learning program can relieve the programmer of this burden by constantly modifying and tuning a set of learned prediction rules. Fourth, there are applications that need to be customized for each computer user separately. Consider, for example, a program to filter unwanted electronic mail messages. Different users will need different filters. It is unreasonable to expect each user to program his or her own rules, and it is infeasible to provide every user with a software engineer to keep the rules up-to-date. A machine learning system can learn which mail messages the user rejects and maintain the filtering rules automatically. Machine learning addresses many of the same research questions as the fields of statistics, data mining, and psychology, but with differences of emphasis. Statistics focuses on understanding the phenomena that have generated the data, often with the goal of testing different hypotheses about those phenomena. Data mining seeks to find patterns in the data that are understandable by people. Psychological studies of human learning aspire to understand the mechanisms underlying the various learning behaviors exhibited by people (concept learning, skill acquisition, strategy change, etc.).

The Ensemble Kalman Filter: Theoretical formulation and practical implementation

Abstract: The purpose of this paper is to provide a comprehensive presentation and interpretation of the Ensemble Kalman Filter (EnKF) and its numerical implementation. The EnKF has a large user group, and numerous publications have discussed applications and theoretical aspects of it. This paper reviews the important results from these studies and also presents new ideas and alternative interpretations which further explain the success of the EnKF. In addition to providing the theoretical framework needed for using the EnKF, there is also a focus on the algorithmic formulation and optimal numerical implementation. A program listing is given for some of the key subroutines. The paper also touches upon specific issues such as the use of nonlinear measurements, in situ profiles of temperature and salinity, and data which are available with high frequency in time. An ensemble based optimal interpolation (EnOI) scheme is presented as a cost-effective approach which may serve as an alternative to the EnKF in some applications. A fairly extensive discussion is devoted to the use of time correlated model errors and the estimation of model bias.

Reinforcement learning in robotics: A survey

Abstract: Reinforcement learning offers to robotics a framework and set of tools for the design of sophisticated and hard-to-engineer behaviors. Conversely, the challenges of robotic problems provide both inspiration, impact, and validation for developments in reinforcement learning. The relationship between disciplines has sufficient promise to be likened to that between physics and mathematics. In this article, we attempt to strengthen the links between the two research communities by providing a survey of work in reinforcement learning for behavior generation in robots. We highlight both key challenges in robot reinforcement learning as well as notable successes. We discuss how contributions tamed the complexity of the domain and study the role of algorithms, representations, and prior knowledge in achieving these successes. As a result, a particular focus of our paper lies on the choice between model-based and model-free as well as between value-function-based and policy-search methods. By analyzing a simple problem in some detail we demonstrate how reinforcement learning approaches may be profitably applied, and we note throughout open questions and the tremendous potential for future research.