Showing papers by "Robert Babuska published in 2005"

Multiagent Reinforcement Learning with Adaptive State Focus.

Abstract: In realistic multiagent systems, learning on the basis of complete state information is not feasible We introduce adaptive state focus Q-learning, a class of methods derived from Q-learning that start learning with only the state information that is strictly necessary for a single agent to perform the task, and that monitor the convergence of learning If lack of convergence is detected, the learner dynamically expands its state space to incorporate more state information (eg, states of other agents) Learning is faster and takes less resources than if the complete state were considered from the start, while being able to handle situations where agents interfere in pursuing their goals We illustrate our approach by instantiating a simple version of such a method, and by showing that it outperforms learning with full state information without being hindered by the deficiencies of learning on the basis of a single agent’s state

Continuous-state reinforcement learning with fuzzy approximation

Abstract: Reinforcement learning (RL) is a widely used learning paradigm for adaptive agents. There exist several convergent and consistent RL algorithms which have been intensively studied. In their original form, these algorithms require that the environment states and agent actions take values in a relatively small discrete set. Fuzzy representations for approximate, model-free RL have been proposed in the literature for the more difficult case where the state-action space is continuous. In this work, we propose a fuzzy approximation architecture similar to those previously used for Q-learning, but we combine it with the model-based Q-value iteration algorithm. We prove that the resulting algorithm converges. We also give a modified, asynchronous variant of the algorithm that converges at least as fast as the original version. An illustrative simulation example is provided.

Automated design environment for serial industrial manipulators

Abstract: – Resulting from the need for fast and insightful modeling combined with the drawbacks of available modeling environments, provides details of work developed on an automated modelling environment., – An automated modeling environment for serial manipulators has been implemented in Matlab/Simulink., – The manipulator configuration is defined by using a graphical user interface and the corresponding mathematical model is automatically generated. The model is exported to Matlab for analysis and control design, as well as to Simulink for simulation and verification purposes. Friction and stiction phenomena are included in the model. The simulation results can be visualized in standard plots and scopes as well as through virtual reality animations., – The modeling environment has been used in the design of a control system for a seven‐degree‐of‐freedom manipulator in a tunnel‐boring machine., – Information on the implementation of an automated modelling environment to facilitate the simultaneous design of the configuration and the corresponding control system of serial manipulators

An improved multiagent reinforcement learning algorithm

Abstract: An improved reinforcement learning algorithm is proposed in this paper. This algorithm is based on linear programming method for finding the best-response policy. A pursuit example is tested and the results show that this algorithm has some properties, such as easy computation, simple operation procedure and can guarantee a good learning convergence.

An overview of nonlinear identification and control with fuzzy systems

Abstract: This chapter gives an overview of system identification techniques for fuzzy models and some selected techniques for model-based fuzzy control. It starts with a brief discussion of the position of fuzzy modelling within the general nonlinear identification setting. The two most commonly used fuzzy models are the Mamdani model and the Takagi-Sugeno model. An overview of techniques for the data-driven construction of the latter model is given. We discuss both structure selection (input variables, representation of dynamics, number and type of membership functions) and parameter estimation (local and global estimation techniques, weighted least squares, multi-objective optimisation). Further, we discuss control design based on a fuzzy model of the process. As the model is assumed to be obtained through identification from sampled data, we focus on discrete-time methods, including: gain-scheduling and state-feedback design, model-inverse control and predictive control. A real-world application example is given.

Evaluation of adaptive fuzzy controllers: a real-world experiment

Abstract: Several stable adaptive fuzzy control schemes based on feedback linearization and Lyapunov synthesis were proposed in the literature. However, most of such controllers have been only tested on relatively simple simulations examples, in which the effects of noise, uncertainties, computational times and other fundamental problems related to hardware implementation are neglected. In this paper, we study and compare direct and indirect adaptive control schemes by means of an experimental benchmark (two coupled DC machines). For the indirect schemes, we consider both the standard adaptive laws based on tracking error and composite adaptive laws based on tracking and prediction error. The goal of this work is to gain insight in the benefits and drawbacks of the different variants of adaptive fuzzy controllers and to evaluate their potential for practical applications