Showing papers by "Robert Babuska published in 2007"

Fuzzy Approximation for Convergent Model-Based Reinforcement Learning

Abstract: Reinforcement learning (RL) is a learning control paradigm that provides well-understood algorithms with good convergence and consistency properties. Unfortunately, these algorithms require that process states and control actions take only discrete values. Approximate solutions using fuzzy representations have been proposed in the literature for the case when the states and possibly the actions are continuous. However, the link between these mainly heuristic solutions and the larger body of work on approximate RL, including convergence results, has not been made explicit. In this paper, we propose a fuzzy approximation structure for the Q-value iteration algorithm, and show that the resulting algorithm is convergent. The proof is based on an extension of previous results in approximate RL. We then propose a modified, serial version of the algorithm that is guaranteed to converge at least as fast as the original algorithm. An illustrative simulation example is also provided.

Stability of Cascaded Takagi-Sugeno Fuzzy Systems

Abstract: A large class of nonlinear systems can be well approximated by Takagi-Sugeno (TS) fuzzy models, with local models often chosen linear or affine. It is well-known that the stability of these local models does not ensure the stability of the overall fuzzy system. Therefore, several stability conditions have been developed for TS fuzzy systems. We study a special class of nonlinear dynamic systems, that can be decomposed into cascaded subsystems. These subsystems are represented as TS fuzzy models. We analyze the stability of the overall TS system based on the stability of the subsystems. For a general nonlinear, cascaded system, global asymptotic stability of the individual subsystems is not sufficient for the stability of the cascade. However, for the case of TS fuzzy systems, we prove that the stability of the subsystems implies the stability of the overall system. The main benefit of this approach is that it relaxes the conditions imposed when the system is globally analyzed, therefore solving some of the feasibility problems. Another benefit is, that by using this approach, the dimension of the associated linear matrix inequality (LMI) problem can be reduced. Applications of such cascaded systems include multi-agent systems, distributed process control and hierarchical large-scale systems.

Distributed Kalman filtering for multiagent systems

Abstract: For naturally distributed systems, such as multiagent systems, the construction and tuning of a centralized observer may be computationally expensive or even intractable An important class of distributed systems can be represented as cascaded subsystems For this class of systems, observers may be designed separately for the subsystems If the subsystems are linear, the Kalman filter provides an efficient means to estimate the states, so that it minimizes the mean squared estimation error Kalman-like filters may be used for the whole system or the individual subsystems In this paper, both a theoretical comparison and simulation examples are presented The theoretical results show that the distributed observers, except for special cases, do not minimize the overall error covariance, and so the distributed observer system is suboptimal However, in practice, the performance achieved by the cascaded observers is comparable and in certain cases outperforms that of the centralized one Moreover, a distributed observer system leads to increased modularity, reduced complexity, and lower computational costs