Book ChapterDOI
Learning Complex Behaviors via Sequential Composition and Passivity-Based Control
Gabriel A. D. Lopes,Esmaeil Najafi,Subramanya Nageshrao,Robert Babuska +3 more
- pp 53-74
TLDR
This paper addresses practical implementations of RL by interfacing elements of systems and control and robotics by using sequential composition and passivity-based control methods towards speeding up learning and providing a stopping time criteria.Abstract:Â
The model-free paradigm of Reinforcement learning (RL) is a theoretical strength. However in practice, the stringent assumptions required for optimal solutions (full state space exploration) and experimental issues, such as slow learning rates, render model-free RL a practical weakness. This paper addresses practical implementations of RL by interfacing elements of systems and control and robotics. In our approach space is handled by Sequential Composition (a technique commonly used in robotics) and time is handled by the use of passivity-based control methods (a standard nonlinear control approach) towards speeding up learning and providing a stopping time criteria. Sequential composition in effect partitions the state space and allows for the composition of controllers, each having different domains of attraction (DoA) and goal sets. This results in learning taking place in subsets of the state space. Passivity-based control (PBC) is a model-based control approach where total energy is computable. This total energy can be used as a candidate Lyapunov function to evaluate the stability of a controller and find estimates of its DoA. This enables learning in finite time: while learning the candidate Lyapunov function is monitored online to approximate the DoA of the learned controller. Once this DoA covers relevant states, from the point of view of sequential composition, the learning process is stopped. The result of this process is a collection of learned controllers that cover a desired range of the state space, and can be composed in sequence to achieve various desired goals. Optimality is lost in favour of practicality. Other implications include safety while learning and incremental learning.read more
Citations
More filters
Journal ArticleDOI
A fast sampling method for estimating the domain of attraction
TL;DR: In this paper, a sampling approach is proposed to estimate the domain of attraction (DoA) of nonlinear systems in real time, which is validated to approximate the DoAs of stable equilibria.
Journal ArticleDOI
Robot Contact Language for Manipulation Planning
TL;DR: A contact-based language for robotic manipulation and planning, based on the contact between a collection of objects, robots, and surfaces, is proposed, which considers making and breaking contact as the bridge between high-level planning and low-level controllers.
Journal ArticleDOI
Learning Sequential Composition Control
TL;DR: A learning approach to augment the standard sequential composition framework by using online learning to handle unforeseen situations and the results show that in both cases a new controller can be rapidly learned and added to the supervisory control structure.
Proceedings ArticleDOI
Towards cooperative sequential composition control
TL;DR: The standard sequential composition is extended by introducing a novel approach to compose multiple sequential composition controllers towards cooperative control, which can fulfill the tasks which are not possible to satisfy with the original controllers individually.
Proceedings ArticleDOI
ROS-based SLAM and Navigation for a Gazebo-Simulated Autonomous Quadrotor
TL;DR: In this paper, a robotic operating system based on autonomous simultaneous localization and mapping (SLAM), and robot navigation implementation of a Parrot AR.2.0 quadrotor, which is equipped with a laser scanner and inertial measurement unit, is presented.
References
More filters
Book
Introduction to Graph Theory
TL;DR: In this article, the authors introduce the concept of graph coloring and propose a graph coloring algorithm based on the Eulers formula for k-chromatic graphs, which can be seen as a special case of the graph coloring problem.
Book
Introduction to Reinforcement Learning
TL;DR: In Reinforcement Learning, Richard Sutton and Andrew Barto provide a clear and simple account of the key ideas and algorithms of reinforcement learning.
Book
Nonlinear Systems Analysis
TL;DR: In this article, the authors consider non-linear differential equations with unique solutions, and prove the Kalman-Yacubovitch Lemma and the Frobenius Theorem.
DissertationDOI
Structured semidefinite programs and semialgebraic geometry methods in robustness and optimization
TL;DR: In this paper, the authors introduce a specific class of linear matrix inequalities (LMI) whose optimal solution can be characterized exactly, i.e., the optimal value equals the spectral radius of the operator.
Journal ArticleDOI
Interconnection and damping assignment passivity-based control of port-controlled Hamiltonian systems
TL;DR: A new PBC theory is developed which extends to a broader class of systems the aforementioned energy-balancing stabilization mechanism and the structure invariance and considers instead port-controlled Hamiltonian models, which result from the network modelling of energy-conserving lumped-parameter physical systems with independent storage elements, and strictly contain the class of EL models.