This article proposes a betterment process for the operation of a mechanical robot in a sense that it betters the next operation of a robot by using the previous operation's data. The process has an iterative learning structure such that the (k + 1)th input to joint actuators consists of the kth input plus an error increment composed of the derivative difference between the kth motion trajectory and the given desired motion trajectory. The convergence of the process to the desired motion trajectory is assured under some reasonable conditions. Numerical results by computer simulation are presented to show the effectiveness of the proposed learning scheme.

Bettering Operation of Robots by Learning

This article surveyed the major results in iterative learning control (ILC) analysis and design over the past two decades. Problems in stability, performance, learning transient behavior, and robustness were discussed along with four design techniques that have emerged as among the most popular. The content of this survey was selected to provide the reader with a broad perspective of the important ideas, potential, and limitations of ILC. Indeed, the maturing field of ILC includes many results and learning algorithms beyond the scope of this survey. Though beginning its third decade of active research, the field of ILC shows no sign of slowing down.

A survey of iterative learning control

Reinforcement learning offers to robotics a framework and set of tools for the design of sophisticated and hard-to-engineer behaviors. Conversely, the challenges of robotic problems provide both inspiration, impact, and validation for developments in reinforcement learning. The relationship between disciplines has sufficient promise to be likened to that between physics and mathematics. In this article, we attempt to strengthen the links between the two research communities by providing a survey of work in reinforcement learning for behavior generation in robots. We highlight both key challenges in robot reinforcement learning as well as notable successes. We discuss how contributions tamed the complexity of the domain and study the role of algorithms, representations, and prior knowledge in achieving these successes. As a result, a particular focus of our paper lies on the choice between model-based and model-free as well as between value-function-based and policy-search methods. By analyzing a simple problem in some detail we demonstrate how reinforcement learning approaches may be profitably applied, and we note throughout open questions and the tremendous potential for future research.

/pdf/reinforcement-learning-in-robotics-a-survey-4w4397qx9i.pdf

Reinforcement learning in robotics: A survey

A direct adaptive tracking control architecture is proposed and evaluated for a class of continuous-time nonlinear dynamic systems for which an explicit linear parameterization of the uncertainty in the dynamics is either unknown or impossible. The architecture uses a network of Gaussian radial basis functions to adaptively compensate for the plant nonlinearities. Under mild assumptions about the degree of smoothness exhibit by the nonlinear functions, the algorithm is proven to be globally stable, with tracking errors converging to a neighborhood of zero. A constructive procedure is detailed, which directly translates the assumed smoothness properties of the nonlinearities involved into a specification of the network required to represent the plant to a chosen degree of accuracy. A stable weight adjustment mechanism is determined using Lyapunov theory. The network construction and performance of the resulting controller are illustrated through simulations with example systems. >

Gaussian networks for direct adaptive control

A new adaptive robot control algorithm is derived, which consists of a PD feedback part and a full dynamics feedfor ward compensation part, with the unknown manipulator and payload parameters being estimated online. The algorithm is computationally simple, because of an effective exploitation of the structure of manipulator dynamics. In particular, it requires neither feedback of joint accelerations nor inversion of the estimated inertia matrix. The algorithm can also be applied directly in Cartesian space.

On the adaptive control of robot manipulators

A new concept called "betterment process" is proposed for the purpose of giving a learning ability of autonomous construction of a better control input to a class of multi-input multi-output servomechanism or mechatronics systems such as mechanical robots. It is assumed that those dynamic systems can be operated repeatedly at low cost and in a relatively short time under invariant initial physical conditions, but the knowledge on precise description of their dynamics is not required. The betterment process is composed of a simple iteration rule that generates autonomously a present actuator input better than the previous one, provided a desired output response is given. The convergence of iteration is proved for a simple betterment process where the k+1th input is composed of the kth input plus an increment of the derivative error between the kth output response and given desired response. Discussions on potential applications of the proposed theory to controlling robots or other mechanical systems are presented together with future subjects to be investigated.

Bettering operation of dynamic systems by learning: A new control theory for servomechanism or mechatronics systems

A new type of a parallel wire-driven robot is proposed in order to reach ultra-high speed. The driving principle of parallel wire systems is described. Since wires can only pull and not push on an object, at least np1 wires are needed in order to move the object in a n-dimensional space. In this paper, taking account of the effect of such redundancy on actuation, the motion stability in wire length coordinates is analyzed by using a Lyapunov function. Using “Vector Closure”, it is proven that the hand position and orientation converge to the corresponding desired values and the internal force also converges to the desired one. Moreover, by making good use of non-linear elasticity of parallel wire driven robots, it is claimed that the internal force arising from redundant actuation can effectively reduce vibration when the high-speed robot stops at desired points. As a result, ultra-high speed with more than 40 g(g:gravitational acceleration) can be attained by using relatively small actuators.

https://www.cambridge.org/core/services/aop-cambridge-core/content/view/0A052A81B94B9430D6DEEA32443CF0C8/S0263574799002477a.pdf/div-class-title-high-speed-manipulation-by-using-parallel-wire-driven-robots-div.pdf

High-speed manipulation by using parallel wire-driven robots

High speed robots are an important component of modern assembly operations. In this paper, we describe the development of an ultrahigh speed robot named FALCON (Fast Load Conveyance), based on a wire driven parallel manipulation system. It achieves peak accelerations of up to 43G and maximum velocities of 13 m/s, even if considerably small DC motors (60W) are used. Due to the use of wires in actuation, the problem of vibration arises. We employ internal force control among wires to reduce the vibration by utilizing nonlinear elasticity from the wire mechanism. Experimental results on point to point control using the prototype system are presented under linear PD feedback, to illustrate the effectiveness of the system.

Development of an ultrahigh speed robot FALCON using wire drive system

To make a robot track a given desired motion trajectory, a learning control scheme is proposed which is based on the repeatability of robot motion. In this scheme the robot obtains a desired motion by repeating trials (test motion). A merit of this control scheme is that the input torque pattern that generates the desired motion can be formed without estimating the physical parameters of robot dynamics. In practice, to allow the robot motion to approach the desired one in each trial, the input torque given to the robot at the present trial is modified only by the velocity signal of the real robot motion at the previous trial and the desired one. The convergence to the desired motion is theoretically proved for a linear time-varying mechanical system, which is an approximate representation of nonlinear robot dynamics in the vicinity of the desired motion. The effectiveness of this control scheme is demonstrated through actual experiments in which a revolute-type manipulator with three degrees of freedom is used, and the desired motion trajectory is given not only in terms of joint-angle coordinates but also in terms of task-oriented coordinates. >

Sadao Kawamura

Papers

Bettering Operation of Robots by Learning

Bettering operation of dynamic systems by learning: A new control theory for servomechanism or mechatronics systems

High-speed manipulation by using parallel wire-driven robots

Development of an ultrahigh speed robot FALCON using wire drive system

Realization of robot motion based on a learning method