A practical neural network-based learning control system is described that is applicable to complex robotic systems involving multiple feedback sensors and multiple command variables. In the controller, one network is used to learn to reproduce the nonlinear relationship between the sensor outputs and the system command variables over particular regions of the system state space. The learned information is used to predict the command signals required to produce desired changes in the sensor outputs. A second network is used to learn to reproduce the nonlinear relationship between the system command variables and the changes in the video sensor outputs. The learned information from this network is then used to predict the next set of video parameters, effectively compensating for the image processing delays. The results of learning experiments using a General Electric P-5 manipulator are presented. These experiments involved control of the position and orientation of an object in the field of view of a video camera mounted on the end of the robot arm, using moving objects with arbitrary orientation relative to the robot. No a priori knowledge of the robot kinematics or of the object speed of orientation relative to the robot was assumed. Image parameter uncertainty and control system tracking error in the video image were found to converge to low values within a few trials. >

Real-time application of neural networks for sensor-based control of robots with vision

The visual information obtained from a camera and an image processing unit are incorporated in an adaptive control algorithm to make a robotic manipulator grasp a moving object. Because of the inherent time delay caused by the image processing, the motion of the moving target is predicted in real time and is used in the online planning of the trajectory for the manipulator motion. Since the dynamics of the target are assumed to be unknown, the prediction is accomplished by means of an autoregressive discrete-time model. On the basis of the predicted motion of the object, the planner determines online at each control sampling instant the desired trajectory point (subgoal) for the controller. The subgoal point is tracked by controlling the end effector with self-tuner until grasping occurs. A simulation study and laboratory experiments are presented to demonstrate the performance of this visual feedback system. >

Real-time vision feedback for servoing robotic manipulator with self-tuning controller

To make robotic manipulators function intelligently in applications such as flexible manufacturing, sensory feedback from an unknown environment is needed. It can help the robot to deal with unexpected events. Visual feedback represents a typical sensing system in which camera images provide feedback information, for example, for grasping a moving object with a robot manipulator. Because image processing is time-consuming, information about the target position cannot be obtained instantaneously for the controller. Because of inherent time delay, the present and future position has to be predicted in real-time. Since the dynamics of the target is assumed to be. unknown, the prediction will be accomplished by means of an auto-regressive discrete-time model. The predicted values and current end-effector position determine the desired trajectory point (subgoal) for the motion. The planner adapts on-line to changes in the target position. The desired trajectory is tracked by the end-effector, which is controlled by a self-hmr. A simulation study and lab experiments are presented to demonstrate the grasping of a moving target by a manipulator using a visual feedback.

CONTROL OF A ROBOTIC MANIPULATOR TOGRASP A MOVING TAFtGET USING VISION

Consideration is given to grasping a moving target by adaptively controlling a robot manipulator. The adaptive controller is capable to compensate for the system modeling errors and unknown payloads without deterioration of its performance. Because of the time needed to extract information from the computer images, a time delay is inherent in the calculated position of the object. This can be circumvented by predicting the position of the object so that the end effector of the manipulator can grasp it. Since the dynamics of the object are assumed to be unknown, a prediction is performed based on an autoregressive (AR) model, which is determined using the information extracted from the computer images. The information is then utilized in the controller so that the manipulator will grasp the moving object. The details of the prediction scheme are given. Control of the manipulator is reviewed. A simulation study on a manipulator for grasping a moving target using vision information and a self-tuning controller is described. The practicality of the approach is demonstrated by the laboratory experiments presented. >

Control of a robotic manipulator to grasp a moving target using vision

A vision-guided robot workstation is presented which picks up workpieces from a fast-moving conveyor belt. The role of computer vision as the feedback transducer strongly affects the closed-loop dynamics of the overall system, and a tracking controller with dynamic visual feedback is designed for achieving fast response and high control accuracy. In view of the long time delay and the heavy noise corruption embedded in visual data, the problem of visual controller design is posed in the framework of stochastic optimal control theory. The Kalman filter is chosen to estimate the state of the target motion and formulated as a joint detection and adaptive estimation method. The generalized predictive control strategy is utilized to compute the optimal path control data and implemented in a weighted version. Experimental results are given to show the effectiveness of the approach. >

Stochastic predictive control of robot tracking systems with dynamic visual feedback

A picture of a scene is used to extract information for an adaptive control algorithm. The object of interest is first located by means of a classifier. The position and orientation of the object are determined from a binary picture. The desired path that the gripper of the manipulator is to follow is specified by discrete points, first in the Cartesian and then in the joint coordinate system. The adaptive self-tuning controller is outlined. The controller design is based on a time series difference equation model in which the parameters are estimated on-line. The gains of the controller are tuned so that a quadratic performance criterion is minimized. The performance of the system designed is then tested experimentally, and the results are presented.

Experiments on manipulator gross motion using self-tuning controller and visual information

A picture of a scene is used to extract information for an adaptive control algorithm. The object of interest is first located by means of a classifier. The position and orientation of the object are determined from a binary picture. The desired path which the gripper of the manipulator is to follow is specified by discrete points, first is the Cartesian and then in the joint coordinate system. The adaptive self-tuning controller is outlined. The controller design is based on time series difference equation model, in which the parameters are estimated on-line. The gains of the controller are tuned so that a quadratic performance criteria is minimized. The performance of the system designed is then tested experimentally. Results of these experiments are presented.

Adaptive path control of a manipulator with visual information

To meet the expectation of the industry, resin transfer molding (RTM) has become one of the most promising polymer processing methods to manufacture fiber-reinforced plastics (FRPs) with light weight, high strength, and multifunctional features. The permeability and porosity of fiber reinforcements are two of the primary properties that control the flow of resin in fibers and are critical to numerical simulations of RTM. In the past, various permeability measurement methods have been developed in the literature. However, limitations still exist. Furthermore, porosity is often measured independently of permeability. As a result, the two measurements do not necessarily relate to the same entity, which may increase the time and labor costs associated with experiments and affect result interpretation. In this work, a measurement system was developed by fusing the signals from capacitive sensing and flow visualization, based on which a novel algorithm was developed. Without complicated sensor design or expensive instrumentation, both in-plane permeability and porosity can be simultaneously estimated. The feasibility of the proposed method was illustrated by experiments and verified with numerical simulations.

https://www.mdpi.com/2073-4360/14/13/2652/pdf?version=1656499280

T. H. Chiu

Papers

Experiments on manipulator gross motion using self-tuning controller and visual information

Adaptive path control of a manipulator with visual information

Sensor Fusion for Simultaneous Estimation of In-Plane Permeability and Porosity of Fiber Reinforcement in Resin Transfer Molding