For solving the singularity problem arising in the control of manipulators, an efficient way is to maximize its manipulability. However, it is challenging to optimize manipulability effectively because it is a nonconvex function to the joint angles of a robotic arm. In addition, the involvement of an inversion operation in the expression of manipulability makes it even hard for timely optimization due to the intensively computational burden for matrix inversion. In this paper, we make progress on real-time manipulability optimization by establishing a dynamic neural network for recurrent calculation of manipulability-maximal control actions for redundant manipulators under physical constraints in an inverse-free manner. By expressing position tracking and matrix inversion as equality constraints, physical limits as inequality constraints, and velocity-level manipulability measure, which is affine to the joint velocities, as the objective function, the manipulability optimization scheme is further formulated as a constrained quadratic program. Then, a dynamic neural network with rigorously provable convergence is constructed to solve such a problem online. Computer simulations are conducted and show that, compared to the existing methods, the proposed scheme can raise the manipulability almost 40% on average, which substantiates the efficacy, accuracy, and superiority of the proposed manipulability optimization scheme.

Manipulability Optimization of Redundant Manipulators Using Dynamic Neural Networks

The problem of dynamic task allocation in a distributed network of redundant robot manipulators for path-tracking with limited communications is investigated in this paper, where ${k}$ fittest ones in a group of ${n}$ redundant robot manipulators with ${n>k}$ are allocated to execute an object tracking task. The problem is essentially challenging in view of the interplay of manipulator kinematics and the dynamic competition for activation among manipulators. To handle such an intricate problem, a distributed coordination control law is developed for the dynamic task allocation among multiple redundant robot manipulators with limited communications and with the aid of a consensus filter. In addition, a theorem and its proof are presented for guaranteeing the convergence and stability of the proposed distributed control law. Finally, an illustrative example is provided and analyzed to substantiate the efficacy of the proposed control law.

Distributed Task Allocation of Multiple Robots: A Control Perspective

In this paper, a neural-dynamic distributed scheme is proposed for the cooperative control of multiple redundant manipulators with limited communications. It is guaranteed that, with the communication network being connected, all manipulators can jointly reach the same desired motion. The proposed distributed scheme is rearranged as a time-varying quadratic program and solved online by a Zhang neural network. Then, theoretical analyses show that, without noise, the proposed distributed scheme is able to execute a given task with exponentially convergent position errors. Moreover, an explicit bound relationship between the control input noise and the end-effector position error is analytically derived. Furthermore, numerical comparisons substantiate the superiority, effectiveness, and accuracy of the proposed distributed scheme.

Neural Dynamics for Cooperative Control of Redundant Robot Manipulators

In this paper, a distributed scheme is proposed for the cooperative motion generation in a distributed network of multiple redundant manipulators. The proposed scheme can simultaneously achieve the specified primary task to reach global cooperation under limited communications among manipulators and optimality in terms of a specified optimization index of redundant robot manipulators. The proposed distributed scheme is reformulated as a quadratic program (QP). To inherently suppress noises originating from communication interferences or computational errors, a noise-tolerant zeroing neural network (NTZNN) is constructed to solve the QP problem online. Then, theoretical analyses show that, without noise, the proposed distributed scheme is able to execute a given task with exponentially convergent position errors. Moreover, in the presence of noise, the proposed distributed scheme with the aid of NTZNN model has a satisfactory performance. Furthermore, simulations and comparisons based on PUMA560 redundant robot manipulators substantiate the effectiveness and accuracy of the proposed distributed scheme with the aid of NTZNN model.

Cooperative Motion Generation in a Distributed Network of Redundant Robot Manipulators With Noises

Dynamic task allocation in multi-robot coordination for moving target tracking: A distributed approach

We consider the problem of source seeking using a group of mobile robots equipped with sensors for source concentration measurement. In the formulation, the robot team cooperatively estimates the gradient of the source field, moves to the source by tracing the gradient-ascending direction, and keeps a predefined formation in movement. We present two control algorithms with all-to-all and limited communications, respectively. For the case of all-to-all communication, rigorous analytic analysis proves that the formation center of the robots converges to the source in the presence of estimation errors with a bounded error, the upper bound of which is explicitly given. In the case of limited communication where centralized quantities are not available, distributed consensus filters are used to distributively estimate the centralized quantities, and then embedded in the distributed control laws. Numerical simulations are given to validate the effectiveness of the proposed approaches. Experimental results on the E-puck robot platform demonstrate satisfactory performances in a light source seeking application.

Cooperative Distributed Source Seeking by Multiple Robots: Algorithms and Experiments

Deep reinforcement learning yields great results for a large array of problems, but models are generally retrained anew for each new problem to be solved Prior learning and knowledge are difficult to incorporate when training new models, requiring increasingly longer training as problems become more complex This is especially problematic for problems with sparse rewards We provide a solution to these problems by introducing Concept Network Reinforcement Learning (CNRL), a framework which allows us to decompose problems using a multi-level hierarchy Concepts in a concept network are reusable, and flexible enough to encapsulate feature extractors, skills, or other concept networks With this hierarchical learning approach, deep reinforcement learning can be used to solve complex tasks in a modular way, through problem decomposition We demonstrate the strength of CNRL by training a model to grasp a rectangular prism and precisely stack it on top of a cube using a gripper on a Kinova JACO arm, simulated in MuJoCo Our experiments show that our use of hierarchy results in a 45x reduction in environment interactions compared to the state-of-the-art on this task

Deep Reinforcement Learning for Dexterous Manipulation with Concept Networks.

The multiple independent processes run in an AI engine on its cloud-based platform. The multiple independent processes are configured as an independent process wrapped in its own container so that multiple instances of the same processes can run simultaneously to scale to handle one or more users to perform actions. The actions to solve AI problems can include 1) running multiple training sessions on two or more AI models at the same time, 2) creating two or more AI models at the same time, 3) running a training session on one or more AI models while creating one or more AI models at the same time, 4) deploying and using two or more trained AI models to do predictions on data from one or more data sources, 5) etc. A service handles scaling by dynamically calling in additional computing devices to load on and run additional instances of one or more of the independent processes as needed.

Artificial intelligence engine having multiple independent processes on a cloud based platform configured to scale

Methods and apparatuses that apply a hierarchical-decomposition reinforcement learning technique to train one or more AI objects as concept nodes composed in a hierarchical graph incorporated into an AI model. The individual sub-tasks of a decomposed task may correspond to its own concept node in the hierarchical graph and are initially trained on how to complete their individual sub-task and then trained on how the all of the individual sub-tasks need to interact with each other in the complex task in order to deliver an end solution to the complex task. Next, during the training, using reward functions focused for solving each individual sub-task and then a separate one or more reward functions focused for solving the end solution of the complex task. In addition, where reasonably possible, conducting the training of the AI objects corresponding to the individual sub-tasks in the complex task, in parallel at the same time.

For hiearchical decomposition deep reinforcement learning for an artificial intelligence model

The AI engine has a first module that chooses from a library of algorithms to use when automatically assembling and building different learning topologies to solve different concepts making up a resulting AI model. The AI engine may integrate both i) one or more dynamic programming training algorithms and ii) one or more policy optimization algorithms, to build the different learning topologies to solve the different concepts contained with an AI model in order to solve a wide variety of problem types. Each concept contained in the AI model can use a most appropriate approach for achieving a mission of that concept. A learning topology representing a first concept can be built by the first module with a first dynamic programming training algorithm, while a learning topology representing a second concept in the same AI model can be built by the first module with a first policy optimization algorithm.

Ruofan Kong

Papers

Cooperative Distributed Source Seeking by Multiple Robots: Algorithms and Experiments

Deep Reinforcement Learning for Dexterous Manipulation with Concept Networks.

Artificial intelligence engine having multiple independent processes on a cloud based platform configured to scale

For hiearchical decomposition deep reinforcement learning for an artificial intelligence model

Artificial Intelligence Engine Having Various Algorithms to Build Different Concepts Contained Within a Same AI Model