Owing to its potential superiorities in terms of flexibility, compliance, and robustness, the musculoskeletal robotic system has become a promising direction for next-generation robots. However, motion learning and generalization of musculoskeletal systems are still challenging problems. In this article, a muscle-synergies-based neuromuscular control is proposed. First, a new computational model of time-varying muscle synergies is constructed, which utilizes both phasic and tonic muscle synergies to characterize the basic features of muscle excitations more sufficiently. Second, a novel neuromuscular control method is proposed for realizing the motion learning and generalization of musculoskeletal systems. Therein, a radial basis function (RBF) neural network is designed to modulate muscle synergies according to different movement targets. Muscle excitations are computed with the combination of modulated muscle synergies. Covariance matrix adaptation evolutionary strategy is applied to realize the synchronous optimization of muscle synergies and the RBF neural network. In the experiment, a sophisticated musculoskeletal system learns to perform center-out reaching tasks through trial-and-error learning on a few targets. With the muscle synergies and neural modulation learned from a few targets, the musculoskeletal system can also reach many unexperienced targets. The proposed method not only improves the speed and accuracy of motion learning but also enhances motion generalization. This article also promotes the development of the musculoskeletal robotic system and the fusion of neuroscience and robotics.

Muscle-Synergies-Based Neuromuscular Control for Motion Learning and Generalization of a Musculoskeletal System

Improving performance of robots using human-inspired approaches: a survey

Anti-interference analysis of bio-inspired musculoskeletal robotic system

Abstract Traditional joint-link robots have been widely used in production lines because of their high precision for single tasks. With the development of the manufacturing and service industries, the requirement for the comprehensive performance of robotics is growing. Numerous types of bio-inspired robotics have been investigated to realize human-like motion control and manipulation. A study route from inner mechanisms to external structures is proposed to imitate humans and animals better. With this idea, a brain-inspired intelligent robotic system is constructed that contains visual cognition, decision-making, motion control, and musculoskeletal structures. This paper reviews cutting-edge research in brain-inspired visual cognition, decision-making, motion control, and musculoskeletal systems. Two software systems and a corresponding hardware system are established, aiming at the verification and applications of next-generation brain-inspired musculoskeletal robots. 

/pdf/brain-inspired-intelligent-robotics-theoretical-analysis-and-2s6sr9q3.pdf

Brain-inspired Intelligent Robotics: Theoretical Analysis and Systematic Application

Musculoskeletal robot arm driven by pneumatic artificial muscle actuators is secure, lightweight and compliant, which makes it an ideal solution for prosthetics and rehabilitation equipment. However, there exists conflicts between anatomical accuracy and engineering feasibility, and the nonlinearity of muscle actuators brings difficulty in accurate mathematical modeling. To overcome these problems, in this paper, we propose an optimized musculoskeletal arm design of ten muscles and four degrees-offreedom, employ Deep Deterministic Policy Gradient (DDPG) to train a data driven controller, and combine the off-policy reinforcement learning algorithm with Hindsight Experience Replay (HER) to deal with sparse rewards situation. TaiChi motion sequence is acquired using Perception Neuron and the trajectory tracking experiments are carried out in simulation platform. Experiment results demonstrate that, the controlled musculoskeletal arm is able to track the TaiChi trajectory and learn well in sparse rewards situation.

Training Musculoskeletal Arm Play Taichi with Deep Reinforcement Learning

Redundant muscles in human-like musculoskeletal robotics bring extra dimensions to solution space, which makes the computation of muscle excitation become an open question. Conventional methods like dynamic optimization and reinforcement learning usually have a high computational cost or an unstable learning process when they are applied into a complex musculoskeletal system. In this paper, inspired by the learning process of human, we build a phased target learning framework which provides different targets to learners from different levels to guide their training process and avoid local optima. By introducing an extra layer of neurons with preference, we improved the Q-network method to generate continuous excitation. In addition, based on the process of information transmission in human nervous system, two kinds of biological noises are designed in our algorithm to enhance exploration ability of our method in solution space. Tracking experiments based on a simplified musculoskeletal arm model indicate that under the guiding of phased targets, our method avoids divergence of excitation and obtains a stable training process. Besides, with its enhanced ability of exploration, our method shows smaller motion errors during the motion. Importantly, the phased target learning framework can be expanded as a general reinforcement learning framework, and it is a preliminary interpretation for modeling human motion learning manners.

/pdf/from-rough-to-precise-human-inspired-phased-target-learning-5g8h4uflio.pdf

From Rough to Precise: Human-Inspired Phased Target Learning Framework for Redundant Musculoskeletal Systems.

Modeling, Characterization, and Application of Soft Bellows-Type Pneumatic Actuators for Bionic Locomotion

Robots with high-precision motion and operation ability are of great application significance. By referring to the biomechanical structure and neural control mechanism of human motion system, the research of musculoskeletal robot system with rigid-flexible coupling characteristics is one of the important ways to improve the operation flexibility and control robustness of robot. Inspired by the equilibrium point hypothesis proposed in neuroscience, this paper proposes a co-optimization algorithm of muscle arrangement and activation to construct constraint force field in the workspace of musculoskeletal robot. When the muscle arrangement is rough due to the insufficient precision of the mechanical structure, the musculoskeletal robot can maintain accurate motion with the help of constraint force field by adopting the optimized constant activation. Experiments are carried out on a musculoskeletal robot model with human-mimetic muscle to demonstrate the effectiveness of the proposed algorithm in movement accuracy, noise robustness and generalization. This work may be of great significance for the further introduction of constraint force field into hardware system of musculoskeletal robot.

Constructing Constraint Force Field in Musculoskeletal Robot by Co-optimizing Muscle Arrangements and Constant Activations

The wheel-legged robot combines the functions of wheeled vehicles and legged robots: high speed and high passability. However, the limited performance of existing joint actuators has always been the bottleneck in the actual applications of large wheel-legged robots. This paper proposed a highly integrated electro-hydrostatic actuator (EHA) to enable high-dynamic performance in giant wheel-legged robots (>200 kg). A prototype with a high force-to-weight ratio was developed by integrating a micropump, a miniature spring accumulator, and a micro-symmetrical cylinder. The prototype achieves a large output force of more than 9400 N and a high force-to-weight ratio of more than 2518 N/kg. Compared with existing EHA-based robots, it has a higher force-to-weight ratio and can bear larger loads. A detailed EHA model was presented, and controllers were designed based on sliding mode control and PID methods to control the output position and force of the piston. The model’s accuracy is improved by identifying uncertain parameters such as friction and leakage coefficient. Finally, both simulations and experiments were carried out. The results verified the fast response of force control (step response within 50 ms, the force tracking control frequency about 6.7 Hz) and the developed EHA’s good potential for future large wheel-legged robots.

/pdf/design-and-experiments-of-electro-hydrostatic-actuator-for-dy43hiz8.pdf

Design and Experiments of Electro-Hydrostatic Actuator for Wheel-Legged Robot with Fast Force Control Response

The invention belongs to the field of intelligent robots, particularly relates to a redundant musculoskeletal system-based staged motion control method and aims to solve the problem that a nerve stimulation signal is difficult to solve caused by high redundancy and high coupling. The system method comprises the steps of acquiring an expected motion state and a simplified motion state; calculatingan average allowance error; judging iterative number for skipping; updating a motion simplifying interval and the simplified motion state by a preset rule; obtaining a muscle stimulation signal and atendon force corresponding to each muscle by a Hill muscle model; acquiring an actual motion state by each tendon force, and calculating errors of the actual motion state and the simplified motion state; acquiring the actual motion state with noise; updating the nerve stimulation signal by a Q-network algorithm and preset execution noise; judging simulation time; and judging the iterative number,and outputting the iterative number. By the method, the solving difficulty of the nerve stimulation signal of a musculoskeletal system is reduced.

Junjie Zhou

Papers

From Rough to Precise: Human-Inspired Phased Target Learning Framework for Redundant Musculoskeletal Systems.

Modeling, Characterization, and Application of Soft Bellows-Type Pneumatic Actuators for Bionic Locomotion

Constructing Constraint Force Field in Musculoskeletal Robot by Co-optimizing Muscle Arrangements and Constant Activations

Design and Experiments of Electro-Hydrostatic Actuator for Wheel-Legged Robot with Fast Force Control Response

Redundant musculoskeletal system-based staged motion control method