We present a new open-source torque-controlled legged robot system, with a low-cost and low-complexity actuator module at its core. It consists of a high-torque brushless DC motor and a low-gear-ratio transmission suitable for impedance and force control. We also present a novel foot contact sensor suitable for legged locomotion with hard impacts. A 2.2 kg quadruped robot with a large range of motion is assembled from eight identical actuator modules and four lower legs with foot contact sensors. Leveraging standard plastic 3D printing and off-the-shelf parts results in a lightweight and inexpensive robot, allowing for rapid distribution and duplication within the research community. We systematically characterize the achieved impedance at the foot in both static and dynamic scenarios, and measure a maximum dimensionless leg stiffness of 10.8 without active damping, which is comparable to the leg stiffness of a running human. Finally, to demonstrate the capabilities of the quadruped, we present a novel controller which combines feedforward contact forces computed from a kino-dynamic optimizer with impedance control of the center of mass and base orientation. The controller can regulate complex motions while being robust to environmental uncertainty.

An Open Torque-Controlled Modular Robot Architecture for Legged Locomotion Research

Reinforcement Learning (RL) of contact-rich manipulation tasks has yielded impressive results in recent years. While many studies in RL focus on varying the observation space or reward model, few efforts focused on the choice of action space (e.g. joint or end-effector space, position, velocity, etc.). However, studies in robot motion control indicate that choosing an action space that conforms to the characteristics of the task can simplify exploration and improve robustness to disturbances. This paper studies the effect of different action spaces in deep RL and advocates for variable impedance control in end-effector space (VICES) as an advantageous action space for constrained and contact-rich tasks. We evaluate multiple action spaces on three prototypical manipulation tasks: Path Following (task with no contact), Door Opening (task with kinematic constraints), and Surface Wiping (task with continuous contact). We show that VICES improves sample efficiency, maintains low energy consumption, and ensures safety across all three experimental setups. Further, RL policies learned with VICES can transfer across different robot models in simulation, and from simulation to real for the same robot. Further information is available at https://stanfordvl.github.io/vices.

Variable Impedance Control in End-Effector Space: An Action Space for Reinforcement Learning in Contact-Rich Tasks

Despite recent successes of reinforcement learning (RL), it remains a challenge for agents to transfer learned skills to related environments. To facilitate research addressing this problem, we propose CausalWorld, a benchmark for causal structure and transfer learning in a robotic manipulation environment. The environment is a simulation of an open-source robotic platform, hence offering the possibility of sim-to-real transfer. Tasks consist of constructing 3D shapes from a given set of blocks - inspired by how children learn to build complex structures. The key strength of CausalWorld is that it provides a combinatorial family of such tasks with common causal structure and underlying factors (including, e.g., robot and object masses, colors, sizes). The user (or the agent) may intervene on all causal variables, which allows for fine-grained control over how similar different tasks (or task distributions) are. One can thus easily define training and evaluation distributions of a desired difficulty level, targeting a specific form of generalization (e.g., only changes in appearance or object mass). Further, this common parametrization facilitates defining curricula by interpolating between an initial and a target task. While users may define their own task distributions, we present eight meaningful distributions as concrete benchmarks, ranging from simple to very challenging, all of which require long-horizon planning as well as precise low-level motor control. Finally, we provide baseline results for a subset of these tasks on distinct training curricula and corresponding evaluation protocols, verifying the feasibility of the tasks in this benchmark.

CausalWorld: A Robotic Manipulation Benchmark for Causal Structure and Transfer Learning

The goal of this review is to evaluate the current status of multi-robot systems in the context of search and rescue. This includes an investigation of their current use in the field, what major technical challenge areas currently preclude more widespread use, and which key topics will drive future development and adoption. Work blending machine learning with classical control techniques is driving progress in perception-driven autonomy, decentralized multi-robot coordination, and human–robot interaction, among others. Ad hoc mesh networking has achieved reliability suitable for safety-critical applications and may be a partial solution for communication. New modular and multimodal platforms may overcome mobility limitations without significantly increasing cost. Multi-agent systems are not currently ready for deployment in search and rescue applications; however, progress is being made in a number of critical domains. As the field matures, research should focus on realistic evaluations of constituent technologies, and on confronting the challenges of simulation-to-reality transfer, algorithmic bias in autonomous agents that rely on machine learning, and novelty-versus-reliability incentive mismatch

https://link.springer.com/content/pdf/10.1007/s43154-021-00048-3.pdf

Multi-Agent Systems for Search and Rescue Applications

We combine Boston Dynamics Spot® with a light-weight, external robot arm to perform dynamic grasping maneuvers. While Spot is a reliable, robust and easy-to-control mobile robot, these highly desirable qualities come with the price that the control access granted to the user is restricted. Consequently Spot’s behavior must largely be treated as a black box, which causes difficulties when combined with a moving payload such as a robotic arm. We overcome the arising challenges by building a model of the combined platform, fitting the corresponding model parameters using experimental data and a straight-forward optimization framework. We use this model to generate control commands for the physical platform using trajectory optimization. We demonstrate that even with a simple model, and control trajectories deployed in a feed-forward manner, the combined platform is capable of executing grasping tasks in a dynamic fashion. Furthermore, we show how the platform can use the additional degrees of freedom of the legs to extend the reachability of the arm.

Go Fetch! - Dynamic Grasps using Boston Dynamics Spot with External Robotic Arm

Dexterous object manipulation remains an open problem in robotics, despite the rapid progress in machine learning during the past decade. We argue that a hindrance is the high cost of experimentation on real systems, in terms of both time and money. We address this problem by proposing an open-source robotic platform which can safely operate without human supervision. The hardware is inexpensive (about \SI{5000}[\$]{}) yet highly dynamic, robust, and capable of complex interaction with external objects. The software operates at 1-kilohertz and performs safety checks to prevent the hardware from breaking. The easy-to-use front-end (in C++ and Python) is suitable for real-time control as well as deep reinforcement learning. In addition, the software framework is largely robot-agnostic and can hence be used independently of the hardware proposed herein. Finally, we illustrate the potential of the proposed platform through a number of experiments, including real-time optimal control, deep reinforcement learning from scratch, throwing, and writing.

TriFinger: An Open-Source Robot for Learning Dexterity.

Online planning of whole-body motions for legged robots is challenging due to the inherent nonlinearity in the robot dynamics. In this work, we propose a nonlinear model predictive control (MPC) framework, the BiConMP which can generate whole body trajectories online by efficiently exploiting the structure of the robot dynamics. BiConMP is used to generate various cyclic gaits on a real quadruped robot and its performance is evaluated on different terrain, countering unforeseen pushes, and transitioning online between different gaits. Furthermore, the ability of BiConMP to generate nontrivial acyclic whole-body dynamic motions on the robot is presented. The same approach is also used to generate various dynamic motions in MPC on a humanoid robot (Talos) and another quadruped robot (AnYmal) in simulation. Finally, an extensive empirical analysis on the effects of planning horizon and frequency on the nonlinear MPC framework is reported and discussed.

BiConMP: A Nonlinear Model Predictive Control Framework for Whole Body Motion Planning

In this letter, we present a method for learning a reactive policy for a simple dynamic locomotion task involving hard impact and switching contacts where we assume the contact location and contact timing to be unknown. To learn such a policy, we use optimal control to optimize a local controller for a fixed environment and contacts. We learn the contact-rich dynamics for our underactuated systems along these trajectories in a sample efficient manner. We use the optimized policies to learn the reactive policy in form of a neural network. Using a new neural network architecture, we are able to preserve more information from the local policy and make its output interpretable in the sense that its output in terms of desired trajectories, feedforward commands and gains can be interpreted. Extensive simulations demonstrate the robustness of the approach to changing environments, outperforming a model-free gradient policy based methods on the same tasks in simulation. Finally, we show that the learned policy can be robustly transferred on a real robot.

/pdf/learning-a-structured-neural-network-policy-for-a-hopping-1s340i6w8i.pdf

Learning a Structured Neural Network Policy for a Hopping Task

Whole-body optimizers have been successful at automatically computing complex dynamic locomotion behaviors. However they are often limited to offline planning as they are computationally too expensive to replan with a high frequency. Simpler models are then typically used for online replanning. In this paper we present a method to generate whole body movements in real-time for locomotion tasks. Our approach consists in learning a centroidal neural network that predicts the desired centroidal motion given the current state of the robot and a desired contact plan. The network is trained using an existing whole body motion optimizer. Our approach enables to learn with few training samples dynamic motions that can be used in a complete whole-body control framework at high frequency, which is usually not attainable with typical full-body optimizers. We demonstrate our method to generate a rich set of walking and jumping motions on a real quadruped robot.

Julian Viereck

Papers

An Open Torque-Controlled Modular Robot Architecture for Legged Locomotion Research

TriFinger: An Open-Source Robot for Learning Dexterity.

BiConMP: A Nonlinear Model Predictive Control Framework for Whole Body Motion Planning

Learning a Structured Neural Network Policy for a Hopping Task

Learning a Centroidal Motion Planner for Legged Locomotion