The fourth industrial revolution, also known as Industry 4.0, is reshaping the way individuals live and work while providing a substantial influence on the manufacturing scenario. The key enabling technology that has made Industry 4.0 a concrete reality is without doubt collaborative robotics, which is also evolving as a fundamental pillar of the next revolution, the so-called Industry 5.0. The improvement of employees’ safety and well-being, together with the increase of profitability and productivity, are indeed the main goals of human-robot collaboration (HRC) in the industrial setting. The robotic controller design and the analysis of existing decision and control techniques are crucially needed to develop innovative models and state-of-the-art methodologies for a safe, ergonomic, and efficient HRC. To this aim, this paper presents an accurate review of the most recent and relevant contributions to the related literature, focusing on the control perspective. All the surveyed works are carefully selected and categorized by target (i.e., safety, ergonomics, and efficiency), and then by problem and type of control, in presence or absence of optimization. Finally, the discussion of the achieved results and the analysis of the emerging challenges in this research field are reported, highlighting the identified gaps and the promising future developments in the context of the digital evolution. Note to Practitioners—The design and development of manufacturing systems are experiencing substantial changes towards full automation. This ongoing challenge is being tackled by academia and industrial practitioners with the adoption of collaborative robots, where the skills and peculiarities of humans (e.g., intelligence, creativity, adaptability, etc.) and robots (e.g., flexibility, pinpoint accuracy, tirelessness, etc.) are combined to better perform a variety of tasks. Nevertheless, due to their different characteristics, there is an emerging need for designing suitable decision and control techniques to ensure a safe and ergonomic HRC, while keeping the highest level of productivity. Against this background, the aim of this paper is to provide researchers and practitioners with a reference source in the related field, which can help them designing and developing suitable solutions to control problems in safe, ergonomic, and efficient collaborative robotics.

/pdf/control-techniques-for-safe-ergonomic-and-efficient-human-3wsavedx.pdf

Control Techniques for Safe, Ergonomic, and Efficient Human-Robot Collaboration in the Digital Industry: A Survey

Vision-based control systems play an important role in modern robotics systems. An important task in implementing such a system is developing an effective algorithm for recognizing human actions and the working environment and the design of intuitive gesture commands. This paper proposes an action recognition algorithm for robotics and manufacturing automation. The key contributions are (1) fusion of multimodal information obtained by depth sensors and cameras of the visible range, (2) modified Gabor-based and 3-D binary-based descriptor using micro-block difference, (3) efficient skeleton-based descriptor, and (4) recognition algorithm using the combined descriptor. The proposed binary micro-block difference representation of 3-D patches from video with a complex background in several scales and orientations leads to an informative description of the scene action. The experimental results showed the effectiveness of the proposed algorithm on datasets.

/pdf/action-recognition-for-the-robotics-and-manufacturing-2lpd4duff6.pdf

Action recognition for the robotics and manufacturing automation using 3-D binary micro-block difference

This paper introduces a computationally efficient control scheme for safe human–robot interaction. The method relies on the Explicit Reference Governor (ERG) formalism to enforce input and state constraints in real-time, thus ensuring that the robot can safely operate in close proximity to humans. The resulting constrained control method can steer the robot arm to the desired end-effector pose (or a steady-state admissible approximation thereof) in the presence of actuator saturation, limited joint ranges, speed limits, static obstacles, and humans. The effectiveness of the proposed solution is supported by theoretical results and numerous experimental validations on the Franka Emika Panda robotic manipulator, a commercially available collaborative 7-DOF robot arm.

/pdf/real-time-motion-control-of-robotic-manipulators-for-safe-ti2oo4lpu7.pdf

Real-time motion control of robotic manipulators for safe human–robot coexistence

Human–Robot Collaboration (HRC) has a pivotal role in smart manufacturing for strict requirements of human-centricity, sustainability, and resilience. However, existing HRC development mainly undertakes either a human-dominant or robot-dominant manner, where human and robotic agents reactively perform operations by following pre-defined instructions, thus far from an efficient integration of robotic automation and human cognition. The stiff human–robot relations fail to be qualified for complex manufacturing tasks and cannot ease the physical and psychological load of human operators. In response to these realistic needs, this paper presents our arguments on the obvious trend, concept, systematic architecture, and enabling technologies of Proactive HRC, serving as a prospective vision and research topic for future work in the human-centric smart manufacturing era. Human–robot symbiotic relation is evolving with a 5C intelligence — from Connection, Coordination, Cyber, Cognition to Coevolution, and finally embracing mutual-cognitive, predictable, and self-organising intelligent capabilities, i.e., the Proactive HRC. With proactive robot control, multiple human and robotic agents collaboratively operate manufacturing tasks, considering each others’ operation needs, desired resources, and qualified complementary capabilities. This paper also highlights current challenges and future research directions, which deserve more research efforts for real-world applications of Proactive HRC. It is hoped that this work can attract more open discussions and provide useful insights to both academic and industrial practitioners in their exploration of human–robot flexible production. 

Proactive human-robot collaboration: Mutual-cognitive, predictable, and self-organising perspectives

Human-robot interaction (HRI) occupies an essential role in the flourishing market for intelligent robots for a wide range of asymmetric personal and entertainment applications, ranging from assisting older people and the severely disabled to the entertainment robots at amusement parks. Improving the way humans and machines interact can help democratize robotics. With machine and deep learning techniques, robots will more easily adapt to new tasks, conditions, and environments. In this paper, we develop, implement, and evaluate the performance of the machine-learning-based HRI model in a collaborative environment. Specifically, we examine five supervised machine learning models viz. the ensemble of bagging trees (EBT) model, the k-nearest neighbor (kNN) model, the logistic regression kernel (LRK), the fine decision trees (FDT), and the subspace discriminator (SDC). The proposed models have been evaluated on an ample and modern contact detection dataset (CDD 2021). CDD 2021 is gathered from a real-world robot arm, Franka Emika Panda, when it was executing repetitive asymmetric movements. Typical performance assessment factors are applied to assess the model effectiveness in terms of detection accuracy, sensitivity, specificity, speed, and error ratios. Our experiential evaluation shows that the ensemble technique provides higher performance with a lower error ratio compared with other developed supervised models. Therefore, this paper proposes an ensemble-based bagged trees (EBT) detection model for classifying physical human–robot contact into three asymmetric types of contacts, including noncontact, incidental, and intentional. Our experimental results exhibit outstanding contact detection performance metrics scoring 97.1%, 96.9%, and 97.1% for detection accuracy, precision, and sensitivity, respectively. Besides, a low prediction overhead has been observed for the contact detection model, requiring a 102 µS to provide the correct detection state. Hence, the developed scheme can be efficiently adopted through the application requiring physical human–robot contact to give fast accurate detection to the contacts between the human arm and the robot arm.

/pdf/asymmetric-identification-model-for-human-robot-contacts-via-3i59eduh.pdf

Asymmetric Identification Model for Human-Robot Contacts via Supervised Learning

Human-Robot Interaction (HRI) is a largely addressed subject today. In order to ensure co-existence and space sharing between human and robot, collision avoidance is one of the main strategies for interaction between them without contact. It is thus usual to use a 3D depth camera sensor (Microsoft Kinect V2) which may involve issues related to occluded robot in the camera view. While several works overcame this issue by applying infinite depth principle or increasing the number of cameras, in the current work we developed and applied an original new approach that combines data of one 3D depth sensor (Kinect) and proprioceptive robot sensors. This method uses the principle of limited safety contour around the obstacle to dynamically estimate the robot-obstacle distance, and then generate the repulsive force that controls the robot. For validation, our approach is applied in real time to avoid collisions between dynamical obstacles (humans or objects) and the end-effector of a real 7-dof Kuka LBR iiwa collaborative robot. Our method is experimentally compared with existing methods based on infinite depth principle when the robot is hidden by the obstacle with respect to the camera view. Results showed smoother behavior and more stability of the robot using our method. Extensive experiments of our method, using several strategies based on distancing and its combination with dodging were done. Results have shown a reactive and efficient collision avoidance, by ensuring a minimum obstacle-robot distance (of $\approx \text{240 mm}$ ), even when the robot is in an occluded zone in the Kinect camera view.

/pdf/collision-avoidance-interaction-between-human-and-a-hidden-3z2ami25bb.pdf

Collision Avoidance Interaction Between Human and a Hidden Robot Based on Kinect and Robot Data Fusion

Human-Robot Interaction (HRI) is a largely ad-dressed subject today. Collision avoidance is one of main strategies that allow space sharing and interaction without contact between human and robot. It is thus usual to use a 3D depth camera sensor which may involves issues related to occluded robot in camera view. While several works overcame this issue by applying infinite depth principle or increasing the number of cameras, we developed in the current work a new and an original approach based on the combination of a 3D depth sensor (Microsoft® Kinect V2) and the proprioceptive robot position sensors. This method uses a principle of limited safety contour around the obstacle to dynamically estimate the robot-obstacle distance, and then generate the repulsive force that controls the robot. For validation, our approach is applied in real time to avoid collision between dynamical obstacles (humans or objects) and the end-effector of a real 7-dof Kuka LBR iiwa collaborative robot.Several strategies based on distancing and its combination with dodging were tested. Results have shown a reactive and efficient collision avoidance, by ensuring a minimum obstacle-robot distance (of ≈ 240mm), even when the robot is in an occluded zone in the Kinect camera view

/pdf/collision-avoidance-in-human-robot-interaction-using-kinect-4m3gq2yin4.pdf

Collision Avoidance in Human-Robot Interaction Using Kinect Vision System Combined With Robot’s Model and Data

Over the last years, physical Human–Robot Interaction (pHRI) has become a particularly interesting topic for industrial tasks. An important issue is allowing people and robots to collaborate in an useful, simple and safe manner. In this work, we propose a new framework that allows the person to collaborate with a robot manipulator, while the robot has its own predefined task. To allow the robot to smoothly switch from its own task to be a compliant collaborator for the person, a variable admittance control is developed. Furthermore, in general the task to accomplish requires the robot to carry variable, unknown, loads at the end-effector. To include this feature in our framework, a robust control is also included to preserve the performance of the robot despite uncertainties coming from the unknown load. To validate our approach, experiments were carried out with a Kuka LBR iiwa 14 R820, first to validate both parts of the controller, and finally, to study a use-case scenario similar to an industrial production line. Results show the efficiency of this approach to allow the person to collaborate at any moment while the robot is capable of performing another task. This flexible framework for object co-manipulation also allows unknown loads up to 2 kg to be handled without making the task more difficult for the person. 

Robust variable admittance control for human-robot co-manipulation of objects with unknown load

The human-robot collaboration is a promising and challenging field of robotics research. One of the main collaboration tasks is the object co-manipulation where the human and robot are in a continuous physical interaction and forces exerted must be handled. This involves some issues known in robotics as physical Human-Robot Interaction (pHRI), where human safety and interaction comfort are required. Moreover, a definition of interaction quality metrics would be relevant.In the current work, the assessment of Human-Robot object co-manipulation task was explored through the proposed metrics of interaction quality, based on human forces throughout the movement. This analysis is based on co-manipulation of objects with different dynamical properties (weight and inertia), with and without including these properties knowledge in the robot control law. Here, the human is a leader of task and the robot the follower without any information of the human trajectory and movement profile. For the robot control law, a well-known impedance control was applied on a 7-dof Kuka LBR iiwa 14 R820 robot.Results show that the consideration of object dynamical properties in the robot control law is crucial for a good and more comfortable interaction. Besides, human efforts are more significant with a higher no-considered weight, whereas it remains stable when these weights were considered.

/pdf/evaluation-of-human-robot-object-co-manipulation-under-robot-219sc41bfu.pdf

Evaluation of Human-Robot Object Co-manipulation Under Robot Impedance Control

Physical Human-Robot Interaction (pHRI) is an important field that has grown considerably over the last years. Nowadays, new possible applications are being evaluated. In order to tackle the new issues that might arise, novel methods for robot control have to be used. To ease the comparison and evaluation of those methods, this work proposes a simulated frame-work to study the collaboration between a robot manipulator and a person while moving payloads. The way of conceiving the simulated framework done in Matlab is presented, detailing how the robot, the object, and the person can be described. Experiments are done on a real robot KUKA LBR iiwa 14 R820 to compare and validate the interaction forces predicted by the proposed framework. Results showed that this method allows to simulate the co-manipulation of objects, even with unknown load, showing forces that are close to the real ones applied by the person. The Matlab code is publicly available at https://gitlab.com/MMujica/simulated_framework_phrc.

Martin Mujica

Papers

Collision Avoidance Interaction Between Human and a Hidden Robot Based on Kinect and Robot Data Fusion

Collision Avoidance in Human-Robot Interaction Using Kinect Vision System Combined With Robot’s Model and Data

Robust variable admittance control for human-robot co-manipulation of objects with unknown load

Evaluation of Human-Robot Object Co-manipulation Under Robot Impedance Control

Simulated Framework for Physical Human-Robot Collaboration to Co-Manipulate Objects