Humans can feel, weigh and grasp diverse objects, and simultaneously infer their material properties while applying the right amount of force—a challenging set of tasks for a modern robot1. Mechanoreceptor networks that provide sensory feedback and enable the dexterity of the human grasp2 remain difficult to replicate in robots. Whereas computer-vision-based robot grasping strategies3–5 have progressed substantially with the abundance of visual data and emerging machine-learning tools, there are as yet no equivalent sensing platforms and large-scale datasets with which to probe the use of the tactile information that humans rely on when grasping objects. Studying the mechanics of how humans grasp objects will complement vision-based robotic object handling. Importantly, the inability to record and analyse tactile signals currently limits our understanding of the role of tactile information in the human grasp itself—for example, how tactile maps are used to identify objects and infer their properties is unknown6. Here we use a scalable tactile glove and deep convolutional neural networks to show that sensors uniformly distributed over the hand can be used to identify individual objects, estimate their weight and explore the typical tactile patterns that emerge while grasping objects. The sensor array (548 sensors) is assembled on a knitted glove, and consists of a piezoresistive film connected by a network of conductive thread electrodes that are passively probed. Using a low-cost (about US$10) scalable tactile glove sensor array, we record a large-scale tactile dataset with 135,000 frames, each covering the full hand, while interacting with 26 different objects. This set of interactions with different objects reveals the key correspondences between different regions of a human hand while it is manipulating objects. Insights from the tactile signatures of the human grasp—through the lens of an artificial analogue of the natural mechanoreceptor network—can thus aid the future design of prosthetics7, robot grasping tools and human–robot interactions1,8–10. Tactile patterns obtained from a scalable sensor-embedded glove and deep convolutional neural networks help to explain how the human hand can identify and grasp individual objects and estimate their weights.

Learning the signatures of the human grasp using a scalable tactile glove.

Tactile sensing in dexterous robot hands - Review

Tactile sensing is an important perception mode for robots, but the existing tactile technologies have multiple limitations. What kind of tactile information robots need, and how to use the information, remain open questions. We believe a soft sensor surface and high-resolution sensing of geometry should be important components of a competent tactile sensor. In this paper, we discuss the development of a vision-based optical tactile sensor, GelSight. Unlike the traditional tactile sensors which measure contact force, GelSight basically measures geometry, with very high spatial resolution. The sensor has a contact surface of soft elastomer, and it directly measures its deformation, both vertical and lateral, which corresponds to the exact object shape and the tension on the contact surface. The contact force, and slip can be inferred from the sensor’s deformation as well. Particularly, we focus on the hardware and software that support GelSight’s application on robot hands. This paper reviews the development of GelSight, with the emphasis in the sensing principle and sensor design. We introduce the design of the sensor’s optical system, the algorithm for shape, force and slip measurement, and the hardware designs and fabrication of different sensor versions. We also show the experimental evaluation on the GelSight’s performance on geometry and force measurement. With the high-resolution measurement of shape and contact force, the sensor has successfully assisted multiple robotic tasks, including material perception or recognition and in-hand localization for robot manipulation.

GelSight: High-Resolution Robot Tactile Sensors for Estimating Geometry and Force.

Tactile sensing is an essential component in human–robot interaction and object manipulation. Soft sensors allow for safe interaction and improved gripping performance. Here we present the...

/pdf/the-tactip-family-soft-optical-tactile-sensors-with-3d-37b410gisb.pdf

The TacTip Family: Soft Optical Tactile Sensors with 3D-Printed Biomimetic Morphologies.

Symbiotic human-robot collaborative assembly

This paper presents the preliminary prototype design and implementation of the Nazarbayev University (NU) Hand, a new semi-anthropomorphic multigrasp robotic hand. The hand is designed to be an end effector for industrial and service robots. The main objective is to develop a low-cost, low-weight and easily manufacturable robotic hand with a sensor module allowing acquisition of data for autonomous intelligent object manipulation. 3D printing technologies were extensively used in the implementation of the hand. Specifically, the structure of the hand is printed using a 3D printer as a complete assembly voiding the need of using fasteners and bearings for the assembly of the hand and decreasing the total weight. The hand also incorporates a sensor module containing a LIDAR, digital camera and non-contact infrared temperature sensor for intelligent automation. As an alternative to teach pendants for the industrial manipulators, a teaching glove was developed, which acts as the primary human machine interface between the user and the NU Hand. The paper presents an extensive performance characterization of the robotic hand including finger forces, weight, audible noise level during operation and sensor data acquisition.

Semi-anthropomorphic 3D printed multigrasp hand for industrial and service robots

Object manipulation performed by robots refers to the art of controlling the shape and location of an object through force constraints with robot end-effectors, both robot hands, and grippers. The success of task execution is usually guaranteed by the sense of touch. In this work, we present an optical tactile sensor incorporating plastic optical fibers, transparent silicone rubber, and an off-the-shelf color camera that can detect: translational and rotational shear forces, and contact location and its normal force. Contact localization is possible thanks to the shear strain. Specifically, one of the layers stretches so that its thickness decreases. The decrease in the thickness results in the color change at the point of contact. Elastic behavior of the sensing media provides a robust rotational and translational shear detection mechanism when torque and planar force, respectively, are applied onto the sensing surface. Thanks to the plastic optofibers, signal processing electronics are placed away from the sensing surface making the sensor immune to hazardous environments. Machine learning techniques were used to benchmark the sensing performance of the sensor. By implementing a multi-output CNN model, the contact type was classified into normal and shear or torsional deformation and their corresponding continuous contact features were estimated.

Shear, Torsion and Pressure Tactile Sensor via Plastic Optofiber Guided Imaging

https://iopscience.iop.org/article/10.1088/1361-665X/ab773a/pdf

Zhanat Kappassov

Papers

Tactile sensing in dexterous robot hands - Review

Semi-anthropomorphic 3D printed multigrasp hand for industrial and service robots

Shear, Torsion and Pressure Tactile Sensor via Plastic Optofiber Guided Imaging

Analytical modeling and design of negative stiffness honeycombs

Touch driven controller and tactile features for physical interactions