Wearable Sensors-Enabled Human–Machine Interaction Systems: From Design to Application

The era of artificial intelligence and internet of things is rapidly developed by recent advances in wearable electronics. Gait reveals sensory information in daily life containing personal information, regarding identification and healthcare. Current wearable electronics of gait analysis are mainly limited by high fabrication cost, operation energy consumption, or inferior analysis methods, which barely involve machine learning or implement nonoptimal models that require massive datasets for training. Herein, we developed low-cost triboelectric intelligent socks for harvesting waste energy from low-frequency body motions to transmit wireless sensory data. The sock equipped with self-powered functionality also can be used as wearable sensors to deliver information, regarding the identity, health status, and activity of the users. To further address the issue of ineffective analysis methods, an optimized deep learning model with an end-to-end structure on the socks signals for the gait analysis is proposed, which produces a 93.54% identification accuracy of 13 participants and detects five different human activities with 96.67% accuracy. Toward practical application, we map the physical signals collected through the socks in the virtual space to establish a digital human system for sports monitoring, healthcare, identification, and future smart home applications.

/pdf/deep-learning-enabled-triboelectric-smart-socks-for-iot-4uv9a5jvur.pdf

Deep learning-enabled triboelectric smart socks for IoT-based gait analysis and VR applications

Cardiovascular diseases remain the leading cause of death worldwide. The rapid development of flexible sensing technologies and wearable pressure sensors have attracted keen research interest and have been widely used for long‐term and real‐time cardiovascular status monitoring. Owing to compelling characteristics, including light weight, wearing comfort, and high sensitivity to pulse pressures, physiological pulse waveforms can be precisely and continuously monitored by flexible pressure sensors for wearable health monitoring. Herein, an overview of wearable pressure sensors for human pulse wave monitoring is presented, with a focus on the transduction mechanism, microengineering structures, and related applications in pulse wave monitoring and cardiovascular condition assessment. The conceptualizations and methods for the acquisition of physiological and pathological information related to the cardiovascular system are outlined. The biomechanics of arterial pulse waves and the working mechanism of various wearable pressure sensors, including triboelectric, piezoelectric, magnetoelastic, piezoresistive, capacitive, and optical sensors, are also subject to systematic debate. Exemple applications of pulse wave measurement based on microengineering structured devices are then summarized. Finally, a discussion of the opportunities and challenges that wearable pressure sensors face, as well as their potential as a wearable intelligent system for personalized healthcare is given in conclusion.

Wearable Pressure Sensors for Pulse Wave Monitoring

Triboelectric nanogenerator based self-powered sensor for artificial intelligence

Sign language recognition, especially the sentence recognition, is of great significance for lowering the communication barrier between the hearing/speech impaired and the non-signers. The general glove solutions, which are employed to detect motions of our dexterous hands, only achieve recognizing discrete single gestures (i.e., numbers, letters, or words) instead of sentences, far from satisfying the meet of the signers’ daily communication. Here, we propose an artificial intelligence enabled sign language recognition and communication system comprising sensing gloves, deep learning block, and virtual reality interface. Non-segmentation and segmentation assisted deep learning model achieves the recognition of 50 words and 20 sentences. Significantly, the segmentation approach splits entire sentence signals into word units. Then the deep learning model recognizes all word elements and reversely reconstructs and recognizes sentences. Furthermore, new/never-seen sentences created by new-order word elements recombination can be recognized with an average correct rate of 86.67%. Finally, the sign language recognition results are projected into virtual space and translated into text and audio, allowing the remote and bidirectional communication between signers and non-signers. Though wearable gloves are widely used for gesture associated applications (e.g. sign language recognition), sentence identification of sign language remains a challenge. Here, the authors report AI-enabled recognition system helps barrier-free communication between signers and non-signers.

/pdf/ai-enabled-sign-language-recognition-and-vr-space-49j926dz9s.pdf

AI enabled sign language recognition and VR space bidirectional communication using triboelectric smart glove.

Designing efficient sensors for soft robotics aiming at human machine interaction remains a challenge. Here, we report a smart soft-robotic gripper system based on triboelectric nanogenerator sensors to capture the continuous motion and tactile information for soft gripper. With the special distributed electrodes, the tactile sensor can perceive the contact position and area of external stimuli. The gear-based length sensor with a stretchable strip allows the continuous detection of elongation via the sequential contact of each tooth. The triboelectric sensory information collected during the operation of soft gripper is further trained by support vector machine algorithm to identify diverse objects with an accuracy of 98.1%. Demonstration of digital twin applications, which show the object identification and duplicate robotic manipulation in virtual environment according to the real-time operation of the soft-robotic gripper system, is successfully created for virtual assembly lines and unmanned warehouse applications. Designing efficient sensors for human machine interaction remains a challenge. Here, the authors present a soft robotic fingers system based on a triboelectric nanogenerator (L-TENG) sensor to capture the continuous motion of soft gripper and a soft tactile (T-TENG) sensor for tactile sensing, that can achieve an object recognition accuracy of 98.1%.

/pdf/triboelectric-nanogenerator-sensors-for-soft-robotics-aiming-4bd7m33b43.pdf

Triboelectric nanogenerator sensors for soft robotics aiming at digital twin applications

Gait and waist motions always contain massive personnel information and it is feasible to extract these data via wearable electronics for identification and healthcare based on the Internet of Things (IoT). There also remains a demand to develop a cost-effective human-machine interface to enhance the immersion during the long-term rehabilitation. Meanwhile, triboelectric nanogenerator (TENG) revealing its merits in both wearable electronics and IoT tends to be a possible solution. Herein, the authors present wearable TENG-based devices for gait analysis and waist motion capture to enhance the intelligence and performance of the lower-limb and waist rehabilitation. Four triboelectric sensors are equidistantly sewed onto a fabric belt to recognize the waist motion, enabling the real-time robotic manipulation and virtual game for immersion-enhanced waist training. The insole equipped with two TENG sensors is designed for walking status detection and a 98.4% identification accuracy for five different humans aiming at rehabilitation plan selection is achieved by leveraging machine learning technology to further analyze the signals. Through a lower-limb rehabilitation robot, the authors demonstrate that the sensory system performs well in user recognition, motion monitoring, as well as robot and gaming-aided training, showing its potential in IoT-based smart healthcare applications.

Wearable Triboelectric Sensors Enabled Gait Analysis and Waist Motion Capture for IoT-Based Smart Healthcare Applications

Self-sustainable flow-velocity detection via electromagnetic/triboelectric hybrid generator aiming at IoT-based environment monitoring

Highly flexible and environmentally adaptive soft robots have received considerable attention. There remains a demand for soft robots to realize the stiffness modulation and variable workspace for robust and versatile manipulations. This article presents a compact soft gripper with a polylactic acid-based variable stiffness module (VSM) and a rigid retractable mechanism to achieve soft-rigid hybrid actuation. The soft gripper can enhance its stiffness by 18-fold without sacrificing flexibility due to the VSM. A heating circuit is designed to divide the VSM into three regions. Each region can be activated separately for varying flexible segments to amplify the dexterity. Meanwhile, the water-cooling system accelerates the heat exchange, thus reducing the cooling time from ∼400 to 39 s. The rigid retractable mechanism can adjust the initial layout of the gripper to expand the workspace and perform manipulation by opening and closing fingers. The soft finger combined with stiffness tunability can maintain its deformation after being stiffened to realize morphing. Therefore, it can efficiently perform a grasp with a high load and avoid repeated heating and cooling, especially for items with a similar shape. The performance of the gripper is further validated by measuring the grasping force and grasping demonstration with various objects, showing its robustness and dexterity in versatile tasks.

Stiffness-Tunable Soft Gripper with Soft-Rigid Hybrid Actuation for Versatile Manipulations.

Twisted and coiled polymer actuator (TCPA), manufactured from low-cost material such as fishing line, has been widely applied and playing a significant role in soft robotic. Therefore, it is not surprising that it has been attracting much attention of researchers. The main consideration for driving TCPA lies in the heating process. However, the heating process causes stiffness degradation and precisely leads its structural to be unstable. Accordingly, it is proposed through this paper, a method of adjusting the pre-stretching to reach the saturated contraction. The pre-stretch is adjusted to bring the TCPA to saturated contraction within the given heat. Due to the contact of adjacent coils of TCPA, there is not enough space to contract, thus TCPA is compressed into the hard structure by increasing the heat. Initially, the stiffness will be decreased and then be increased during the entire heating process. So the flexibility of the motion state and the stability working state can be achieved. Besides that, the proposed method can also improve the rigor of TCPA by more than two times during saturated contraction. It has been shown by some relevant results of a custom platform experiment.

Quan Zhang

Papers

Triboelectric nanogenerator sensors for soft robotics aiming at digital twin applications

Wearable Triboelectric Sensors Enabled Gait Analysis and Waist Motion Capture for IoT-Based Smart Healthcare Applications

Self-sustainable flow-velocity detection via electromagnetic/triboelectric hybrid generator aiming at IoT-based environment monitoring

Stiffness-Tunable Soft Gripper with Soft-Rigid Hybrid Actuation for Versatile Manipulations.

Research on the mechanism of variable stiffness of the twisted and coiled polymer actuator during saturated contraction