Showing papers on "Haptic technology published in 2021"

A Survey of Mid-Air Ultrasound Haptics and Its Applications

Abstract: Ultrasound haptics is a contactless haptic technology that enables novel mid-air interactions with rich multisensory feedback. This article surveys recent advances in ultrasound haptic technology. We discuss the fundamentals of this haptic technology, how a variety of perceptible sensations are rendered, and how it is currently being used to enable novel interaction techniques. We summarize its strengths, weaknesses, and potential applications across various domains. We conclude with our perspective on key directions for this promising haptic technology.

Smart Tactile Gloves for Haptic Interaction, Communication, and Rehabilitation

Abstract: Wearable human machine interfaces (HMI) such as smart gloves have attracted considerable interest in recent years. The quality of the interactive experience with the real and virtual world using wearable HMI technologies depends on the intuitive two‐way haptic interfaces they offer and the real‐time touch‐based information they send and receive. Herein, various smart glove solutions and their application in interaction, rehabilitation, virtual (VR) and augmented reality (AR), and augmentative and alternative communication (AAC) tasks are reviewed. While the early variants of such systems were based on commercial touch sensors and displays integrated (e.g., stitched) on wearables, electronic skin (e‐skin)‐type technologies with multifunctional capabilities are being explored nowadays for rich user experience and comfort. In this regard, instead of using separate touch sensors and actuators, miniaturized integrated devices providing both touch sensing and vibrotactile actuation have also been reported recently. Such advances, the associated challenges, and the advantages they offer for users to enjoy the full characteristic benefits of VR/ARs for interaction, immersion, and imagination are discussed. Finally, the huge potential the smart‐glove‐type solutions hold for advances in various application areas such as robotics, health care, sensorial augmentation for nondisabled and tactile Internet is also discussed.

SensAct: The Soft and Squishy Tactile Sensor with Integrated Flexible Actuator

Abstract: Herein, a novel tactile sensing device (SensAct) with a soft touch/pressure sensor seamlessly integrated on a flexible actuator is presented. The squishy touch sensor is developed with custom‐made graphite paste on a tiny permanent magnet, encapsulated in Sil‐Poxy, and the actuator (15 μ‐thick coil) is fabricated on polyimide by Lithographie Galvanoformung Abformung (LIGA) micromolding method. The actuator can operate in two modes (expansion and contraction/squeeze) and two states (vibration and nonvibration). The sensor was tested with up to 12 N applied forces and exhibited ≈70% average relative resistance variation (ΔR/Ro), ≈0.346 kPa−1 sensitivity, and ≈49 ms response time with excellent repeatability (≈12.7% coefficient of variation) at 5 N. During simultaneous sensing and actuation, the modulation of coil current, due to ΔR/Ro (≈14% at 2 N force) in the sensor, allows the close loop control (ΔI/Io ≈385%) of expansion/contraction (≈69.8 μm expansion in nonvibration state and ≈111.5 μm peak‐to‐peak in the vibration state). Finally, the soft sensor is embedded in the 3D‐printed fingertip of a robotic hand to demonstrate its use for pressure mapping along with remote vibrotactile stimulation using SensAct device. The self‐controllable actuation of SensAct could provide eSkin the ability to tune stiffness and the vibration states could be utilized for controlled haptic feedback.

A Comprehensive Survey of the Tactile Internet: State-of-the-Art and Research Directions

Abstract: The Internet has made several giant leaps over the years, from a fixed to a mobile Internet, then to the Internet of Things, and now to a Tactile Internet. The Tactile Internet goes far beyond data, audio and video delivery over fixed and mobile networks, and even beyond allowing communication and collaboration among things. It is expected to enable haptic communications and allow skill set delivery over networks. Some examples of potential applications are tele-surgery, vehicle fleets, augmented reality and industrial process automation. Several papers already cover many of the Tactile Internet-related concepts and technologies, such as haptic codecs, applications, and supporting technologies. However, none of them offers a comprehensive survey of the Tactile Internet, including its architectures and algorithms. Furthermore, none of them provides a systematic and critical review of the existing solutions. To address these lacunae, we provide a comprehensive survey of the architectures and algorithms proposed to date for the Tactile Internet. In addition, we critically review them using a well-defined set of requirements and discuss some of the lessons learned as well as the most promising research directions.

Development of a Tactile Sensing Robot-Assisted System for Vascular Interventional Surgery

Abstract: The challenge of vascular interventional surgery is that surgeons require to be exposed to X-ray for a long time, operating guidewires and catheters to complete the treatment. To reduce the burden of the surgeons, it is of great significance to develop a tactile sensing robot-assisted system for vascular interventional surgery. Therefore, a slave manipulator with the function of collaborative operating guidewires and catheters was developed to replace doctors to perform the surgery in the operating room. In addition, a master manipulator based on magnetorheological fluids was located on the master side, and the haptic force feedback of the system was realized by generating the tactile force acting on the doctor’s hand. To verify the proposed system, a series of experiments were carried out, the results of experiments in “Vitro” indicated that the proposed system has good performance in collaborative operating and can accurately deliver a guidewire and a catheter to the target position. The maximum tracking error of the axial motion was less than 2 mm, and the maximum tracking error of the radial motion was less than 2 degrees, which is acceptable. And under the guidance of the force feedback, the safety of the system was obviously higher than that of without force feedback, after the experiment was completed by 5 participants, the safety increased by 4.32% on average. So, we can get the results that our system is feasible and effective.

Touching the audience: musical haptic wearables for augmented and participatory live music performances

Abstract: This paper introduces the musical haptic wearables for audiences (MHWAs), a class of wearable devices for musical applications targeting audiences of live music performances. MHWAs are characterized by embedded intelligence, wireless connectivity to local and remote networks, a system to deliver haptic stimuli, and tracking of gestures and/or physiological parameters. They aim to enrich musical experiences by leveraging the sense of touch as well as providing new capabilities for creative participation. The embedded intelligence enables the communication with other external devices, processes input data, and generates music-related haptic stimuli. We validate our vision with two concert-experiments. The first experiment involved a duo of electronic music performers and twenty audience members. Half of the audience used an armband-based prototype of MHWA delivering vibro-tactile feedback in response to performers’ actions on their digital musical instruments, and the other half was used as a control group. In the second experiment, a smart mandolin performer played live for twelve audience members wearing a gilet-based MHWA, which provided vibro-tactile sensations in response to the performed music. Overall, results from both experiments suggest that MHWAs have the potential to enrich the experience of listening to live music in terms of arousal, valence, enjoyment, and engagement. Nevertheless, results showed that the audio-haptic experience was not homogeneous across participants, who could be grouped as those appreciative of the vibrations and those less appreciative of them. The causes for a lack of appreciation of the haptic experience were mainly identified as the sensation of unpleasantness caused by the vibrations in certain parts of the body and the lack of the comprehension of the relation between what was felt and what was heard. Based on the reported results, we offer suggestions for practitioners interested in designing wearables for enriching the musical experience of audiences of live music via the sense of touch. Such suggestions point towards the need of mechanisms of personalization, systems able to minimize the latency between the sound and the vibrations, and a time of adaptation to the vibrations.

Large-Scale Vision-Based Tactile Sensing for Robot Links: Design, Modeling, and Evaluation

Abstract: The sense of touch allows individuals to physically interact with and better perceive their environment. Touch is even more crucial for robots, as robots equipped with thorough tactile sensation can more safely interact with their surroundings, including humans. This article describes a recently developed large-scale tactile sensing system for a robotic link, called TacLINK, which can be assembled to form a whole-body tactile sensing robot arm. The proposed system is an elongated structure comprising a rigid transparent bone covered by continuous artificial soft skin. The soft skin of TacLINK not only provides tactile force feedback but can change its form and stiffness by inflation at low pressure. Upon contact with the surrounding environment, TacLINK perceives tactile information through the three-dimensional (3-D) deformation of its skin, resulting from the tracking of an array of markers on its inner wall by a stereo camera located at both ends of the transparent bone. A finite element model (FEM) was formulated to describe the relationship between applied forces and the displacements of markers, allowing detailed tactile information, including contact geometry and distribution of applied forces, to be derived simultaneously, regardless of the number of contacts. TacLINK is scalable in size, durable in operation, and low in cost, as well as being a high-performance system, that can be widely exploited in the design of robotic arms, prosthetic arms, and humanoid robots, etc. This article presents the design, modeling, calibration, implementation, and evaluation of the system.

Integrating Wearable Haptics and Obstacle Avoidance for the Visually Impaired in Indoor Navigation: A User-Centered Approach

Abstract: Recently, in the attempt to increase blind people autonomy and improve their quality of life, a lot of effort has been devoted to develop technological travel aids. These systems can surrogate spatial information about the environment and deliver it to end-users through sensory substitution (auditory, haptic). However, despite the promising research outcomes, these solutions have met scarce acceptance in real-world. Often, this is also due to the limited involvement of real end users in the conceptual and design phases. In this article, we propose a novel indoor navigation system based on wearable haptic technologies. All the developmental phases were driven by continuous feedback from visually impaired persons. The proposed travel aid system consists of a RGB-D camera, a processing unit to compute visual information for obstacle avoidance, and a wearable device, which can provide normal and tangential force cues for guidance in an unknown indoor environment. Experiments with blindfolded subjects and visually impaired participants show that our system could be an effective support during indoor navigation, and a viable tool for training blind people to the usage of travel aids.

Haptic and Visual Feedback Assistance for Dual-Arm Robot Teleoperation in Surface Conditioning Tasks

Abstract: Contact driven tasks, such as surface conditioning operations (wiping, polishing, sanding, etc), are difficult to program in advance to be performed autonomously by a robotic system, specially when the objects involved are moving In many applications, human-robot physical interaction can be used for the teaching, specially in learning from demonstrations frameworks, but this solution is not always available Robot teleoperation is very useful when user and robot cannot share the same workspace due to hazardous environments, inaccessible locations, or because of ergonomic issues In this sense, this article introduces a novel dual-arm teleoperation architecture with haptic and visual feedback to enhance the operator immersion in surface treatment tasks Two task-based assistance systems are also proposed to control each robotic manipulator individually To validate the remote assisted control, some usability tests have been carried out using Baxter, a dual-arm collaborative robot After analysing several benchmark metrics, the results show that the proposed assistance method helps to reduce the task duration and improves the overall performance of the teleoperation

Introducing GARMI - A Service Robotics Platform to Support the Elderly at Home: Design Philosophy, System Overview and First Results

Abstract: This letter introduces GARMI, a service robotics platform conceptualized with a focus on assisting elderly at home. GARMI is designed to provide support with household tasks, as an avatar for tactile-enabled communication and as an interface for telemedicine and emergency assistance. Its unique humanoid design features a sensor-equipped multi-modal head designed for natural human-machine communication as well as a whole-body torque-control interface for safe physical human-machine interaction. GARMI's modular software architecture and distinctive whole-body control scheme allows multimodal dynamic coupling. Additionally, every system component can actively produce as well as sense forces and can thus serve as a haptic feedback interface when interacting with its environment and users. Furthermore, GARMI is the first mobile humanoid designed with specialized use-inspired avatar stations: PARTI for dual-arm-based exoskeleton-like remote-control with force-feedback and MUCKI for transparent remote doctor-patient interaction with both audiovisual and safe haptic feedback. The specialized software architecture allows for rapid prototyping and field-testing of new behaviors for telemedicine, multi-modal interaction and autonomous physical and service assistance. Our first results reveal the potential of our use-driven force-based whole-body control mobile humanoid for daily living and telemedicine scenarios in an elderly care research facility.

MPC-Based Haptic Shared Steering System: A Driver Modeling Approach for Symbiotic Driving

Abstract: Advanced driver assistance systems (ADAS) aim to increase safety and reduce mental workload. However, the gap in the understanding of the closed-loop driver–vehicle interaction often leads to reduced user acceptance. In this article, an optimal torque control law is calculated online in the model predictive control (MPC) framework to guarantee continuous guidance during the steering task. The research contribution is in the integration of an extensive prediction model covering cognitive behavior, neuromuscular dynamics, and the vehicle-steering dynamics, within the MPC-based haptic controller to enhance collaboration. The driver model is composed of a preview cognitive strategy based on a linear-quadratic-gaussian, sensory organs, and neuromuscular dynamics, including muscle coactivation and reflex action. Moreover, an adaptive cost-function algorithm enables dynamic allocation of the control authority. Experiments were performed in a fixed-base driving simulator at Toyota Motor Europe involving 19 participants to evaluate the proposed controller with two different cost functions against a commercial lane keeping assist system as an industry benchmark. The results demonstrate the proposed controller fosters symbiotic driving and reduces driver–vehicle conflicts with respect to a state-of-the-art commercial system, both subjectively and objectively, while still improving the path-tracking performance. Summarising, this article tackles the need to blend human and ADAS control, demonstrating the validity of the proposed strategy.

Mechanical and Kinematic Design of Surgical Mini Robotic Manipulator used into SP-LAP Multi-DOF Platform for Training and Simulation

Abstract: Surgical robotics is the new age of minimally invasive procedures, for this reason, it is important to acquire fundamental skills using training and simulation. Therefore, a research study was developed to present the design of 2 robotic arms, which are used into SP-LAP training platform for remote control haptic manipulation in order to magnify natural eye-hand-instrument alignment, boost motor dexterity and minimally invasive, enhance ergonomics, ensure feasibility, safety, and patient risk reduction. The proposed medical robotic manipulator has 3 degrees of freedom (DOF), 4 links and 3 joints, which have 9 DOF in combination with the multi-DOF system, to promote the application for new surgical techniques and approaches, based on the ergonomics during triangulation. In conclusion, the analysis is focused on mechanical 3D design and kinematics. The next step of the project is to evaluate adding 2 DOF more, and also work on the mechatronic integration of sensors and actuators, and then, perform test and validation of the cyber-physical system.

Printing Multi‐Material Organic Haptic Actuators

Abstract: Haptic actuators generate touch sensations and provide realism and depth in human-machine interactions. A new generation of soft haptic interfaces is desired to produce the distributed signals over large areas that are required to mimic natural touch interactions. One promising approach is to combine the advantages of organic actuator materials and additive printing technologies. This powerful combination can lead to devices that are ergonomic, readily customizable, and economical for researchers to explore potential benefits and create new haptic applications. Here, an overview of emerging organic actuator materials and digital printing technologies for fabricating haptic actuators is provided. In particular, the focus is on the challenges and potential solutions associated with integration of multi-material actuators, with an eye toward improving the fidelity and robustness of the printing process. Then the progress in achieving compact, lightweight haptic actuators by using an open-source extrusion printer to integrate different polymers and composites in freeform designs is reported. Two haptic interfaces-a tactile surface and a kinesthetic glove-are demonstrated to show that printing with organic materials is a versatile approach for rapid prototyping of various types of haptic devices.

A soft thumb-sized vision-based sensor with accurate all-round force perception

Abstract: Vision-based haptic sensors have emerged as a promising approach to robotic touch due to affordable high-resolution cameras and successful computer-vision techniques. However, their physical design and the information they provide do not yet meet the requirements of real applications. We present a robust, soft, low-cost, vision-based, thumb-sized 3D haptic sensor named Insight: it continually provides a directional force-distribution map over its entire conical sensing surface. Constructed around an internal monocular camera, the sensor has only a single layer of elastomer over-molded on a stiff frame to guarantee sensitivity, robustness, and soft contact. Furthermore, Insight is the first system to combine photometric stereo and structured light using a collimator to detect the 3D deformation of its easily replaceable flexible outer shell. The force information is inferred by a deep neural network that maps images to the spatial distribution of 3D contact force (normal and shear). Insight has an overall spatial resolution of 0.4 mm, force magnitude accuracy around 0.03 N, and force direction accuracy around 5 degrees over a range of 0.03--2 N for numerous distinct contacts with varying contact area. The presented hardware and software design concepts can be transferred to a wide variety of robot parts.

HapticBots: Distributed Encountered-type Haptics for VR with Multiple Shape-changing Mobile Robots

Abstract: HapticBots introduces a novel encountered-type haptic approach for Virtual Reality (VR) based on multiple tabletop-size shape-changing robots. These robots move on a tabletop and change their height and orientation to haptically render various surfaces and objects on-demand. Compared to previous encountered-type haptic approaches like shape displays or robotic arms, our proposed approach has an advantage in deployability, scalability, and generalizability—these robots can be easily deployed due to their compact form factor. They can support multiple concurrent touch points in a large area thanks to the distributed nature of the robots. We propose and evaluate a novel set of interactions enabled by these robots which include: 1) rendering haptics for VR objects by providing just-in-time touch-points on the user’s hand, 2) simulating continuous surfaces with the concurrent height and position change, and 3) enabling the user to pick up and move VR objects through graspable proxy objects. Finally, we demonstrate HapticBots with various applications, including remote collaboration, education and training, design and 3D modeling, and gaming and entertainment.

Bionic Compound Eye-Inspired High Spatial and Sensitive Tactile Sensor

Abstract: Haptics plays an important role in grasping and manipulating objects for human hands and artificial robotic hands. The density of tactile receptors at the tips of the fingers is very high, reaching 140 units/cm2. However, the density of the existing tactile sensors is very low. To increase the density of the 3-D tactile sensor, a novel high-density arrayed and curved 3-D tactile sensor based on bionic compound eye with the density up to 42 units/cm2 is proposed in this article. The tactile sensor can detect both normal force and shear force since it is isotropic. The force measurement setup has been successfully built with a sensitivity of 0.000280 mN/Gray for normal force and 0.0262 N/ $\mu \text{m}$ for shear force. The sensor will provide a technical basis for force measurement of surgical robots and manipulators.

Soft Actuator Materials for Electrically Driven Haptic Interfaces

Abstract: Haptics involves human touch sensing and tactile feedback and plays a crucial role in physical interactions of humans with their environment. There is an ever‐increasing interest in development of haptic technologies, due to their role in various applications such as robotics, virtual and augmented reality, healthcare, and smart electronics. Electrically driven actuation mechanisms for soft materials like dielectric elastomer actuators (DEAs), electrohydraulic soft actuators (ESAs), ionic polymer−metal composites (IPMCs), and liquid crystal elastomers (LCEs) hold the potential for the development of the next generation of haptic feedback devices due to a variety of advantages such as light weight and compact design, untethered activation and control, large actuation strains, and distributed and localized actuation. Herein, a detailed look is taken at the advancement in material designs for these electrically driven soft actuators. A detailed analysis of the different strategies for improving the electromechanical performance of existing material systems is presented. Approaches adopted to synthesize novel material systems are explained. Advancements in compliant electrode materials for the electrically driven soft actuators are also described. The conclusion reflects on the main challenges in the field and provides perspectives on recent advancements expected to have a significant impact.

Soft Biomimetic Optical Tactile Sensing With the TacTip: A Review

Abstract: Reproducing the capabilities of the human sense of touch in machines is an important step in enabling robot manipulation to have the ease of human dexterity. A combination of robotic technologies will be needed, including soft robotics, biomimetics and the high-resolution sensing offered by optical tactile sensors. This combination is considered here as a SoftBOT (Soft Biomimetic Optical Tactile) sensor. This article reviews the BRL TacTip as a prototypical example of such a sensor. Topics include the relation between artificial skin morphology and the transduction principles of human touch, the nature and benefits of tactile shear sensing, 3D printing for fabrication and integration into robot hands, the application of AI to tactile perception and control, and the recent step-change in capabilities due to deep learning. This review consolidates those advances from the past decade to indicate a path for robots to reach human-like dexterity.

What you touch, touches you: The influence of haptic attributes on consumer product impressions

Abstract: The present research builds upon the touch literature to show that the salience of haptic product attributes related to product surface texture (smooth, rough) and weight (light weight, heavy weight) influence consumer product impressions. We propose that haptic cue congruity across texture and weight drive consumer product impressions depending on a consumer's need for touch (NFT). We show that high autotelic-NFT consumers who touch for sensory pleasure enjoyed the incongruity between smooth texture and heavy weight haptic cues, and consequently showed favorable evaluations towards exciting brands. In contrast, low autotelic-NFT consumers prefer the anticipated match between smooth texture and light weight haptic cues, which conformed to their expectation about the nature of a sophisticated brand personality. Further, we show how the interactive effect of haptic attributes and a consumer's autotelic-NFT on willingness to purchase is mediated by product personality. Implications for theory and practice are discussed.

Variable Stiffness Devices Using Fiber Jamming for Application in Soft Robotics and Wearable Haptics.

Abstract: Variable stiffness actuation has applications in a wide range of fields, including wearable haptics, soft robots, and minimally invasive surgical devices. There have been numerous design approaches to control and tune stiffness and rigidity; however, most have relatively low specific load-carrying capacities (especially for flexural loads) in the most rigid state that restricts their use in small or slender devices. In this article, we present an approach to the design of slender, high flexural stiffness modules based on the principle of fiber jamming. The proposed fiber jamming modules (FJMs) consist of axially packed fibers in an airtight envelope that transition from a flexible to a rigid beam when a vacuum is created inside the envelope. This FJM can provide the flexural stiffness of up to eight times that of a particle jamming module in the rigid state. Unlike layer jamming modules, the design of FJMs further allows them to control stiffness while bending in space. We present an analytical model to guide the parameter choices for the design of fiber jamming devices. Finally, we demonstrate applications of FJMs, including as a versatile tool, as part of a kinesthetic force feedback haptic glove and as a programmable structure.

Unscripted Retargeting: Reach Prediction for Haptic Retargeting in Virtual Reality

Abstract: Research is exploring novel ways of adding haptics to VR. One popular technique is haptic retargeting, where real and virtual hands are decoupled to enable the reuse of physical props. However, this technique requires the system to know the users' intended interaction target, or requires additional hardware for prediction. We explore software-based reach prediction as a means of facilitating responsive, unscripted retargeting. We trained a Long Short-Term Memory network on users' reach trajectories to predict intended targets. We achieved an accuracy of 81.1 % at approximately 65% of movement. This could enable haptic retargeting during the last 35% of movement. We discuss the implications for possible physical proxy locations.