Exploring Fingers’ Limitation of Texture Density Perception on Ultrasonic Haptic Displays

TL;DR: Psychophysical experiments to investigate the perceptual threshold of individual fingers on both the right and left hand of right-handed participants using active dynamic touch for spatial period discrimination of both sinusoidal and square-wave gratings on ultrasonic haptic touchscreens indicate that users’ finger identity significantly affect the perception of both gratings in the case of one-finger exploration.

Abstract: Recent research in haptic feedback is motivated by the crucial role that tactile perception plays in everyday touch interactions. In this paper, we describe psychophysical experiments to investigate the perceptual threshold of individual fingers on both the right and left hand of right-handed participants using active dynamic touch for spatial period discrimination of both sinusoidal and square-wave gratings on ultrasonic haptic touchscreens. Both one-finger and multi-finger touch were studied and compared. Our results indicate that users’ finger identity (index finger, middle finger, etc.) significantly affect the perception of both gratings in the case of one-finger exploration. We show that index finger and thumb are the most sensitive in all conditions whereas little finger followed by ring are the least sensitive for haptic perception. For multi-finger exploration, the right hand was found to be more sensitive than the left hand for both gratings. Our findings also demonstrate similar perception sensitivity between multi-finger exploration and the index finger of users’ right hands (i.e. dominant hand in our study), while significant difference was found between single and multi-finger perception sensitivity for the left hand.

Summary

1 Introduction

• Current commercial touchscreen devices rarely provide a compelling haptic feedback to human fingers despite the use of touch as input; haptic feedback is typically limited to vibration.
• Alongside vibrotactile actuation, two techniques, electrostatic-vibration [2, 18] and electroadhesion [22] use electrostatic force generated, respectively, by applying a voltage to the screen surface or by applying DC excitation of the tactile display.
• In all of the mentioned studies only one finger (index in most cases) for texture perception of tactile surfaces has been examined; the authors are aware of no work that has contrasted finger sensitivity, nor any work that explores single versus multi-finger sensitivity.
• The authors explore the limitation of individual human fingers and different hands on texture density perception in the case of two waveform types for ultrasonicbased haptic displays.

2 Experiment

• The authors carried out a psychophysical experiment to explore the limitations of touch perception of different finger types (index, middle, etc.) in dynamic active touch.
• The authors investigated both single and multi-finger tactile explorations of sinusoidal and square-wave textures on ultrasonic-based tactile displays.
• Texture is defined as the sequence of periodic haptic feedback effects generated by a specific type of signal waveform (such as square or sine) and accordingly its specific value of spatial period and amplitude.
• The experiment conformed to the principles of the Declaration of Helsinki and a general explanation of the experimental task was given to each participant before beginning the experimental procedure.

2.1 Participants

• By design, all of the participants were right-handed.
• The total experiment time was 50-60 minutes for each participant.
• Participants wore active noise-cancelling headphones (Panasonic RP-DJS200, Japan) during the experiment, while Gaussian white noise was played at a comfortable listening level in order to prevent potential interference from external auditory cues.

2.2 Experimental set-up

• The authors used an enhanced visual-tactile actuator (E-ViTa), a tactile feedback display based on ultrasonic vibrations for haptic rendering [26] .
• E-ViTa is developed on a Banana Pi, a single-board computer (Shenzhen LeMaker Technology Co. Ltd, China) with a 1 GHz ARM Cortex-A7, dual-core CPU and 1 GB of RAM working in parallel with STM32f4 microcontroller (STMicroelectronics, France).
• The communication between the microcontroller and the single board computer is provided via the Serial Peripheral Interface (SPI) bus at 10 kHz.
• This singleboard computer is connected to a 12.5cm capacitive touchscreen (Banana-LCD 5"-TS, MAREL, China) for detecting the user's finger position on the display with a sampling frequency of 62 Hz.
• A power electronic circuit converts a 12V DC voltage source into an AC voltage, controlled in amplitude and frequency and supplied to the piezoelectric cells.

2.3 General procedure

• A one-up-one-down staircase procedure (adaptive procedure) with fixed step sizes, commonly used in psychophysics [24, 17] was used in their investigation.
• The stimuli consisted of textures with sinusoidal and square wave gratings, which were tested on all fingers of both hands.
• Sample of data collected from a single participant using 1-up 1-down staircase procedure, also known as Fig. 2.
• Turnover points are marked with red color.
• The participants were free to explore the surface as long as they wanted.

3.1 One-finger exploration

• The median values of the individual fingers' 50% perceptual thresholds for discriminating from the 100 µm reference spatial period ranged between 150 µm and 300 µm for the sinusoidal grating and from 200 µm to 350 µm for the square grating.
• Index finger was found to be more sensitive than middle finger (Wilcoxon matched-pairs signed rank test: N=60, W=594, p < 0.0001) in all conditions.
• The authors then compared them with a Wilcoxon matchedpairs signed rank test.
• 50 % psychophysical thresholds for all finger types across both types of gratings were averaged for each participants.
• The same procedure was performed to compare the psychophysical thresholds (median±IQR) between both types of gratings.

3.2 Multi-finger exploration

• Participants performed the same task by exploring the actuated surface simultaneously with four fingers (except thumb).
• The results from multi-finger exploration showed similar trends to the exploration with one finger (Fig. 6 ).
• For both, the right and left hand, the authors compared the sensitivity of the index finger, which is the exploring finger in most studies on tactile perception and was also found to be the most sensitive in their experiments (Fig. 7 ).
• For the right hand, the authors did not observe a significant difference between the two exploratory techniques for any of the two types of gratings (Wilcoxon matched-pairs signed rank test: N=15, W=39, p = 0.09 for the sinusoidal grating and N=15, W=17, p = 0.30 for the square grating).

4 Conclusion and perspective

• In this paper the authors investigated the effects and limitations of different human fingers for texture density perception for both single and multi-finger exploratory techniques and for both sine and square wave gratings of right-handed participants by leveraging ultrasonic vibration.
• Furthermore, their findings showed that there was no significant difference between the sensitivity rate of the index finger (the most sensitive single-finger type) and multi-finger tactile exploration for users' dominant hand.
• Given these results, the authors hypothesize the index finger has a major impact on the overall multi-finger sensitivity of the user's dominant hand for both types of gratings.
• Funded by European ERDF grants (IRCICA, CPER MAUVE) and ANR funding agency.

Figures

Fig. 2: Sample of data collected from a single participant using 1-up 1-down staircase procedure. Turnover points (reversals) are marked with red color. The sensory threshold was calculated by averaging the correct detected texture over the last five turnover points.

Fig. 3: The setup of our experiment in the two tactile exploration conditions

Fig. 6: Left: the 50 % psychophysical thresholds (median±IQR) when exploration was simultaneously performed with four fingers on a sinusoidal grating were compared between the right and left hand. Right: The same comparison was made for the exploration of the square grating.

Fig. 7: Comparison of the 50 % psychophysical thresholds (median±IQR) between when tactile exploration of the gratings was performed with the index finger and when it was performed with four fingers simultaneously. The comparison was made for both hands.

Fig. 4: The psychophysical threshold, computed as the 50% just noticeable difference between the comparison and reference stimuli of all finger types of both hands. The experiment was performed for sinusoidal and square virtual gratings. The finger types are as follow: (1) thumb, (2) index, (3)middle, (4) ring, (5) little. The boxplots show the median value and the error bars show the interquartile range.

Fig. 5: a) The psychophysical threshold, computed as the 50% just noticeable difference for comparing the sensory thresholds (median±IQR) of thumb,index and middle fingers for both gratings. b) 50 % psychophysical thresholds for all finger types across both types of gratings were averaged for each participants. The resulting thresholds (median±IQR) were compared between the right and left hand. c) The same procedure was performed to compare the psychophysical thresholds (median±IQR) between both types of gratings.

Fig. 1: Structure of the E-Vita ultrasonic based haptic display used in our experiment [26]

Full text

HAL Id: hal-01837910
https://hal.inria.fr/hal-01837910
Submitted on 13 Jul 2018
HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entic research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.
L’archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diusion de documents
scientiques de niveau recherche, publiés ou non,
émanant des établissements d’enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.
Exploring Fingers’ Limitation of Texture Density
Perception on Ultrasonic Haptic Displays
Farzan Kalantari, David Gueorguiev, Edward Lank, Nicolas Bremard, Laurent
Grisoni
To cite this version:
Farzan Kalantari, David Gueorguiev, Edward Lank, Nicolas Bremard, Laurent Grisoni. Exploring
Fingers’ Limitation of Texture Density Perception on Ultrasonic Haptic Displays. EuroHaptics 2018
- 11th International Conference on Haptics: Science, Technology, and Applications, Jun 2018, Pisa,
Italy. �hal-01837910�

Exploring Fingers’ Limitation of Texture Density
Perception on Ultrasonic Haptic Displays
Farzan Kalantari
1
, David Gueorguiev
2
, Edward Lank
3
, Nicolas Bremard
1
, and
Laurent Grisoni
1
1
University of Lille - Science & Technology, CNRS, Lille, France
farzan.kalantari@ed.univ-lille1.fr, nicolas.bremard@inria.fr,
laurent.grisoni@univ-lille1.fr
2
Max-Planck Institute for Intelligent Systems, Stuttgart, Germany
dgueorguiev@is.mpg.de
3
University of Waterloo, Waterloo, Canada
lank@uwaterloo.ca
Abstract. Recent research in haptic feedback is motivated by the cru-
cial role that tactile perception plays in everyday touch interactions.
In this paper, we describe psychophysical experiments to investigate the
perceptual threshold of individual fingers on both the right and left hand
of right-handed participants using active dynamic touch for spatial pe-
riod discrimination of both sinusoidal and square-wave gratings on ultra-
sonic haptic touchscreens. Both one-finger and multi-finger touch were
studied and compared. Our results indicate that users’ finger identity
(index finger, middle finger, etc.) significantly affect the perception of
both gratings in the case of one-finger exploration. We show that index
finger and thumb are the most sensitive in all conditions whereas little
finger followed by ring are the least sensitive for haptic perception. For
multi-finger exploration, the right hand was found to be more sensitive
than the left hand for both gratings. Our findings also demonstrate simi-
lar perception sensitivity between multi-finger exploration and the index
finger of users’ right hands (i.e. dominant hand in our study), while sig-
nificant difference was found between single and multi-finger perception
sensitivity for the left hand.
Keywords: Haptic display; Tactile perception; Ultrasonic vibration; Fin-
ger sensitivity; Spatial texture density
1 Introduction
Current commercial touchscreen devices rarely provide a compelling haptic feed-
back to human fingers despite the use of touch as input; haptic feedback is
typically limited to vibration. As Buxton et al. [5] investigated in 1985, flat
touchscreens need haptic feedback in order to ease end users’ common interac-
tion tasks, to enhance the efficiency of interfaces, and to increase the realism of
visual environments. Therefore, researchers have explored different technologies
to generate dynamic haptic feedback to enhance input on touchscreen devices.

2 Farzan Kalantari et al.
Within this space of dynamic haptic effects, different technologies are commonly
used. First, vibrotactile actuators such as solenoids, vibrotactile coils, and ERM
motors can be utilized for tactile rendering on touchscreens as discussed in [6].
These actuators are used presently on smartwatches, mobile phones and tablets,
but typically provide for on-or-off sensation. Alongside vibrotactile actuation,
two techniques, electrostatic-vibration [2, 18] and electroadhesion [22] use elec-
trostatic force generated, respectively, by applying a voltage to the screen sur-
face or by applying DC excitation of the tactile display. Both of these techniques
increase the friction between the finger and the interaction surface when acti-
vated, thus varying the perceived stickiness of the surface. Finally, a fourth type
of haptic feedback leverages ultrasonic vibrations to create an air-gap between
a user’s finger and the display to reduce friction when activated, a phenomenon
called the “squeeze film effect” [4, 1, 7, 28]. In the remainder of this paper, we are
particularly interested in the user’s tactile perception of the latter technology.
It is well-documented in literature that the human sense of touch has a
fundamental role in the haptic perception of different surfaces. Touch is quite
sensitive in perceiving different materials [10] and textures [16], and we leverage
this sensitivity in haptic effects by taking into account its fundamental limits
[23]. The texture perception of the human sense of touch remains a complex
phenomenon which varies between different people and is mediated by the user’s
fingers’ mechanoreceptors [21].
This complexity of touch perception has resulted in various investigations
to better understand and explore haptic perception difficulties, particularly on
tactile surfaces. Yoshioka et al. [29] have shown that the neural mechanisms
underlying texture perception of a variety of real textured surfaces and objects
differ between direct touch (through a finger) and indirect touch (through a
probe). Hughes et al. [12] investigated participants’ abilities to discriminate spa-
tial density gradients of different textures. Nefs et al. [20] measured discrimi-
nation thresholds for sinusoidal gratings using active dynamic touch and found
that amplitude differences as small as 2 µm can be detected with spatial periods
between 0.25 and 1 cm. Verrillo et al. [25] studied the relationship between vibra-
tion frequency and perceived intensity of the stimuli and showed that it obeys a
power law function with an exponent of 0.89 for frequencies under 350 Hz. Wi-
jekoon et al. [27] demonstrated that there are significant correlations between
intensity perception and signal frequency and amplitude of texture waveform for
texture perception on electrovibration haptic displays, and the highest sensitivity
was found at a frequency of 80 Hz.
In the case of tactile perception of ultrasonic haptic displays, those that
leverage the squeeze film effect, several studies have also been performed on
touch perception. Biet al. [3] studied the differential sensory thresholds for the
spatial periods of real and virtual square-wave gratings on an ultrasonic haptic
plate. Kalantari et al. [14] studied the limitation of tactile elements for texture
perception and how to optimize interaction performance of end users through
the perception of different haptic effects [15], as well as how tactile and auditory
signals can be combined to enhance the user’s spatial perception in musical

Title Suppressed Due to Excessive Length 3
interactions on ultrasonic displays [13]. Gueorguiev et al. [9, 11] investigated the
tactile perception of transient changes of different frictional signals on ultrasonic
based haptic devices.
Despite all of this work, however, in all of the mentioned studies only one
finger (index in most cases) for texture perception of tactile surfaces has been
examined; we are aware of no work that has contrasted finger sensitivity, nor any
work that explores single versus multi-finger sensitivity. Given that single-touch
interaction need not be limited to the index finger, and given the prevalence of
multi-touch as an input paradigm on touch screens, one can ask the followings:
Do we have identical texture perception among all our fingers and hands while
interacting with a haptic display? Do we have the same sensory threshold for
perceiving different kind of textures? What are the differences between the tactile
perception of one-finger and multi-finger explorations on haptic displays? In this
paper, we explore the limitation of individual human fingers and different hands
on texture density perception in the case of two waveform types for ultrasonic-
based haptic displays.
2 Experiment
We carried out a psychophysical experiment to explore the limitations of touch
perception of different finger types (index, middle, etc.) in dynamic active touch.
We investigated both single and multi-finger tactile explorations of sinusoidal and
square-wave textures on ultrasonic-based tactile displays. In this study, texture
is defined as the sequence of periodic haptic feedback effects generated by a
specific type of signal waveform (such as square or sine) and accordingly its
specific value of spatial period and amplitude. We have investigated the spatial
period of determined textures (with a constant amplitude of 1.25 µm peak to
peak) which can be accurately perceivable by participants.
The experiment conformed to the principles of the Declaration of Helsinki
and a general explanation of the experimental task was given to each participant
before beginning the experimental procedure.
2.1 Participants
Fifteen healthy volunteers (5 females) from the age of 22 to 34 with a mean age
of 28.4 (SD=3.48) took part in our experiment. By design, all of the participants
were right-handed. The total experiment time was 50-60 minutes for each par-
ticipant. Participants wore active noise-cancelling headphones (Panasonic RP-
DJS200, Japan) during the experiment, while Gaussian white noise was played
at a comfortable listening level in order to prevent potential interference from
external auditory cues.
2.2 Experimental set-up
We used an enhanced visual-tactile actuator (E-ViTa), a tactile feedback display
based on ultrasonic vibrations for haptic rendering [26]. E-ViTa is developed

4 Farzan Kalantari et al.
on a Banana Pi, a single-board computer (Shenzhen LeMaker Technology Co.
Ltd, China) with a 1 GHz ARM Cortex-A7, dual-core CPU and 1 GB of RAM
working in parallel with STM32f4 microcontroller (STMicroelectronics, France).
The communication between the microcontroller and the single board computer
is provided via the Serial Peripheral Interface (SPI) bus at 10 kHz. This single-
board computer is connected to a 12.5cm capacitive touchscreen (Banana-LCD
5"-TS, MAREL, China) for detecting the user’s finger position on the display
with a sampling frequency of 62 Hz.
Ten 14 × 6 × 0.5 mm piezoelectric cells actuate a 154 × 81 × 1.6 mm fixed glass
plate, resonating at 60750 Hz with a half wavelength of 8 mm. A power electronic
circuit converts a 12V DC voltage source into an AC voltage, controlled in am-
plitude and frequency and supplied to the piezoelectric cells. The microcontroller
synthesizes a pulse-width modulation (PWM) signal to drive a voltage inverter
that actuates the piezoceramics. The detailed structure of E-ViTa haptic display
is illustrated in figure 1.
Fig. 1: Structure of the E-Vita ultrasonic based haptic display used in our ex-
periment [26]
2.3 General procedure
A one-up-one-down staircase procedure (adaptive procedure) with fixed step sizes,
commonly used in psychophysics [24, 17] was used in our investigation. In this
procedure the stimulus level at any trial is determined by the previous response
of a participant. The 1-up-1-down staircase procedure offers the compelling ad-
vantage of reducing the total time of our experiment, since we investigate a high
number of trials and conditions for each participant.
The stimuli consisted of textures with sinusoidal and square wave gratings,
which were tested on all fingers of both hands. Tactile exploration was also
performed with the right and left hands (multi-finger exploration) for the two
types of gratings. In the latter experimental situation, the participants were
asked to use all their fingers except thumb in order to have sufficient active
region of haptic feedback on the E-ViTa 5" display. The procedure for each

Frequently asked questions

Q1. What are the contributions mentioned in the paper "Exploring fingers’ limitation of texture density perception on ultrasonic haptic displays" ?

In this paper, the authors describe psychophysical experiments to investigate the perceptual threshold of individual fingers on both the right and left hand of right-handed participants using active dynamic touch for spatial period discrimination of both sinusoidal and square-wave gratings on ultrasonic haptic touchscreens. Both one-finger and multi-finger touch were studied and compared. The authors show that index finger and thumb are the most sensitive in all conditions whereas little finger followed by ring are the least sensitive for haptic perception. 

Q2. What are the future works in "Exploring fingers’ limitation of texture density perception on ultrasonic haptic displays" ?

In the future, the authors would like to investigate if these results generalize to lefthanded users, which would confirm the importance of hand dominance. As well, a potential use of different sensory thresholds of finger types, which the authors found in their study, is to leverage these sensitivities to novel finger identification techniques on tactile displays [ 8, 19 ] in order to allow users to perform different interaction tasks. 

Q3. What is the reason for the differences in sensitivity?

While hand dominance is most likely the driving force behind variable sensitivity, it is possible that perceptual differences could result from left-right physiological differences. 

Q4. What is the advantage of the 1-up-1-down staircase procedure?

The 1-up-1-down staircase procedure offers the compelling advantage of reducing the total time of their experiment, since the authors investigate a high number of trials and conditions for each participant. 

Q5. What are the sensitive fingers for perceiving differences in spatial textures?

Their results indicate that the index and the thumb are the most sensitive fingers for perceiving differences in spatial textures for both sine and square virtual gratings; the little finger, followed by the ring, is the least sensitive for texture perception in one-finger exploration for the two types of gratings. 

Q6. What is the effect of the enhanced perception sensitivity?

This enhanced perception sensitivity permits the dominant hand to control exploratory motion in order to optimize the consistency of tactile feedback across fingers. 

Q7. What is the common type of feedback used on touchscreens?

Current commercial touchscreen devices rarely provide a compelling haptic feedback to human fingers despite the use of touch as input; haptic feedback is typically limited to vibration. 

Q8. What was the sensitive finger in the experiment?

The little finger was the least sensitive, i.e., had the highest perceptual threshold level in all conditions, and the ring finger was the second least sensitive in all conditions. 

Q9. What is the significance of the results?

In terms of implications, for designers, these results provide guidance on the need to vary haptic stimuli depending on whether the stimulus is designed for the dominant or non-dominant hand, depending on whether it is designed for the index finger or for any finger, and depending on whether it is to be a single-finger of multi-finger interaction.