A Computer Assistant for Remote Collaborative
Troubleshooting of Domestic Medical Instruments
Olivier A. Blanson Henkemans, Vanessa M.
Sawirjo, Charles A.P.G. van der Mast
Human Machine Interaction group
Delft University of Technology
Delft, the Netherlands
O.A.BlansonHenkemans@TUDelft.nl
Mark A. Neerincx, Jasper Lindenberg
TNO Defense, Security & Safety
Soesterberg, the Netherlands
Mark.Neerincx@TNO.nl
Abstract—Patients suffering from chronic illness, such as
diabetes, use various domestic instruments as part of their self-
care. For older adults, there is a need for assistance to use the
instruments adequately and to solve technical failures. Following
the eHealth concept, we designed a computer assistant for an
older adult and a technical specialist, which supports remote
collaborative troubleshooting which tailors the feedback to the
users’ needs. We evaluated two feedback styles, i.e., cooperative
and directive, in the TNO Experience lab, with older and younger
adults playing the role of patient and technical specialist,
respectively, in “failure scenarios”. Results show that most
effective troubleshooting occurs with teams consisting of a older
patient receiving cooperative feedback and a younger technical
specialist receiving directive feedback. In addition, the patient
experienced more effort than the technical specialist. Further,
different personal characteristics had moderating effects on the
evaluation of the feedback styles. Our study concluded that
different user groups require different feedback styles and that
computer assistance for remote collaborative troubleshooting will
be optimal when this feedback is personalized.
Keywords: eHealth, medical instruments, troubleshooting,
CSCW, computer assistant, personalized feedback.
I.
I
NTRODUCTION
John (58) has diabetes type II and as part of his self-care
he monitors his health with the use of domestic medical
instruments, such as a glucometer. For John, it is essential that
these instruments work accurately as his other self-care
activities, e.g., diet, exercise and medication regime, depend on
it. In case of a technical failure, John experiences difficulty
resolving the failure due to low frequency of occurrence,
complex operating manuals, and the need for accurate
operation. He generally prefers to call the help desk that
remotely supports him with the troubleshooting process.
However, John, frequently, as an older novice user, faces
challenges when communicating on the phone with a younger
technical specialist.
Like John, many patients face similar challenges. They are
dependent on domestic medical instruments to make educated
choices in their self-care [1], but experience problems
troubleshooting technical failures. Although medical
instruments are described as easy to use, previous research
finds them to be anything but [2].
Patients and the health care service can potentially benefit
from eHealth. Remote care for people at home through
information and communication technology (ICT) has multiple
benefits, such as lowering costs, aging in place, and monitoring
and assessing care needs [3]. A previous study showed that a
computer assistant can improve the troubleshooting of medical
instruments by a patient at home [4]. Following the eHealth
concept, we added functions to the assistant that support remote
collaborative troubleshooting of domestic medical instruments
(Fig. 1). The patient and technical specialist each have a
personal computer assistant that detects failures and provides
assistance through feedback and demonstration. In addition,
they mediate the communication between the patient and
technical specialist, by offering relevant information of user,
usage and context.
Figure 1. Patient and technical specialist have a personal computer assistant
that monitors the environment, supports troubleshooting and mediates
communication through verbal, textual and graphical demonstration.
To facilitate support of the patient and technical specialist,
the computer assistant has two requirements. First, it should
support troubleshooting, defined as a search for likely causes of
faults through a potentially erroneous problem space of
possible causes [5]. Second, it should support remote
collaboration [6] between older novice patients in their home
environment and younger technical specialists in their
professional environment. The latter implies it should provide
relevant information and accommodate feedback to different
users with different types of personal characteristics, such as
age, expertise and cognitive abilities [7].
In a previous study [4] we evaluated the influence of
different computer assistant feedback styles on supervision of
Research was supported by SenterNovem IOP-MMI, of the Dutch
Ministry of Economic Affairs, supporting the SuperAssist project.
individuals in complex task environments. The assistant
provided cooperative feedback, i.e., it has a coaching character,
explains and educates, and expects high participation of the
users, or directive feedback, i.e., it has an instructing character
with brief reporting and expects low participation of the user.
Results showed that the cooperative assistant was more
effective and satisfactory, whereas the directive assistant was
more efficient. Furthermore, personal characteristics, i.e.,
cognitive abilities and personality traits, proved to have a
moderating effect on how people evaluated the assistant.
Consequently, our main research question reads, is a
computer assistant applying different feedback styles suitable
for remote collaborative troubleshooting? From our earlier
findings, we formed two assumptions. First, a cooperative
feedback style is oriented towards user satisfaction and long-
term development and the directive dialogue mode is oriented
towards quick and efficient problem solving in cases of
anomalies. Thus, we expect the best team performance for a
patient receiving cooperative and a technical specialist
receiving directive feedback. Second, personal characteristics
will moderate how the users evaluated the assistant.
In medical research on human subjects, consideration of
their wellbeing should take precedent over the interest of
science. Although we are empirically studying the use of a
computer assistant and its benefit to the end user, we do not
want to inordinately hinder them [8]. In collaboration with
Leiden University Medical Center, we conducted domain, task
and scenario analyses with diabetes patients and interviewed
diabetic specialists. In the current study, we evaluated an
intelligent assistant with subjects who are not diabetics, but
resemble the prospective users. Following, a computer assistant
prototype we developed and evaluated the patients in a field
study. To validate our hypotheses, we conducted a laboratory
experiment at the TNO Experience Lab, with older adults,
playing the role of patient, and younger adults, playing the role
of technical specialist, performing activities according to
scenarios. A benefit of this Smart Home Environment is the
opportunity to facilitate natural behavior in a controlled
environment [9].
II. D
ESIGN OF
C
OMPUTER
A
SSISTANT
The computer assistants interact with the patient and the
technical specialist through a Patient and Technical Specialist
Interface, respectively. Both interfaces consist of an assistant
feedback window and chat service, which facilitates
communication between the older adult and technical
specialist. In addition, the Patient Interface displays the
interface of the medical instrument currently in use.
The patient’s assistant monitors the medical instruments,
i.e., glucometer, blood pressure meter and digital pill box. In
case of a malfunction, it notifies the technical specialist’s
assistant. The assistants mediate the communication between
patient and technical specialist. The mediation follows the
general conceptualization of troubleshooting [10], which
consists of representing the problem (assessing discrepancies
between the system’s current state and ideal state); diagnosis or
fault isolation, including exploring the problem space,
generating hypotheses, gathering information, hypothesis
evaluation and decision making; selecting, implementing and
evaluating solution options; adding experience to knowledge
database. To generate feedback, the assistant’s knowledge and
reasoning is based on the medical instruments’ operation
manual. The manual serves as reference for the assistant,
concerning the possible errors, cause, and solution. Table I
depicts the different actors and their troubleshooting activities.
TABLE I. A
CTIVITIES OF OLDER PATIENT
,
YOUNGER TECHNICAL
SPECIALIST AND COMPUTER ASSISTANTS DURING TROUBLESHOOTING PROCESS
Patient(P) P Computer
Assistant(PCA)
TS Computer
Assistant(TSCA)
Technical
Specialist(TS)
Define problem field
Observe
current
instrument
state;
Establish
problem
Monitor medical
instrument
technical data;
Establish problem;
Send overview to
P & TSCA
Receive overview
problem from
TSCA;
Present problem to
TS
Receive
overview
problem
Diagnosis
Receive
diagnosis
from PCA
Receive diagnosis
from TSCA;
Present diagnosis
to P
Provide relevant
data from
knowledge
database to TS;
Receive diagnosis
from TS;
Send diagnosis to
PCA
Start diagnosis
process;
Receive
relevant data
from
knowledge
database from
TSCA;
Send diagnosis
to TSCA
Perform compensatory actions
Receive
compensatory
actions from
PCA;
Manipulate
instrument
Receive
compensatory
actions from
TSCA;
Present
compensatory
actions to P
Check success of
compensatory
action
Provide relevant
data from
knowledge
database to TS;
Receive
compensatory
actions from TS;
Send
compensatory
actions to PCA
Start repair
process;
Receive
relevant data
from TSCA;
Establish
compensatory
actions;
Send
compensatory
actions to
TSCA
Store comments to knowledge database
Add
comments to
knowledge
database
Add process
knowledge
database
Add process
knowledge
database
Add comments
to knowledge
database
To illustrate the function of the computer assistant, Fig. 2
shows the interface of patient John experiencing technical
problems with his glucometer and Fig. 3 shows the interface of
the supporting specialist. The assistant provides relevant
information about the problem field and mediates the
communication between the patient and the technical specialist
by suggesting relevant template sentences. The patient and the
technical specialist collaboratively solve the failure. Finally, the
process will be stored in the knowledge diary. The cooperative
assistant supports the user by involving the user in the
diagnosis process, explaining step by step the failure and cause,
suggesting compensatory actions, and indicating how these
problems can be prevented. In contrast, the directive assistant
supports the user by stating the independently assessed failure
and cause and instructing the compensatory actions.
III. E
VALUATION
Our goal was to evaluate the influence of cooperative and
directive feedback styles on remote collaborative
troubleshooting between teams consisting of an older patient
and a younger technical specialist. Each team performed a
scenario where patient and technical specialist both received
cooperative feedback; a scenario where they both received
directive feedback; a scenario where the patient received
cooperative and the technical specialist received directive
feedback; a scenario where the patient received directive and
the technical specialist received cooperative feedback. To
control for transfer effects, we counter-balanced the feedback
combinations across the four scenarios.
Subjects were a sample of 16 younger and 16 older adults.
The younger adults were aged between 19 and 35 (M=22,
SD=2.68) and the older adults were aged between 42 and 80
(M=59, SD=10.56). Eight younger adults were male and eight
female. Ten older adults were male and six female. Subjects
received a small incentive for their participation in this study
which lasted 3 hours.
First, we surveyed subjects’ demographics and computer
experience and tested them on spatial ability. Also, we
measured participants’ locus of control (LOC). The locus of
control theory [11] refers to two types of people: those with a
predominantly internal locus of control (i.e., attribute events to
their own control), and those with an external locus of control
(i.e., attribute events to external circumstances).
Subjects collaboratively troubleshot medical instrument
failures remotely through their interfaces according to four
scenarios. The scenarios represent every day use of the medical
instruments and possible failures that can occur. The older
adults had to work with medical instruments. Examples of
malfunctions are low battery, a stuck lever, and an erroneously
placed pill. Subjects did not have prior knowledge about these
medical instruments, but the technical specialists were
instructed to study the operation manuals for a fixed time at the
beginning of the experiment.
During the experiment, we observed the influence of the
assistants’ feedback styles on the experienced usability,
concerning effectiveness, efficiency, and satisfaction.
Effectiveness was measured by the number of technical failures
that were solved. Efficiency was measured by logging the time
required to solve a failure and mental effort experienced.
Mental effort concerns the resources required to perform the
task and was measured using the RSME [12]. After every
scenario, subjects indicated on a scale ranging from 0 (no
effort) to 150 (extreme effort), how much effort they
experienced. Afterwards, to evaluate the satisfaction, we asked
subjects which feedback style they preferred.
To evaluate the computer assistant feedback usability, we
calculated the z-score (to compare performances on different
tasks) and conducted a repeated measure ANOVA. The team
members score similarly on objective variables effectiveness
and time and we compared the different combination of
feedback styles for each team. Team members scored
differently on subjective variable effort and we compared the
effort of the different individuals. To assess if participants had
a preference for feedback style, we performed a χ²-test. To
determine whether there was a relationship between personal
characteristics and evaluation of the assistant’s usability, we
conducted multiple regression analyses on the participants’
performance data with 6 predictors, i.e., age, gender, education
level, computer experience, spatial ability (SPAT), and locus of
control (LOC).
Figure 2. Patient Interface with computer assistant feedback frame, medical
isntrument interface and chat service.
Figure 3. Technical Specialist Interface with computer assistant feedback
frame and chat service.
IV. R
ESULTS
Results show that feedback influences effectiveness. Fig. 4
shows the means and standard error of the different feedback
combinations. Teams consisting of a patient receiving
cooperative and technical specialist receiving directive
feedback solved significantly more technical failures than the
teams consisting of a patient receiving directive and a technical
specialist receiving cooperative feedback
,
F(1,15)=2.09, p<.05.
Concerning efficiency, we studied time required and effort
experienced while solving technical malfunctions. We did not
find significant influence of different feedback combination on
the time teams required. We did find that patients significantly
experienced more effort (M=.45, SD=.24) than technical
specialists (M=-.42, SD=.22), F(1,15)=14.76, p<.05.
To evaluate the subject’s satisfaction with the feedback
styles, we surveyed their preference. Of the older adults, 11 of
the 16 subjects (69%) indicated they preferred the cooperative
feedback style, χ²(1)=4.50, p<.05. The younger adults’
preference was bisect.
Figure 4. Effectiveness measured as malfunctions solved by teams consisting
of patient (P) and technical specialist (TS ) receiving different feedback i.e.,
cooperative and directive, combinations.
Results show that personal characteristics moderated the
evaluation of the computer assistant. Table II and III list the
factors that accounted for the variance in the evaluation by the
patient and technical specialist, respectively, of the assistant’s
usability, concerning effectiveness, time, effort and preference.
TABLE II. P
RECENTAGE OF VARIANCE IN PATIENTS
’
EVALUATION OF
COMPUTER ASSISTANT
’
S USABILITY
F R² Factors (%)
Effective-
ness
F(5,10)=4.24, p<.05 68% Gender (33), Age (8),
Computer experience (12),
LOC (10), SPAT (5)
Time
F(3,12)=6.93, p<.05 66% Age (54), Education (6),
SPAT (6)
Preference
F(4,11)=4.46, p<.05 62% Spatial abilty (34), Age (11),
Education (15), LOC (4)
TABLE III. P
ERCENTAGE OF VARIANCE IN TECHNICAL SPECIALIST
’
S
EVALUATION OF COMPUTER ASSISTANT
’
S USABILITY
F R² Factors (%)
Effort
F(1,14)=3.63, p<.05 21% Education (21)
Preference
F(1,14)=4.34, p<.05 24% LOC (24)
V. D
ISCUSSION
In this study, we evaluated a computer assistant that
supports remote troubleshooting of domestic medical
instruments. The patient and technical specialist each have an
assistant that monitors their environment and provides
feedback, either cooperative or directive, according to the
general conceptualized troubleshooting process. Additionally,
it mediates the communication by offering relevant contextual
information.
In summary, teams consisting of a patient receiving
cooperative feedback and a technical specialist receiving
directive feedback were the most effective. Patients
experienced more effort than technical specialist. Concerning
satisfaction, older patients preferred the cooperative feedback
style. In addition, other personal characteristics, i.e., gender,
educational level, spatial ability, locus of control, and computer
experience, influenced the evaluation of the feedback styles.
These results correspond with earlier findings [4] showing that
the cooperative style was more effective and satisfactory and
personal characteristics have a moderating effect on evaluating
telecare technology.
These results have an implication for the design of
computer assistance for the support of remote collaborative
troubleshooting. Users with different personal characteristics
and level of technical expertise require different feedback
styles. Also, older patients experience more effort then the
younger specialists. A future design-requirement is to alleviate
this difference. Only when the feedback style is well geared to
different users, will the troubleshooting progress optimally.
In conclusion, the study displayed that the computer
assistant was usable for remote collaborative troubleshooting
technical failures that occur with domestic medical instruments.
The assistants accommodate different users by providing
different feedback styles. Consequently, the assistants have the
potential to help different users, such as older patients and
younger technical specialists collaborate successfully.
R
EFERENCES
[1] A. L. Mykityshyn, A. D. Fisk, and W. A. Rogers. “Learning to use a
home medical device: mediating age-related differences with training,”
Human Factors, 2002, vol. 44(3), pp. 354-64.
[2] W. A. Rogers, A.L. Mykityshyn, R. H. Campbell, and A. D. Fisk, “Only
3 easy steps? User-centered analysis of a “simple” medical device,”
Ergonomics in Design, 2001, vol. 9, pp. 6-14.
[3] Business Insights, “ICT Opportunities in Healthcare: Key issues, growth
prospects and market opportunities in Europe and the US,” RBI142;
2005.
[4] O. A. Blanson Henkemans, M. A. Neerincx, J. Lindenberg, C. A. P. G.
van der Mast, “SuperAssist: A User-Assistant Collaborative
Environment for the supervision of medical instrument use at home.” 1st
International Conference on Pervasive Computing Technologies for
Healthcare, 2006, Nov 29-Dec 1, Innsbruck, Austria.
[5] A. Schaafstal, J. M. Schraagen, and M. van Berlo, “Cognitive task
analysis and innovation of training: The case of structured
troubleshooting,” Human Factors, 2000, vol. 42, no. 1, pp. 75–86.
[6] P. Dourish, “Re-Space-ing Place: “Place” and “Space” Ten Years On.”
CSCW'06, 2006, November 4–8, Banff, Alberta, Canada.
[7] S. J. Czaja, N. Charness, A. D. Fisk, C. Hertzog, S.N. Nair, W.A.
Rogers, and J. Sharit, “Factors Predicting the Use of Technology:
Findings From the Center for Research and Education on Aging and
Technology Enhancement (CREATE)”. Psychology and Aging, 2006,
vol. 21(2), pp. 333–352.
[8] C. Coyle, G. Doherty, M. Matthews, and J. Sharry, “Computer in talk-
based mental health interventions,” Interacting with Computers, 2007,
vol. 19, pp. 545–562.
[9] O. A. Blanson Henkemans, K. E. Caine, W. A. Rogers, A. D. Fisk, M.
A. Neerincx, B. de Ruyter, « Medical Monitoring for Independent
Living: User-Centered Smart Home Technologies for Older Adults,” M.
Jordanova and F. Lievens, Eds. Proceedings of Med-e-Tel 2007, Apr 18-
20, Luxemburg, Luxemburg;.
[10] D. H. Jonassen, J. Howland, J. Moore, and R. M. Marra, “Learning to
solve problems with technology: A constructivist perspective,” 2nd. Ed.
Columbus, OH: Merrill/Prentice-Hall, 2003.
[11] J. Rotter, “Generalized expectancies for internal versus external control
of reinforcements,” Psychological Monographs, 1966, vol. 80, pp. 1-28.
[12] F. R. H. Zijlstra, Efficiency in work behaviour: A design approach for
modern tools, Delft: Delft University Press, 1993.