Methods Inf Med 1/2010
3 © Schattauer 2010Review Article
The Impact of CPOE Medication
Systems’ Design Aspects on Usability,
Workflow and Medication Orders
A Systematic Review
R. Khajouei
1, 2
; M. W. M. Jaspers
1
1
Department of Medical Informatics, Academic Medical Center – University of Amsterdam, Amsterdam, The Netherlands;
2
Kerman University of Medical Sciences, Kerman, Iran
Keywords
CPOE, medication systems, medication errors,
assessment-evaluation, design aspects, user-
computer interface, usability evaluation,
human factors
Summary
Objectives: To examine the impact of design
aspects of computerized physician order entry
(CPOE) systems for medication ordering on
usability, physicians’ workflow and on medi-
cation orders.
Methods: We systematically searched Pub -
Med, EMBASE and Ovid MEDLINE for articles
published from 1986 to 2007. We also evalu-
ated reference lists of reviews and relevant ar-
ticles captured by our search strategy, and the
web-based inventory of evaluation studies in
medical informatics 1982–2005. Data about
design aspects were extracted from the rel-
evant articles. Identified design aspects were
categorized in groups derived from principles
for computer screen and dialogue design and
user guidance from the International Stan-
Methods Inf Med 2010; 49: 3–19
doi: 10.3414/ME0630
received: September 17, 2008
accepted: March 3, 2009
prepublished: July 6, 2009
Correspondence to:
Reza Khajouei, MSc, J1B-121
Monique Jaspers, J1B-114–2
Department of Medical Informatics
Academic Medical Center
PO Box 22700
1105 AZ Amsterdam
The Netherlands
E-mail: r.khajouei@amc.uva.nl,
m.w.jaspers@amc.uva.nl
dard Organization, and if CPOE-specific, from
the collected data.
Results: A total of 19 papers met our inclu-
sion criteria. Sixteen studies used qualitative
evaluation methods and the rest both quali-
tative and quantitative. In total 42 CPOE de-
sign aspects were identified and categorized
in seven groups: 1) documentation and data
entry components, 2) alerting, 3) visual clues
and icons, 4) drop-down lists and menus, 5)
safeguards, 6) screen displays, and 7) auxiliary
functions.
Conclusions: Beside the range of functional-
ities provided by a CPOE system, their subtle
design is important to increase physicians’
adoption and to reduce medication errors.
This requires continuous evaluations to inves-
tigate whether interfaces of CPOE systems
follow normal flow of actions in the ordering
process and if they are cognitively easy to
understand and use for physicians. This paper
provides general recommendations for CPOE
(re)design based on the characteristics of
CPOE design aspects found.
1. Introduction
Computerized physician order entry (CPOE)
systems can have a significant im
pact on the
safety and quality of drug management,
and they have been identified as being vital
to reducing serious medical errors [1].
Studies on CPOE have shown reductions in
incomplete and inappropriate prescrip-
tions [2–6], and in adverse drug events [7],
improvements in antibiotic ordering pat-
terns [7, 8], and decreases in length of stays
and costs [9]. In contrast, evidences point
at reluctance of physicians to use CPOE
systems [10, 11], due to increasing time for
ordering, decreasing interaction with pa-
tients and nurses, and lack of integration
with workflow, reducing the ultimate suc-
cess of CPOE. Complex CPOE systems that
place heavy cognitive demands on the users
may result in suboptimal use of system fea-
tures designed to support physicians in the
medication ordering tasks [12, 13]. CPOE
interface designs that do not conform to
physicians’ task behavior and decision-
making processes may obscure the appro-
priate order entry strategy [14, 15] and, in
turn, lead to inefficient workflow and user
frustration. Moreover, poor CPOE inter-
face design induces lack of usability and
facilitates medical error and may even lead
to disaster if critical information is not
presented in an effective manner. Both
quantitative and qualitative studies have
highlighted CPOE system design flaws that
led to errors in orders. Many adverse drug
events for example resulted from poor
CPOE interface design rather than from
human error [16–18]. Thus, the design of a
CPOE medication system will influence its
ease of use and the final outcome of the
medication ordering process.
Usability is often referred to capability
of a product to be used easily. This cor-
responds with the definition of usability as
a software quality put forward by the Inter-
national Standard Organization (ISO) in
ISO/IEC 9126 [19]: “a set of attributes of
software which bear on the effort needed
for use and on the individual assessment of
such use by a stated or implied set of users”.
Based on ISO 9241 [20] usability is the
extent to which a product can be used by
specified users to achieve specified goals
with effectiveness, efficiency and satisfac-
4
Methods Inf Med 1/2010 © Schattauer 2010
R. Khajouei; M. W. M. Jaspers: The Impact of CPOE Medication Systems’ Design Aspects
system and be specified as subjective rat-
ings of (dis)comfort experienced with
CPOE use, or the extent to which system
efficiency and learnability have been
achieved.
Despite the impact of CPOE design on
the medication ordering process, no litera-
ture review has focused specifically on the
influence of CPOE design aspects on us-
ability, physicians’ workflow and medi-
cation orders. Current review studies
[22–27] on CPOE systems investigated the
effect of these systems on outcomes such as
medication safety, costs, adverse drug
events, adherence to guidelines, and work
efficiencies. Determining design aspects of
CPOE systems exerting a positive or
negative influence on system usability,
physicians’ workflow and final outcomes of
medication ordering might give clues
about how to optimize the design of these
systems to be easy to use, aligned with
physicians’ ordering processes, and effec-
tive in ordering medications.
The main objective of this study is to
answer the following research questions.
What design aspects of CPOE medication
systems influence their usability, physi -
cians’ workflow, and medication orders?
And how the design of CPOE could be
changed to improve usability, workflow
and medication ordering process? To an -
swer these questions, we reviewed the lit-
erature for studies describing original data
on a (usability) evaluation of CPOE medi-
ation systems’ design aspects. Based on the
results we provided recommendations,
benefited from principles for computer
screen and dialogue design and user guid-
ance of ISO, to enable CPOE system de-
signers to create systems that are more user-
friendly, more efficient, and safer to use.
2. Methods
We searched the literature from 1986 to
2007 using PubMed, EMBASE and Ovid
tion in a specified context of use. Physi -
cians as users of CPOE medication systems
have to accomplish a series of sequential
tasks to achieve the goal of setting out a
medication order as a part of their work-
flow. Workflow itself is a step-by-step
process including a linear sequence of ac-
tivities, to be executed by certain users, to
provide the necessary input for the next
step [21]. Effectiveness of a CPOE system
can be defined as the accuracy and com-
pleteness with which physicians achieve the
ordering of medications. Errors in medi-
cation orders affect accuracy whereas in-
complete orders influence completeness.
Efficiency can be defined as resources ex-
pended in relation to the accuracy and
completeness of a medication order. In the
context of CPOE usability, efficiency is re-
lated to the cognitive demands put on the
physician in setting out the medication
order supported by the CPOE system.
Satisfaction can be defined as the phy -
sicians’ attitudes towards using a CPOE
Fig. 1
Groups of keywords
and MeSH terms
used in the search
strategy (MeSH
terms are in bold)
5 R. Khajouei; M. W. M. Jaspers: The Impact of CPOE Medication Systems’ Design Aspects
© Schattauer 2010 Methods Inf Med 1/2010
MEDLINE for English-language publica -
tions reporting on (usability) evaluation
studies of CPOE medication systems in
both inpatient and outpatient settings. In
searching these databases, four groups of
key terms were constructed related to:
A) CPOE and Electronic Prescribing Sys-
tems, B) Computerized Patient Records,
Computer-Assisted Drug Therapy and
Pharmacy Systems, C) Medication Order-
ing, D) Evaluation Studies, Usability, and
Work fl ow.
Figure 1 shows the keywords
and
Figure 2 shows the search strategy
used to identify relevant articles. We used
these clusters of key terms in the following
4-step process to automatically retrieve as
many as possible publications on CPOE
systems for medication ordering. 1) Key
terms in each group were combined by the
operator “OR”; 2) groups A and D were
combined using “AND” to capture studies
about CPOE medication system usability,
task behavior and workflow; 3) groups B
and C were combined with “AND” to re-
trieve articles addressing CPOE medi-
cation systems not indexed by CPOE-
related keywords. We then combined the
resulting set of articles with group D; 4) re-
sults of steps 2 and 3 were added using the
“OR” operator to accumulate all of the
evaluation studies associated with usability
of computerized physician medication
order entry systems and physicians’ task
behavior or workflow. Generally we used
the following combinations in the search
strategy to extract relevant studies: (A AND
D) OR (B AND C AND D).
Two reviewers independently reviewed
and assessed titles and abstracts of the result-
Fig. 2 Search flow
6 R. Khajouei; M. W. M. Jaspers: The Impact of CPOE Medication Systems’ Design Aspects
Methods Inf Med 1/2010 © Schattauer 2010
ing papers against predefined inclusion and
exclusion criteria. First, editorials, letters,
commentaries and conceptual papers were
excluded. Articles in proceedings were ex-
cluded if a more comprehensive article of
that study was retrieved from an inter-
national journal.
Articles were selected if they reported
original data from a (usability) evaluation
study of a CPOE medication system used in
ambulatory or inpatient health care set-
tings and if they reported on effects of de-
sign aspects of CPOE on usability, phy -
sicians’ workflow and final influence on
medication orders set out. We considered
all computerized systems; both stand-alone
or integrated with other systems, used for
ordering medication for a patient, as CPOE
medication systems. Articles concerning
the use of CPOE other than for medi -
cation ordering, for example concerning
laboratory and radiology ordering systems,
retrieved in the first step, were manually
excluded. Feasibility evaluation studies on
CPOE medication systems and studies on
deployment and technical infrastructures
of CPOE medication systems not directly
related to design aspects of CPOE were
excluded. Studies that reported on the
impact of general CPOE components (such
as decision support tools), or impact of
CPOE on patient outcomes were also ex-
cluded, unless they described that certain
effects were related to specific CPOE de -
sign aspects. In the absence of an ab-
stract or when inclusion of an article could
not be decided upon on the basis of the ab-
stract, full texts of the articles were re-
viewed. We additionally evaluated refer-
ence lists of relevant articles and of re -
view articles captured by our search strat-
egy for relevant publications. We finally
searched the web-based inventory of evalu-
ation studies in medical informatics 1982
to 2005 [28] for studies not captured
by our search strategy. Any disagree -
ments between reviewers concerning the
selection of articles were resolved through
discussion. Subsequently, from the selected
articles data was extracted by two re -
viewers using a standard report form
(
Fig. 3).
Each of the design aspects found in the
articles was matched to the ISO prin -
ciples and recommendations for computer
screen- and dialogue design and user guid-
ance [29–32]. Subsequently, corresponding
information from the ISO standards was
added to the data collection form. To facili-
tate data presentation, this standard report
form was used to cluster the CPOE design
aspects into seven groups. Groups were
formed based on similarity and homogene-
ity of design aspects, and of corresponding
ISO recommendations by the consensus of
two reviewers. The groups were deter-
mined so that all extracted design aspects
could be placed in one group without any
ambiguity. Recommendations for optimiz-
ing design aspects of CPOE user interface
were articulated, using guidance and
requirements put forward by ISO for those
corresponding with the ISO recommen-
dations.
Fig. 3 Data extraction form
7 R. Khajouei; M. W. M. Jaspers: The Impact of CPOE Medication Systems’ Design Aspects
Table 1
Selected publications on evaluation of CPOE medication systems’ design aspects
Study CPOE system Method Qualitative/
Quantitative
Summative/
Formative *
Setting
1 Bradly et al., 2006
[38]
CPOE with CDS inte-
grated in EMR
Pre-post test descriptive
study
Quantitative Summative Inpatient
2 Ash et al., 2007
[16]
CPOE integrated in
HER vendor-supplied
Ethnographic study, ob-
servation and interview
Qualitative Summative Outpatient
3 Horsky et al., 2005
[43]
Commercial CPOE Think aloud method Qualitative Formative Laboratory setting
4 Banet et al., 2006
[35]
CPOE added to a com-
mercial emergency de-
partment information
system
Pre-post test repeated
time-motion studies,
questionnaire survey
Qualitative,
quantitative
Summative Inpatient, Out-
patient
5 Zhan et al., 2006
[13]
Full CPOE Analysis of medication
errors reported
Qualitative,
quantitative
Summative Inpatient , out-
patient
6 Beuscart-Zephir
et al., 2005 [37]
CPOE integrated in pa-
tient care information
system (PCIS)
Activity analysis,
heuristic evaluation,
think-aloud method
Qualitative Formative Inpatient
7 Horsky et al., 2005
[42]
– Analysis of order entry
logs, visual and cognitive
evaluation, semi struc-
tured interviews
Qualitative Formative Inpatient
8 Koppel et al., 2005
[17]
A widely used CPOE
system
Structured interviews,
real-time observations,
focus groups, question-
naire survey
Quantitative ,
qualitative
Formative –
9 Horsky et al., 2004
[12]
Development version of
a commercially
POE system, with DSS
Cognitive walkthrough,
think-aloud method
Qualitative Formative Laboratory setting
10 Cheng et al., 2003
[14]
CPOE integrated in an
EMR
Observational case study Qualitative Formative Inpatient
11 Horsky et al., 2003
[15]
A development version
of a commercially POE
system with DSS inte-
grated in an EMR
Cognitive walkthrough,
think-aloud method
Qualitative Formative Laboratory setting
12 Bates et al., 1999
[36]
CPOE with DSS, inte-
grated in home-grown
BICS (Brigham inte-
grated computing
system)
Prospective time series
analysis
Quantitative Summative Inpatient
13 Caudill-Slosberg and
Weeks, 2005 [39]
– Case study using cause-
and-effect/fishbone
analysis
Qualitative Summative Inpatient
User groups
Not applicable
Clinicians
Internal medi-
cine residents
Registered
nurses
Not applicable
Nurses and
physicians
Clinicians
House staff,
pharmacists,
nurses, nurse-
managers, at-
tending physi -
cians, and infor-
mation technol-
ogy managers
Physicians
Physicians, nurs-
ing staff, two
pharmacists,
and one respira -
tory therapist
(RT)
Internal medi-
cine physicians
Not applicable
Not applicable
© Schattauer 2010 Methods Inf Med 1/2010
