Using the 4 Pillars™ Immunization Toolkit to Increase
Pneumococcal Immunizations for Older Adults: A Cluster
Randomized Trial
Richard K. Zimmerman, MD, MPH, MA
1
, Anthony E. Brown, MD, MPH
2
, Valory N. Pavlik,
PhD
2
, Krissy K. Moehling, MPH
1
, Jonathan M. Raviotta, MPH
1
, Chyongchiou J. Lin, PhD
1
,
Song Zhang, MS
1
, Mary Hawk, DrPH
3
, Shakala Kyle, MA
1
, Suchita Patel, DO
4
, Faruque
Ahmed, PhD
4
, and Mary Patricia Nowalk, PhD, RD
1
1
Department of Family Medicine (RKZ, MPN, CJL, KKM, SZ, JMR, SK), University of Pittsburgh
School of Medicine, Pittsburgh, PA
2
Department of Family Medicine, Baylor College of Medicine, Houston, TX (VNP, AEB)
3
Department of Behavioral and Community Health Sciences (MH), University of Pittsburgh
Graduate School of Public Health, Pittsburgh PA
4
Centers for Disease Control and Prevention (SP, FA), Atlanta, GA
Abstract
BACKGROUND—Quality improvement in primary care has focused on improving adult
immunization.
OBJECTIVES—Test the effectiveness of a step-by step, evidence-based guide, the 4 Pillars™
Immunization Toolkit, to increase adult pneumococcal vaccination.
DESIGN—Randomized controlled cluster trial (RCCT) in Year 1 (6/1/2013–5/31/2014) and a
pre-post study in Year 2 (6/1/2014–1/31/2015) with data analyzed in 2016. Baseline year was
6/1/2012–5/31/2013. Demographic and vaccination data were derived from de-identified EMR
extractions.
SETTING—25 primary care practices stratified by city (Houston, Pittsburgh), location (rural,
urban, suburban) and type (family medicine, internal medicine), randomized to receive the
intervention in Year 1 (n=13) or Year 2 (n=12).
PARTICIPANTS—A cohort of 18,107 patients ≥65 years at baseline with a mean age of 74.2
years; 60.7% were women, 16.5% were non-white and 15.7% were Hispanic.
INTERVENTION—The Toolkit, provider education, and one-on-one coaching of practice-based
immunization champions. Outcome measures were 23-valent pneumococcal polysaccharide
vaccine (PPSV) and pneumococcal conjugate vaccine (PCV) rates and percentage point (PP)
changes.
Corresponding Author:
Mary Patricia Nowalk, PhD, RD, Department of Family Medicine, 3518 5
th
Avenue, Pittsburgh PA, 15213,
tnowalk@pitt.edu, 412-383-2355. Alternate Corresponding Author: Richard K. Zimmerman, MD, MPH, Department of Family
Medicine, 3518 5
th
Avenue, Pittsburgh PA, 15213, zimmer@pitt.edu, 412/383-2354.
HHS Public Access
Author manuscript
J Am Geriatr Soc
. Author manuscript; available in PMC 2018 January 01.
Published in final edited form as:
J Am Geriatr Soc
. 2017 January ; 65(1): 114–122. doi:10.1111/jgs.14451.
Author Manuscript Author Manuscript Author Manuscript Author Manuscript
RESULTS—In the RCCT, all intervention and control groups had significantly higher PPSV
vaccination rates with average increases ranging from 6.5–8.7 PP (
P
<0.01). The intervention was
not related to higher likelihood of PPSV vaccination. In the Year 2 pre-post study, the likelihood of
PPSV and PCV vaccination was significantly higher in the active intervention sites than the
maintenance sites in Pittsburgh, but not in Houston.
CONCLUSION—In a randomized controlled cluster trial, both intervention and control groups
increased PPSV among adults ≥65 years. In a pre-post study, private primary care practices using
the 4 Pillars™ Immunization Toolkit significantly improved PPSV and PCV uptake compared
with practices that were in the maintenance phase of the study.
Keywords
Immunization; adults; pneumococcal polysaccharide vaccine; pneumococcal conjugate vaccine;
Tdap vaccine
INTRODUCTION
Adult immunizations have been garnering increased attention as an important area for
quality improvement for several reasons. Firstly, Healthy People 2020 goals for vaccines
given to adults have not been met. In 2013, United States’ (U.S.) pneumococcal vaccination
uptake was 21.2% among those 19–64 years with high risk conditions,
1
compared with a
goal of 60%,
2
and 59.7% among those ≥65 years,
1
compared with a goal of 90%.
2
Secondly,
the National Quality Strategy (NQS) established under the Affordable Care Act has set a
long term goal to “promote healthy living and well-being through receipt of effective clinical
preventive services across the lifespan in clinic and community settings.”
3
The Center for
Medicare and Medicaid Services has also made reporting of influenza and pneumococcal
vaccines one requirement for providers to avoid negative payment adjustments.
4
Clearly,
adult immunizations are viewed as an integral part of quality care.
Provider barriers to adult vaccination include economic barriers arising from Medicare
coverage that varies among vaccines, inconvenient vaccine storage in some medical offices,
urgent and chronic medical conditions competing with time for prevention efforts,
uncertainty about the patient’s vaccination status, missed immunization opportunities, and
lack of patient and provider reminders.
5–10
Patient-related barriers include not knowing that
a vaccine was recommended, not believing that the clinician recommended the vaccine, and
fear of vaccine side effects.
11–14
Facilitators of vaccination include standing order programs
(SOPs), prompts from the electronic medical record, use of teamwork, and longer time with
the physician.
15
Based on a comprehensive literature review, The Community Preventive Services Task Force
found that improving vaccination uptake requires behavior changes at the system, provider,
support staff, and patient levels.
16
Sustainable change requires a coordinated, multipronged,
adaptable approach; hence, the need for a practice improvement toolkit that can support
change among diverse practice cultures is evident. The 4 Pillars™ Immunization Toolkit
(Toolkit) is the product of years of health services research on the barriers to and facilitators
of adult immunizations from the provider and patient perspectives,
17,18
national and local
Zimmerman et al.
Page 2
J Am Geriatr Soc
. Author manuscript; available in PMC 2018 January 01.
Author Manuscript Author Manuscript Author Manuscript Author Manuscript
surveys,
5,6,19
and pilot studies on a toolkit to increase use of standing orders for adult
vaccines.
20
We undertook a 2-year study of 25 primary care practices to test the effectiveness of the
Toolkit for increasing uptake of pneumococcal polysaccharide vaccine (PPSV) and in year 2,
the newly recommended 13-valent pneumococcal conjugate vaccine (PCV) among adults
ages 65 and older. The purpose of this study was to report the effect of the intervention on
pneumococcal vaccination rates, percentage point differences and likelihood of
pneumococcal vaccination. The data are presented in two ways. The randomized controlled
cluster trial (RCCT) analysis compared changes in vaccine uptake in the intervention and
control groups at the end of Year 1. In Year 2, controls were crossed over into active
intervention and the Year 1 intervention groups became maintenance groups. Year 2 data
were analyzed using a pre-post design in which changes in vaccine uptake were measured
from the end of Year 1 to the end of the Year 2 intervention.
METHODS
The baseline year was 6/1/2012–5/31/2013. The RCCT took place during Year 1 (6/1/2013–
5/31-2014) and the pre-post study took place during Year 2 (6/1/2014–1/31/2015). The new
CDC recommendations for PCV were published and payment discussions with the Centers
for Medicare and Medicaid Services occurred in the fall of 2014. This trial was approved by
the Institutional Review Boards of the University of Pittsburgh, Baylor College of Medicine
and the Harris Health System.
Sample Size and Sites
Optimal Design software (University of Michigan, Version 1.77. 2006) was used to calculate
sample size for a RCCT seeking a 10–15% absolute increase in vaccination rate, and a
minimum practice size of 100 patients. A sample size of 20 clusters or sites (10 Intervention
and 10 Control practices) was determined to be necessary to achieve 80% power with an
alpha of 0.05. Eligible primary care family medicine (FM) and internal medicine (IM)
practices from a practice-based research network (PBRN) in Pittsburgh (FM Pittnet), a
clinical network in Southwestern Pennsylvania (Community Medicine, Inc.) and a PBRN of
safety net practices in Houston (SPUR-Net) were solicited for participation. When 25 sites
agreed to participate, solicitation ceased. All sites used a common electronic medical record
(EMR), EpicCare within their respective health systems.
Cluster Randomization
Cluster randomization allocates clinical practices rather than individuals to the intervention
arms.
21
Some practices had more than one site; thus each site was considered as a cluster.
Eligibility requirements included having at least 100 patients ≥18 years of age, preliminary
baseline vaccination rates for at least one adult vaccine (influenza, pneumococcal, Tdap)
<50% and a willingness to make office changes to increase vaccination rates. Participating
practices were stratified first by city (Pittsburgh or Houston), location (urban, suburban or
rural), and discipline (internal or family medicine). The practices were then randomized into
the Year 1 intervention or Year 2 intervention within strata (Figure 1). Year 2 intervention
Zimmerman et al.
Page 3
J Am Geriatr Soc
. Author manuscript; available in PMC 2018 January 01.
Author Manuscript Author Manuscript Author Manuscript Author Manuscript
sites (controls) were informed that their intervention would take place during Year 2 of the
study and were not contacted again until the next year.
Toolkit
The 4 Pillars™ Immunization Toolkit is founded on four evidence-based
16,22
key strategies:
Pillar 1 - Convenient vaccination services; Pillar 2 – Communication with patients about the
importance of immunization and the availability of vaccines; Pillar 3 - Enhanced office
systems to facilitate immunization; Pillar 4 - Motivation through an office immunization
champion (IC). Supplemental Table 1 describes some of the strategies contained in the
Toolkit. The Toolkit includes background on the importance of protecting patients against
vaccine-preventable diseases, barriers to increasing vaccination from both provider and
patient perspectives and strategies to eliminate those barriers. Practices were expected to
implement strategies from each of the 4 pillars.
In Year 1, the Toolkit was a printed and bound document, supplemented by a web-based
practice transformation dashboard. The dashboard was developed from the work of Fixsen et
al.
23
who established an empirically-based implementation framework that includes
systematic uptake, establishment, and maintenance of research findings into routine practice.
The core components include: staff selection and training on the specific evidence-based
practices, expert consultation and coaching of staff and administration, program evaluation
to assess and provide feedback, facilitative administrative supports to ensure data are used to
focus and inform decision making, and systems interventions.
Once the practice was registered, any staff member could log into the dashboard. The IC was
responsible for registering the practice and its staff members, and identifying strategies that
the practice would implement. The Toolkit provided step-by-step guidance for implementing
the strategies, and the dashboard showed the practices’ progress through the change process.
Practices could monitor their progress on graphs that reported biweekly numbers of vaccines
given. In Year 2, the Toolkit was fully digitized to a website (4pillarstoolkit.pitt.edu) that
incorporated all of the paper-based background information, as well as the dashboard
capabilities.
Interventions
The intervention was designed using Diffusion of Innovations theory,
24
and included the
Toolkit, provider education, and one-on-one coaching of the immunization champion for
each practice. One of two investigators (AEB, MPN) visited each intervention site to
introduce the study and the Toolkit, and to work with staff to develop practice-specific ideas
for implementing Toolkit strategies. Each practice identified an IC who was responsible for
updating the practice transformation dashboard (Year 1) as intervention strategies were
employed, and in Year 2 use the web-based Toolkit to guide strategy implementation and
record progress in the dashboard, assisted by the research liaison as needed. Other tasks for
the IC included participating in the biweekly telephone-call with a research liaison for
coaching, ensuring that chosen strategies were being implemented and working to maintain
motivation of the staff.
Zimmerman et al.
Page 4
J Am Geriatr Soc
. Author manuscript; available in PMC 2018 January 01.
Author Manuscript Author Manuscript Author Manuscript Author Manuscript
The Toolkit was updated with new pneumococcal vaccination recommendations during Year
2 at the end of September 2014, within a few days of their publication and each practice’s
leadership team was directly notified by email of the changes to the pneumococcal
recommendations. In November 2014, additional information was sent to the Pittsburgh
practices about local insurance coverage for PCV vaccines. Information on CMS coverage
for PCV was sent to all practices in early January 2015.
Data collection
De-identified demographic (date of birth, sex, race or ethnicity, health insurance coverage),
office visit (dates) and vaccination data (vaccines given and dates) were derived from EMR
data extractions performed by the UPMC Center for Assistance in Research using the
eRecord and from a similar data extraction from the EMR by staff of the SPUR-NET for the
Texas sites. Data extractions took place at the end of each year of the study. A longitudinal
data base was created with only those patients who were ≥65 years at baseline and who had
a visit each year during the three-year study period, creating a cohort of individuals who
would have been patients of the practice during their practice’s respective baseline
(6/1/2012–5/31-2013), control or intervention (Year 1; 6/30/2013–5/31/2014), and
intervention or maintenance years (Year 2; 6/1/2014–1/31/2015).
Statistical analyses
Descriptive analyses were performed for patient demographic characteristics (age, sex, race,
and ethnicity). Health insurance was not included in the regression analyses because all
insurances would have covered pneumococcal vaccine in those aged ≥65 years. Because of
significant differences in patient populations, size and structure of the practices in Houston
and Pittsburgh, these sites were grouped separately for analysis. Age was used as a
continuous variable. Race and ethnicity were recorded differently in each city. In Pittsburgh,
with few Hispanic patients, ethnicity was rarely recorded; hence patients were grouped by
race into white and non-white with blacks and Hispanics assigned to the non-white group
and only race data are presented and used in analysis. In Houston, with few non-Hispanic
patients, race was rarely recorded; hence only ethnicity (Hispanic and non-Hispanic) are
presented and used in analysis.
The two pneumococcal vaccines (PPSV and PCV) would typically be administered once
during the 3-year project period (PCV was a new recommendation late in Year 2). The
analytical periods were baseline: 6/1/2012–5/31/2013; Year 1: 6/1/2013–5/31/2014; and
Year 2: 6/1/2014–1/31/2015. Proportions were reported for categorical variables and means
and standard deviations were reported for continuous variables. The primary outcome
measures were the cumulative PPSV and PCV vaccination rates reported at baseline, Year 1
and Year 2. Chi-square tests were performed to test for differences in cumulative vaccination
rates at different time points.
Year 1 RCCT analyses
To determine which factors were related to PPSV vaccination while accounting for the
clustered nature of the data, Cox proportional hazard models with the robust sandwich
estimate were fitted, taking account of heterogeneity in demographic characteristics
Zimmerman et al.
Page 5
J Am Geriatr Soc
. Author manuscript; available in PMC 2018 January 01.
Author Manuscript Author Manuscript Author Manuscript Author Manuscript