Does Financial Education Impact Financial Literacy
and Financial Behavior, and If So, When?
Tim Kaiser and Lukas Menkhoff
Abstract
In a meta-analysis of 126 impact evaluation studies, we find that financial education significantly impacts fi-
nancial behavior and, to an even larger extent, financial literacy. These results also hold for the subsample of
randomized experiments (RCTs). However, intervention impacts are highly heterogeneous: financial education
is less effective for low-income clients as well as in low- and lower-middle–income economies. Specific behav-
iors, such as the handling of debt, are more difficult to influence and mandatory financial education tentatively
appears to be less effective. Thus, intervention success depends crucially on increasing education intensity and
offering financial education at a “teachable moment.”
JEL classification: D14, I21
The financial behavior of consumers and small-scale entrepren eurs is receiving increased interest.
Evidence suggests a remarkable incidence of suboptimal individual financial decisions despite the fact
that these decisions are highly relevant for ind ividual welfare. The most p rominent case of such an im -
portant financial decision in advanced economies is the amount and kind of retirem ent savings ( cf.
Duflo and Saez 2003). Studies show that u ndersaving is prevalent in m any advanced economi es and
that households tend to save in inefficient ways, indicatin g that many may be unable to cope with the
increasingly complex financial markets (e.g., Lusar di and Mitchell 2007; Choi et al. 2011; Behrman
et al. 2012; van Rooij et al. 2012). This kind o f behavior also stretches across other areas, includin g
portfolio composition (Campbell 2006; Choi et al. 20 10; Buch er-Koenen and Ziegelmeyer 2014; von
Gaudecker 201 5), excessive and o verly expensive borrowing (Sta ngo and Zinman 2009; Gathergood
2012; Agarwal and Mazumder 2013; Gerardi et al. 2013; Zinman 2015), as well as participation in
Tim Kaiser is a research associate at the University of Kiel, Germany and the German Institute for Economic Research
(DIW Berlin); his email address is tkaiser@diw.de. Lukas Menkhoff (corresponding author) is the head of department of
International Economics at the German Institute for Economic Research (DIW Berlin) and Professor of Economics at the
Humboldt-University of Berlin; his email address is lmenkhoff@diw.de. We thank the authors who responded to our
requests to provide their datasets or further details about their studies for their kind cooperation. Moreover, we appreciate
valuable comments from participants at the Research in Behavioral Finance Conference 2016 in Amsterdam, the Meta-
Analysis in Economics Research Network Colloquium 2016 in Conway, What Works Global Summit 2016 in London, the
Conference in Behavioral Economics and Financial Literacy 2016 in Barcelona, and seminar participants in Berlin, Halle,
Hamburg, Kampala, Kiel, and Vienna. In particular, we thank the editor (Eric Edmonds), three anonymous referees, Martin
Brown, Nathan Fiala, Greg Fisher, Antonia Grohmann, Roy Kouwenberg, Jochen Kluve, Andreas Lutter, Christian Martin,
Olivia Mitchell, Bob Reed, Anna Sokolova, Tom Stanley, Bertil Tungodden, Ludger Wo¨ ssmann, and Dean Yang. Research
assistance by Melanie Kru¨ ger and Iven Lu¨ tzen, and financial support by DFG through CRC TRR 190 are gratefully ac-
knowledged. An online appendix for this article can be found at The World Bank Economic Review website.
V
C
The Author 2017. Published by Oxford University Press on behalf of the International Bank for Reconstruction and Development / THE WORLD BANK.
All rights reserved. For Permissions, please e-mail: journals.permissions@oup.com
The World Bank Economic Review, 31(3), 2017, 611–630
doi: 10.1093/wber/lhx018
Article
Downloaded from https://academic.oup.com/wber/article-abstract/31/3/611/4471971
by Joint Bank/Fund Library user
on 21 November 2017
financial markets in general (van Rooij et al. 2011). Related problems arise in developing countries of-
ten with even more serious consequences as peop le are exposed to heavy s hocks without having suffi-
cient insurance or mitigation instruments (e.g., Cole et al. 2011; Dr exler e t al. 2014; Gibson et al.
2014; Sayinzoga et al. 2016). All this str ongly motivates pr ovidi ng financial education to fost er finan-
cial behavior.
In surprising contrast to this obvious motivation for financial education stands the lack of com-
pelling evidence that providing financial education is an effective policy for targeting individual fi-
nancial behavior (Hastings et al. 2013; Zinman 2015). Narrative literature reviews are inconclusive,
either emphasizing the effectiveness of education measures (e.g., Fox et al. 2005; Lusardi and
Mitchell 2014) or emphasizing the opposite (e.g., Willis 2011). Further, the two available meta-
analyses of this issue do not converge in their findings: Fernandes et al. (2014) summarize overall
unreliable effects of financial education, whereas Miller et al. (2015) show that education can be ef-
fective in targeting spe cific financial behaviors. Given t his inconclusive evidence on a most impor-
tant issue, what can we learn in order to explain the heterogeneity in findings and to make financial
education more effective?
We go beyond the extant literature and systematically code the circumstances of financial education
for our meta-analysis. This allows us to examine the determinants of a positive impact of education.
Another unique characteristic of our analysis is the focus on both objectives of financial education (i.e.,
improvements in financial literacy and financial behavior). Hence, we investigate the role of financial lit-
eracy for financial behavior in a unified setting. Finally, our study benefits from a rapidly rising field
(see fig. S1.1 in the supplemental appendix S1).
We follow the established procedures for t he meta-analysis appr oach (e.g., Lipsey and Wilson
2001). The result is a sample of 126 studies reporting 539 effect sizes. Studies targeting entrepreneurs
and exclusively measuring bus iness outcomes (s uch as reven ues) are om itted by design. We only con-
sider studies reporting about interventions, such as trainings and counseling efforts. T hus, we focus
strictly on exogenous variation in financial education and neglect works exclusively analyzing the pos-
sible impact of cross-sectional (baseline) differences in financial literacy on financial behavior. Finally,
we carefully code inter ventions as w e exami ne in d etail how financial education was delivered to the
target groups.
Our meta-analysis results in six principle findings: (i) increasing financial literacy helps. Financial ed-
ucation has a strong positive impact on financial literacy with an effect size of 0.26 (i.e., above the
threshold value of 0.20 that characterizes “small” statistical effect sizes [see Cohen 1977]). Moreover,
effects on financial literacy are positively correlated with effects on financial behavior; (ii) financial edu-
cation has a positive, measurable impact on financial behavior with an effect size of 0.09. An effect size
of 0.08 is still found under rigorous randomized experiments (RCTs); (iii) effects of financial education
depend on the target group. First, teaching low-income participants (relative to the country mean) and
target groups in low- and lower-middle–income economies has less impact, which is an obvious chal-
lenge for policymakers targeting the poor. Secon d, it appears to be challenging to impact financial be-
havior as country incomes and mean years of schooling increase, probably because high baseline levels
of general education and financial literacy cause diminishing marginal returns to additional financial ed-
ucation; (iv) success of finan cial education depends on the type of financial behavior targeted. We pro-
vide evidence that borrowing behavior may be more difficult to impact than saving behavior by
conventional financial education; (v) increasing intensity supports the effect of financial educa tion; and
(vi) the characteristics of financial education can make a difference. Mak ing financial education manda-
tory is associated with deflated effect sizes. By contrast, a positive effect is associated with providing fi-
nancial education at a “teachable moment” (i.e., when teaching is directly linked to decisions of
immediate relevance to the target group (cf. Miller et al. 2015:13).
612 Kaiser and Menkhoff
Downloaded from https://academic.oup.com/wber/article-abstract/31/3/611/4471971
by Joint Bank/Fund Library user
on 21 November 2017
Complementing these findings, the meta-a nalysis also provides interesting non-results because several
characteristics of financial education are without systematic impact on financial behavior. These include
the age and gender of participants, the setting, or the choice of intervention channel thr ough which fi-
nancial educa tion is delivered.
The findings reported above clearly motivate to implement financial education because it can posi-
tively affect financial literacy and financial behavior. However, its limited effectiveness raises two addi-
tional problems for policymakers: First, what can be done to make financial education generally more
effective? Second, as a particularly obstinate aspect of the general question raised before, how can one
reach those people who do not participate voluntarily? Problematic groups in this respect include low-
income individuals, residents of low-income countries, and all those who do not self-select into educa-
tion measures, as indicated by negative effects from mandatory courses and RCTs. For these groups, it
appears that financial education needs an improved approach to be successful. More research and expe-
rience is necessary to better identify the determinants of successful financial education (e.g., Hastings
et al. 2013).
Our study follows several earlier survey studies about financial education. Most of these studies have
a narrative character, among them widely cited works such as Fox et al. (2005), Willis (2011), Hastings
et al. (2013), and Lusardi and Mitchell (2014). This gives the authors some flexibility about selecting
and interpreting the most relevant studies. A quantitative meta-a nalysis is more rigid in approach but
has the advantag es that transparent rules of procedure ensure replicable results and that quantitative
relations can be derived. Overall, narrative surveys and meta-analyses complement each other.
We perform a meta-analysis because there are just two earlier systematic accounts of the financial ed-
ucation literature that leave much room for more research. The study by Miller et al. (2015) covers only
19 papers due to its extremely restrictive selection criteria, requiring interventions on identical outcomes.
This limits the sample sizes to about five studies and estimates per subsample, which does not allow in-
vestigating the sources of heterogeneity.
Thus, the most similar study to our work is Fernandes et al. (20 14) , which cover s 90 effect sizes from
financial educa tion reported in 77 papers. Despite an overlap of 44 percent with their sample of studies,
our research differs in four crucial ways, which explains our new results: (i) most important is that we
analyze determinants of program effectiveness in a broader way by applying respective coding; (ii) we
consider various outcomes per study (on average about four per study) and their respective effect sizes;
moreover, (iii) we cover recent and mostly randomized experiments providing evidence of effective inter-
ventions; and (iv) we cover additional studies focusing exclusively on financial literacy as the outcome
variable.
This paper is structured in seven further sections. Section I introduces our meta-analytic approach.
Section II describes our data. Section III provides first results of the meta-analysis, while section IV uses
these results to explain heterogeneity of financial education treatment effects. Robustness tests are men-
tioned in section V, and section VI concludes with policy considerations and venues for future research.
I. Meta-analytic Method
Meta-analysis is a quantitative method to synthesize findings from multiple empirical studies on the
same empirical research question. In a meta-analysis, the dependent variable is comprised of a summary
statistics reported in the primary research reports, while the explanatory variables may include charac-
teristics of the research design, the sample studied, or, in case of impact evaluations, the policy interven-
tion itself (cf. Stanley 2001: 131). Meta-analyses can provide ans wers to two specific questions (cf.
Muller 2015; Pritchett and Sandefur 2015; Vivalt 2015). First, is the combined (statistical) effect across
all studies reporting effects of similar interventions on similar outcomes significantly different from
zero? And, second, what explains heteroge neity in the reported findings?
The World Bank Economic Review 613
Downloaded from https://academic.oup.com/wber/article-abstract/31/3/611/4471971
by Joint Bank/Fund Library user
on 21 November 2017
In order to be able to aggregate summary statistics reported across heterogeneous studies, one must
standardize these statistics into a common metric. If all studies would operationalize and measure out-
comes in the same unit, meta-analysis could be performed directly using economic effect sizes (e.g., elas-
ticities or marginal effec ts) in contrast to statistical effect sizes (cf. Stanley and Doucouliagos 2012).
This, however, is rarely the case in a large sample of heterogeneous (quasi-)experimental impact
evaluations.
Thus, we use a standard approach of coding a variable capturing intervention success and impact. Our
impact measure (effect size) is the standardized mean difference (SMD) for each treatment effect estimate.
We use the bias corrected standardized mean difference (Hedges’ g) as our effect size mea sure, which is de-
fined as the mean dif ference in outcomes between the treatment (M
T
) and control (M
C
) (i.e., the treatment
effect) groups as a proportion of the pooled standard deviation (SD
p
)ofthedependentvariable:
g ¼
M
T
M
C
SD
p
(1)
with
SD
p
¼
ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
n
T
1ðÞSD
T
2
þ n
C
1ðÞSD
C
2
n
T
2
þ n
C
2
2
s
: (2)
where n
T
and SD
T
are the sample size and standard deviation of the treatment group, and n
C
and SD
C
are for the control group. Additionally, we capture the standard error of each standardized mean differ-
ence (g), which is defined as:
SE
g
¼
ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
n
T
þ n
C
n
T
n
C
þ
g
2
2ðn
T
þ n
C
Þ
s
(3)
Hedges’ g informs about the size and direction of an effect in scale-free standard deviation units. This
metric is only slightly different from other popular effect size measures in experimental impact evalua-
tions, such as Cohen’s d and Glass D (see, e.g., Banerjee et al. 2015). Hedges’ g, however, introduces mi-
nor corrections that reduce bias in the effect size estimate in cases with small sample sizes and when the
sample sizes of treatment and control groups are unequally distributed. Results are qualitatively robust
to using alternative measures or relying on (partial) correlations (cf. Lipsey and Wilson 2001).
As a rule of thumb, Cohen (1977) suggests that effect sizes smaller than 0.20 should be considered as
a “small effect”; effect sizes around 0.50 indicate a “medium effect”; while effect sizes greater than 0.80
constitute “large effects.” Where pure mean comparisons, standard deviations, and sample sizes for each
experimental outcome are not reported directly we exhaust all possibilities to calculate or estimate effect
sizes (g) and its corresponding standard error from the range of available statistical data (cf. Lipsey and
Wilson 2001).
In the estimation of summary effects of the literature, our main approach follows a full pooling least
squares meta-regression framework (e.g., Card et al. 2015). Accordingly, the financial education treat-
ment effect (g) can be explained by exogenous, observable characteristics, the impact g on an outcome i,
reported in study j is expressed as a linear function
g
ij
¼ a þ x
ij
b þ
ji
(4)
where x
ij
b is a vector of observable (exogenous) study-level covariates, such as intensity of intervention,
a is an intercept, and
ji
denotes an error-term independent from x
ij
b. We estimate our models using mul-
tiple effect sizes per study and account for heteroscedasticity by clustering standard errors at the study-
level. Reassuringly, results are not sensitive to a set of changes in estimation strategy and accounting for
publication selection bias (see section V and supplemental appendix S3).
614 Kaiser and Menkhoff
Downloaded from https://academic.oup.com/wber/article-abstract/31/3/611/4471971
by Joint Bank/Fund Library user
on 21 November 2017
II. Sample Description
This section describes the selection of studies, the extraction of effect sizes and study-level covariates,
and types of financial education programs.
Selection of Studies
We follow the established meta-analytical protocol (cf. Lipsey and Wilson 2001, Stanley 2001). This
starts with systematically searching the relevant databases, including working papers, for the following
keywords: (i) financial literacy; (ii) financial knowledge; (iii) financial education; (iv) financial capabil-
ity; and (v) combinations of these keywords with “intervention.” Moreover, we consider all records
from meta-analyses (Fernandes et al. 2014; Miller et al. 2015) and narrative literature reviews (Fox et al.
2005; Collins and O’Rourke 2010; Willis 2011; Xu and Zia 2012; Hastings et al. 2013; Blue et al. 2014;
Lusardi and Mitchell 2014). This search resulted in over 500 potentially relevant published journal
articles and over 600 results from working paper databases with some apparent overlap. We stopped
collecting studies in October 2016 (see appendix S1).
From this collection, we drop studies that do not meet our three criteria for inclusion: (i) reporting on
impacts of an exogenous educational intervention on financial literacy and/or financial behavior; (ii) provid-
ing a quantitative assessment of intervention impact that allows coding an effect size statistic (g) and its stan-
dard error; and (iii) relying on an observed counterfactual in the estimation of intervention impacts. This
selection process leads to a final sample of 126 independent intervention studies t hat report 539 effect sizes
(further details in tables S1.1 and S1.2 in the supplemental a ppendix S1). Of these, 90 studies report 349 ef-
fect sizes on financial behavior, and 67 studies report 190 effect sizes on financial literacy. Among these 90
plus 67 studies, there are 31 studies reporting effect sizes on both financial literacy and behavior.
RCTs are rare in the early years of the literature, but theirsharehasrisendramatically, with the majority of
studies conducted from 2011 onward being randomized evaluations (see fig. 1). This development in the litera-
ture is very favorable for meta-analyses because it ensures a high internal validity of research findings reported
in the primary studies and helps to clearly distinguish between selection and treatment effects.
Figure 1. Number of Studies in Our Sample by Research Design per Year
Source: Authors’ calculations based on the data source discussed in the text.
The World Bank Economic Review 615
Downloaded from https://academic.oup.com/wber/article-abstract/31/3/611/4471971
by Joint Bank/Fund Library user
on 21 November 2017
