Showing papers by "Arkansas Department of Education published in 2015"

Finance for All: The Impact of Financial Literacy Training in Compulsory Secondary Education in Spain

Abstract: We estimate the impact on objective measures of financial literacy of a 10-hour financial education program among 15-year-old students in compulsory secondary schooling. We use a matched sample of students and teachers in Madrid and two different estimation strategies. Firstly, we use reweighting estimators to compare the performance in a test of financial knowledge of students in treatment and control schools. In another specification, we use school fixed-effect estimates of the effect of the course on changes in scores in tests of financial knowledge. The program increased treated students’ financial knowledge by between one-fourth and one-third of a standard deviation. We uncover heterogeneous effects, as students in private schools did not increase their knowledge much, possibly owing to a less intensive implementation of the program. Secondly, we analyze the bias that arises because the set of schools that participate in financial literacy programs is not random. Such selection bias is estimated as the pre-program performance in financial PISA of students in applicant schools relative to a nationally representative sample of schools. We then study whether estimators that condition on school and parental characteristics mitigate selection bias.

When You Say Nothing at All: The Predictive Power of Student Effort on Surveys

Abstract: Character traits and noncognitive skills are important for human capital development and long-run life outcomes. Research in economics and psychology now shows this clearly. But research into the exact determinants of noncognitive skills have been slowed by a common data limitation: most large-scale datasets do not contain adequate measures of noncognitive skills.This is a particularly acute problem in education policy evaluation. We demonstrate that there are important latent data within any survey dataset that can be used as proxy measures of noncognitive skills. Specifically, we examine the amount of conscientious effort that students exhibit on surveys, as measured by their item response rates. We use six nationally representative, longitudinal surveys of American youth. We find that the percentage of questions left unanswered during the baseline year, when respondents were adolescents, is a significant predictor of later-life outcomes. Respondents with higher item response rates are more likely to attain higher levels of education. The pattern of findings gives compelling reasons to view item response rates as a promising behavioral measure of noncognitive skills for use in future research in education. We posit that response rates are a partial measure of conscientiousness, though additional research from the field of psychology is required to determine what exact noncognitive skills are being captured by item response rates.

Conducting Experiments in Public Management Research: A Practical Guide

Abstract: This article provides advice on how to meet the practical challenges of experimental methods within public management research. We focus on lab, field, and survey experiments. For each of these types of experiments we outline the major challenges and limitations encountered when implementing experiments in practice and discuss tips, standards, and common mistakes to avoid. The article is multi-authored in order to benefit from the practical lessons drawn by a number of experimental researchers.

Does Retirement Impact Health Care Utilization

Abstract: The objective of this paper is to estimate the causal effect of retirement on health care utilization. To do so, we use data from the 1992-2008 waves of the Health and Retirement Study (HRS) and the 2004-2006 waves of the Survey of Health, Aging, and Retirement in Europe (SHARE). In particular, we estimate the causal impact of retirement on health care utilization as measured by: doctor visits, visits to a general practitioner, nights in the hospital, and preventative care use. This paper uses panel data and instrumental variable methods, exploiting variation in statutory retirement ages across countries, to estimate the causal effects. Cross-country comparisons allow us to examine the role of a health care system’s use of the general practitioner as a gate keeper to specialists in this relationship. We find that while retirement is associated with increased health care use, our causal estimates show that retirement leads to fewer doctor visits in both the US and continental Europe. Nights in the hospital are unaffected by retirement status. Further we find that health care systems with primary care physicians who act as gatekeepers are particularly effective at decreasing doctor visits at retirement. Therefore, we conclude that increasing the statutory retirement age to help the solvency of the retirement systems will also increase doctor visits as individuals continue to work longer. In the US, the burden of this increased utilization will likely be borne by private insurance companies and public insurance to the extent it covers working individuals in their 60’s. European evidence suggests that this increase in doctor visits due to delayed retirement will be particularly evident in health systems without strong gatekeeper roles for general practitioners.

Comparisons of Student Perceptions of Teacher's Performance in the Classroom: Using Parametric Anchoring Vignette Methods for Improving Comparability

Abstract: Self-reports are an indispensable source of information in education research but might be affected by reference group bias if the frame of reference (i.e. implicit standards), used to answer the questions, differs across students. The anchoring vignettes method was introduced, in other areas of social science, precisely to correct for this source of bias. However, studies that make use of this approach in education are rare and more research is needed to study its potential. This paper uses data from PISA 2012 to investigate the use of the parametric model of the anchoring vignettes method to correct for differential implicit standards in cross-country comparisons of student’s perceptions of an important dimension of teacher quality: teacher’s classroom management. Our results show significant heterogeneity in implicit standards across countries. We also show how correlations between countries’ average teacher classroom management levels and external variables can be improved substantially when heterogeneity in implicit standards is adjusted for. We conclude that the anchoring vignettes method shows a good potential to enhance the validity and comparability of self-reported measures in education.

Effects of Dual-Language Immersion on Students’ Academic Performance

Abstract: Using data from seven cohorts of language immersion lottery applicants in a large, urban school district, we estimate the causal effects of immersion on students’ test scores in reading, mathematics, and science, and on English learners’ (EL) reclassification. We estimate positive intent-to-treat (ITT) effects on reading performance in fifth and eighth grades, ranging from 13 to 22 percent of a standard deviation, reflecting 7 to 9 months of learning. We find little benefit in terms of mathematics and science performance, but also no detriment. By sixth and seventh grade, lottery winners’ probabilities of remaining classified as EL are three to four percentage points lower than those of their counterparts. This effect is stronger for ELs whose native language matches the partner language.

The Intergenerational Transmission of Noncognitive Skills and Their Effect on Education and Employment Outcomes

Abstract: We use information on second-generation migrants to study the existence of a cultural component on the formation process of noncognitive skills and its effect on education and employment outcomes. Our measures of noncognitive skills include: personality traits that children are encouraged to learn and civic capital. Individuals whose cultural heritage places a lower value on child qualities positively associated to the conscientiousness personality factor report lower education, worse occupational status and lower wages on average. Individuals with a higher inherited civic capital declare a higher educational level, but we find no effect of civic capital on adult labor market outcomes.

How Does Retirement Impact Health Behaviors? An International Comparison

Abstract: Recent work has found that retirement may lead to improvements in health, although the literature has not yet reached a consensus. This could be due to actual differences in the relationship of interest between countries or due to methodological differences between studies. The first goal of this paper is to estimate the causal impact of retirement on self-reported health using consistent estimation techniques on three harmonized longitudinal data sets, representative of the United States, England, and continental Europe. Using panel data and instrumental variable methods exploiting variation in statutory retirement ages, this paper then estimates how retirement causally affects health and health-related behaviors. We find, in all settings, retirement leads to better self-reported health, but that magnitude of the effect varies considerably. We also find that retirement increases the amount of exercise for those retiring from nonphysical jobs in all settings. The effect of retirement on addictive behaviors (drinking and smoking) was more mixed across settings. These findings suggest that public health interventions targeted to get near retirees to exercise more could allow countries to reap the benefits of a longer-working life while minimizing the associated health decline.

Non-Cognitive Abilities and Spanish Regional Differences in Student Performance in PISA 2009

Abstract: The goal of this paper is to analyze the role that non-cognitive skills and, in particular, regional differences in those skills, play on the observed differences in 15-year-old student’s academic performance, across Spanish regions, on PISA 2009. Previous research has shown the relevance of differences in student’s personal, family and school characteristics in accounting for academic differences across Spanish regions but it has also found that a sizeable part of the observed differences remained unexplained. We have found that differences in the distribution of certain non-cognitive skills associated to academic performance like focus, perseverance and resilience play a prominent role in accounting for differences in student performance in PISA 2009. We observe these skills by developing new measures of student effort on standardized tests. In particular, our estimates suggest that a standard deviation reduction in the dispersion of non-cognitive skills across Spanish regions would lead to a 25% reduction in the magnitude of the observed differences in student performance across regions. This is a relevant effect as, for example, a one standard deviation reduction in the regional dispersion of parent’s educational levels or occupational status would only lead to at most a 2% reduction in the magnitude of observed differences in performance on PISA across Spanish regions. Put plainly, a substantial portion of the regional variation in test scores appears attributable to effort on the PISA test, and not necessarily just differences in actual knowledge.

How can we Accurately Measure whether Students are Gaining Relevant Outcomes in Higher Education

Abstract: The main objective of this study is to empirically test a number of theory-based models (i.e. fixed effects (FE), random effects (RE), and aggregated residuals (AR)) to measure both, the generic knowledge as well as the degree attainment rates and early labor outcomes, gained by students in different programs and institutions in higher education. There are four main findings: First, the results of the paper confirm the need of using models that address the issue of student selection into programs and institutions in order to avoid biased estimates. Second, our findings provide suggestive evidence in favor of using FE models. Third, the results also illustrate the need to use appropriate statistical corrections (e.g., Heckman type selection models) to also address the issue related to students dropping out of college. Finally, our findings confirm our hypotheses that rankings of specific college-program combinations change depending on different educational and labor outcome measures considered. This finding emphasizes the need to use complementary indicators related to the mission of the specific post-secondary institutions that are being ranked. The results of this paper illustrate the importance of validating empirical models intended to rank college-program contributions according to a number of educational and early labor market outcomes. Finally, given the sensitivity of the models to different model specifications, it is not clear that they should be used to make any high-stakes decisions in higher education. They could, however, serve as part of a broader set of indicators to support programs and colleges as part of a formative evaluation.

Assessing the Impact of Investment Shortfalls on Unfunded Pension Liabilities: The Allure of Neat, but Faulty Counterfactuals

Abstract: In this paper I provide a methodological critique of the conventional method for assessing the impact of investment shortfalls and other contributors to unfunded pension liabilities, and offer a methodologically sound replacement with substantive policy implications. The conventional method – simply summing the annual actuarial gain/loss figures over time – provides a neat, additive decomposition of the sources of the rise in the Unfunded Accrued Liability (UAL). In doing so, however, it implicitly assumes that in the counterfactual exercise, amortization would adjust dollar-for-dollar with the interest on additional UAL. That is, even if the total (and average) shortfall from covering interest is substantial, the marginal shortfall is assumed to be zero. This is not how contribution shortfalls arise under funding formulas typically used by public plans in the United States. Using the actual funding formula in the counterfactual – with contribution shortfalls on the margin -- leads to much higher estimates of the UAL impact of investment shortfalls than the conventional method. The reason is that there are large interactions over time between investment shortfalls and marginal contribution shortfalls. The conventional counterfactual implicitly assumes away these interactions. The resulting additivity is alluring, but illusory. The conventional method also leads to untenable results on other UAL-drivers. Most striking is the implication that the cumulative UAL impact of pension obligation bonds (POB’s) is no different from the initial impact of receiving the proceeds, independent of the return (actual or assumed) on those proceeds. The underlying problem with the conventional framework is that it has emerged without careful attention to the counterfactual scenarios it is meant to address. This paper provides explicit and internally consistent counterfactuals to better understand the conventional method and its flaws, as well as the reasons for using instead the actual amortization formula in the counterfactual. Mathematical methods are used to illuminate the theoretical issues that lie behind any simulations. The analytical results are illustrated empirically with an adapted version of the actuarial history of the Connecticut State Teachers’ Retirement System (CSTRS), FY00-FY14. The example is instructive because it is a highly underfunded system, notable for its high (and unreduced) assumed rate of return (8.5 percent), as well as its use of $2 billion in POB proceeds to reduce the UAL in FY08, just before the market crash.

No Excuses Charter Schools: A Meta-Analysis of the Experimental Evidence on Student Achievement

Abstract: While charter schools differ widely in philosophy and pedagogical views, the United States’s most famous urban charter schools typically use the No Excuses approach. Enrolling mainly poor and minority students, these schools feature high academic standards, strict disciplinary codes, extended instructional time, and targeted supports for low-performing students. The strenuous and regimented style is controversial amongst some scholars, but others contend that the No Excuses approach is needed to rapidly close the achievement gap. We conduct the first meta-analysis of the achievement impacts of No Excuses charter schools. Focusing on experimental studies, we find that No Excuses charter schools significantly improve math scores and reading scores. We estimate gains of 0.25 and 0.16 standard deviations on math and literacy achievement, respectively, as the effect of attending a No Excuses charter school for one year. Though the effect is large and meaningful, we offer some caveats to this finding and discuss policy implications for the United States as well as other countries.

Falling Below the Line: Minimum Subgroup Size and Special Education Enrollment

Abstract: The No Child Left Behind Act of 2001 (NCLB) brought high-stakes accountability testing into every American public school with the goal of 100 percent proficiency for all students. Making annual yearly progress (AYP) toward this proficiency goal for the total student population as well as at-risk subgroups was required in order for schools to avoid possible sanctions, such as school restructuring. In implementing NCLB, states had flexibility to determine the minimum size of these subgroups as to provide statistical reliability and accountability for as many schools as possible. If a school did not meet the state’s minimum subgroup size, the proficiency of the students in the group were not calculated as part of AYP. The subjectivity of identification along with the lack of reliability in test score results makes manipulating the subgroup of students with disabilities possible and advantageous to schools. Using data from over 1,000 Arkansas schools for the years 2004-05 to 2013-14, school-level fixed effects analyses show that falling below the minimum subgroup cutoff of 40 is associated with a 1.5 percentage point decrease in students with disabilities at the school. For every student a school is above the cutoff, there is an increase of 0.09 percentage points in special education enrollment. Possible implications are discussed.

Customer Satisfaction and Educational Outcomes: Experimental Impacts of the Market-Based Delivery of Public Education

Abstract: School choice, through private school vouchers or direct government subsidies, is a mechanism of outsourcing government services in the United States, Europe, South America, and the Pacific Rim. While extensive research exists on the effects of private school choice programs, nearly all focus on test score outcomes. Lost in the heated debates about the effectiveness of private school vouchers is substantial discussion of the effects on parental satisfaction. Drawing from a federally funded evaluation of a means-tested private school choice program in Washington, DC, we examine whether customer satisfaction is greater when education is delivered through a market-based governance structure. Because the program was oversubscribed in its early years of operation, vouchers were awarded by lottery, allowing us to experimentally determine the impacts. Our analysis reveals evidence that the program had a sustained positive impact on parental satisfaction. Moreover, positive student achievement and attainment impacts strengthen the validity of parental satisfaction as a reliable outcome measure.

Collective Bargaining and District Costs for Teacher Health Insurance: An Examination of the Data from the BLS and Wisconsin

Abstract: District costs for teachers’ health insurance are, on average, higher than employer costs for private-sector professionals. How much of this is attributable to collective bargaining? This paper examines the question using data from the National Compensation Survey (NCS) of the Bureau of Labor Statistics (BLS) and the state of Wisconsin. In addition, the impact of collective bargaining on employer costs is decomposed into the impact on total premiums and the employer’s share of those premiums. The BLS data show that unionization is associated with higher total premiums, higher employer costs, and lower employee contributions in both the public and private sectors. This suggests that the high unionization rate among teachers plays a significant role in districts' higher average cost. Varying strength of teachers unions across states also helps explain the wide variation in district costs. In states with strong unions, such as Wisconsin, prior to 2011, district insurance costs can be very expensive. It is in those states that the opportunities for district cost reduction are most promising. I examine newly available data from Wisconsin to quantify the impact of that state's 2011 change in collective bargaining law, Act 10. I find a sharp reduction in district costs from lower-cost policies and higher teacher contributions: 13 to 19 percent in the first year after Act 10, and 18 to 23 percent after the second year, relative to projected district costs.

The Effect of Public and Private Schooling on Anti-Semitism

Abstract: Most major American Jewish organizations oppose voucher and other school choice programs based in part on the fear that private, mostly religious, schools do not check the development of anti-Semitism as well as do government-operated public schools. To examine whether private and public schools differ in their effect on the emergence of anti-Semitic attitudes in adults later in life, we conducted a large survey of a nationally representative sample of adults in the United States. Subjects were asked to provide details on the type of school they attended each year between 1st and 12th grade, including whether the school was public or private, religious or secular, and whether it was affiliated with a particular religious institution. We also adapted a series of measures used by the Anti-Defamation League (ADL) to gauge people’s anti-Semitism.We find that the more people attended private school when they were younger, the more favorable their attitudes toward Jews. This finding holds even after controlling for a variety of background characteristics, including age, gender, race, childhood family religion, childhood economic circumstances, mother and father’s education, being raised by two parents, and being born in the United States. The reduction in anti-Semitism associated with private schooling is roughly as large as that produced by having parents who are college educated rather than high school dropouts. The benefit of attending private school on reducing anti-Semitism is concentrated among religiously affiliated private schools. Secular private schools are similar to secular public schools in the level of anti-Semitism among their former students. We therefore have some reason to believe that religious, mostly Christian, institutions are playing an important role in restraining anti-Semitism. The overall picture on American anti-Semitism is more worrisome than earlier research by the ADL suggests. The ADL measure of anti-Semitism asks respondents to agree or disagree with a series of 11 anti-Semitic statements. But the ADL survey failed to offer subjects neutral response options, like “don’t know” or “no opinion.” In our study, we added those options and discovered that between one-third and one-half of the subjects switched to a neutral answer. A large portion of people who the ADL would have coded as not anti-Semitic are in fact ignorant or indifferent when confronted with anti-Semitic stereotypes. Although the level of anti-Semitism uncovered in our survey remains relatively low, the situation is more concerning than earlier research would lead us to believe.If we wish to reduce anti-Semitism, major Jewish organizations may wish to reconsider their historic opposition to vouchers and other private school choice programs. Rather than posing a threat, private, especially religious, schools appear to help restrict the development of anti-Semitism.

Does Financial Literacy Contribute to Food Security

Abstract: Food insecurity, not having consistent access to adequate food for active, healthy lives for all household members is most common among low income households. However, income alone is not sufficient to explain who experiences food insecurity. This study investigates the relationship between financial literacy and food security. We find that low income households who exhibit financial literacy are less likely to experience food insecurity.