Showing papers by "Arkansas Department of Education published in 2016"
TL;DR: A meta-analysis of the international randomized controlled trials (RCTs) evaluating the achievement effects of school vouchers has never been conducted as discussed by the authors, which is a meta-analytic consolidation of the evidence from all RCTs evaluating the participant test score effects.
Abstract: School voucher programs (a.k.a. opportunity scholarships) are scholarship programs - frequently government funded - that pay for students to attend private schools of their choice. Many private school vouchers programs have been initiated around the world with the goal of increasing the academic performance of students. Voucher programs are often viewed as a way to increase achievement and satisfaction for individual students and families, while at the same time creating competitive pressures that encourage other schools in the area to improve. Countries like Chile and India have developed extensive school voucher programs. While many studies have been conducted on school vouchers, a meta-analysis of the international randomized controlled trials (RCTs) evaluating the achievement effects of vouchers has never been conducted. This study is a meta-analytic consolidation of the evidence from all RCTs evaluating the participant test score effects of school vouchers internationally. Our search process turned up 9,443 potential studies, 19 of which ultimately were included. These 19 studies represent 11 different voucher programs. A total of 262 effect sizes are included, with a two-stage consolidation of those estimates yielding a total of 44 drawn from the last year of the studies. We have included only math and reading outcomes as other subjects are rarely reported and are difficult to compare across countries. We also differentiate between English and reading outcomes and present English results as a subcomponent of the reading effects to account for the effect of local language in the international context. Our meta-analysis indicates overall positive and statistically significant achievement effects of school vouchers that vary by subject (math or reading), location (US v. non-US), and funding type (public or private). Generally, the impacts are larger (1) for reading than for math, (2) for programs outside the US relative to those within the US, and (3) for publicly-funded programs relative to privately-funded programs.
62 citations
TL;DR: The authors found that the percentage of questions skipped during the baseline year when respondents were adolescents is a significant predictor of later-life educational attainment, net of cognitive ability, and they posit that response rates are a measure of conscientiousness, though additional research is required to determine what exact noncognitive skills are being captured by item response rates.
48 citations
TL;DR: This paper examined the experimental effects of using an LSP scholarship to enroll in a private school on student achievement in the first two years following the program's expansion and found that the use of LSP has negatively impacted both ELA and math achievement, although only the latter estimates are statistically significant.
Abstract: The Louisiana Scholarship Program (LSP) is a statewide initiative offering publicly-funded vouchers to enroll in local private schools to students in low-performing schools with family income no greater than 250 percent of the poverty line. Initially established in 2008 as a pilot program in New Orleans, the LSP was expanded statewide in 2012. This paper examines the experimental effects of using an LSP scholarship to enroll in a private school on student achievement in the first two years following the program’s expansion. Our results indicate that the use of an LSP scholarship has negatively impacted both ELA and math achievement, although only the latter estimates are statistically significant. Moreover, we observe less negative effect estimates in the second year of the program.
36 citations
TL;DR: This paper examined crime rates for young adults who experienced Milwaukee's citywide voucher program as high school students and a comparable group of their peers who had been public school students, and found a consistent statistically significant negative relationship between students that stayed in the voucher program through 12th grade and criminal activity.
Abstract: In this report we examine crime rates for young adults who experienced Milwaukee's citywide voucher program as high school students and a comparable group of their peers who had been public school students. Using unique data collected as part of a longitudinal evaluation of the program, we consider criminal activity by youth initially exposed to voucher schools and those in public schools at the same time. We also consider subsequent criminal activity by the students that stayed in the voucher program through 12th grade compared to those who were in public schools for the same period. We show that the mere exposure to private schooling through a voucher is associated with lower rates of criminal activity but the relationship is not robust to different analytic samples or measures of crime. We find a more consistent statistically significant negative relationship between students that stayed in the voucher program through 12th grade and criminal activity (meaning persistent voucher students commit fewer crimes). These results are apparent when controlling for a robust set of student demographics, test scores, and parental characteristics. We conclude that merely being exposed to private schooling for a short time through a voucher program may not have a significant impact on criminal activity, though persistently attending a private school through a voucher program can decrease subsequent criminal activity, especially for males.
31 citations
TL;DR: This paper found that more conscientious teachers are more effective at improving their student conscientiousness but not their student test scores, while traditional measures of teacher quality largely fail to capture a teacher’s ability to improve students conscientiousness, though teacher quality based on student ratings and one particular classroom observation protocol are exceptions.
Abstract: Although research has been unable to find strong links between observable teacher characteristics and a teacher’s ability to improve student achievement, it has generally not considered the role that teacher non-cognitive skills play in affecting student outcomes. In this article, we validate several novel performance-task measures of teacher conscientiousness based upon the effort that teachers exert completing a survey and use these measures to examine the role that teacher conscientiousness plays in affecting both student test scores and student non-cognitive skills. We conduct our analysis using the Measure of Effective Teaching Longitudinal Database where teachers were randomly assigned to their classrooms in the second year of the study. We exploit this random assignment to estimate causal impacts of teachers on their students’ outcomes during the second year of the MET project. We find that our survey-effort measures of teacher conscientiousness capture important dimensions of teacher quality. More conscientious teachers are more effective at improving their student conscientiousness but not their student test scores. Additional analysis suggests that traditional measures of teacher quality largely fail to capture a teacher’s ability to improve student conscientiousness, though measures of teacher quality based upon student ratings and one particular classroom observation protocol are exceptions.
26 citations
TL;DR: In this paper, a review examines the existing evidence on the impacts of Promise Programs on community development, K-12 academic achievement, and student post-secondary outcomes, concluding that Promise Programs are successful at improving housing prices, attracting residents to Promise zones, improving student K- 12 academic outcomes, and increasing postsecondary enrollment.
Abstract: This review examines the existing evidence on the impacts of Promise Programs on community development, K-12 academic achievement, and student postsecondary outcomes. Promise Programs are place-based, guaranteed college scholarships offered to all students who graduate from a certain school or district while meeting the minimum thresholds of the program. We delineate Promise Programs by their design - whether the scholarships are available to all students, are awarded based on merit, or are awarded based on need. We also note the applicability of Promise Programs - whether the funds be used at a wide range of postsecondary institutions, or if they are narrowly targeted towards certain institutions. We find suggestive evidence that Promise Programs are successful at improving housing prices, attracting residents to Promise zones, improving student K-12 academic outcomes, and increasing postsecondary enrollment. However, the number of studies examining Promise Programs remains limited, and skewed towards particular programs.
18 citations
TL;DR: This paper found that survey effort measures of character skills, in particular measures of careless answering in surveys, show great promise for being good proxy measures of relevant non-cognitive skills, such as conscientiousness, grit, self-control, and a growth mindset.
Abstract: Though researchers now are aware of the potential importance of character skills, such as conscientiousness, grit, self-control, and a growth mindset, researchers struggle to find reliable measures of these skills. In this paper, we use data collected from the Understanding America Study, a nationally representative internet panel to study the validity of innovative measures of character skills based on measures of survey effort. We believe surveys themselves can be seen as a behavioral tasks and that respondents provide meaningful information about their character skills by way of the effort they put forward on surveys. In particular, we compare measures of grit, conscientiousness and other personality traits, and growth mindset, based on self-reports, and survey effort measures of character. We study the relationship across each other and their relationship with academic and life outcomes such as income and labor-market outcomes, after controlling for cognitive ability and other relevant demographic characteristics. Our results show that survey effort measures of character skills, in particular measures of careless answering in surveys, show great promise for being good proxy measures of relevant non-cognitive skills.
17 citations
TL;DR: The authors examined the short-term effects of the Louisiana Scholarship Program (LSP) on students' non-cognitive skills and civic values and found little evidence of differences between LSP scholarship recipients and non-recipients.
Abstract: This report examines the short-term effects of the Louisiana Scholarship Program (LSP) on students’ non-cognitive skills and civic values. While a growing number of studies have evaluated K-12 school voucher programs along academic dimensions, few have focused on the development of non-cognitive skills and civic values. This study aims to address that gap by providing the first analysis of differences in self-reported measures of grit, locus of control, self esteem, and political tolerance associated with the LSP. Using results from a phone survey of applicants to the program, we find little evidence of differences between LSP scholarship recipients and non-recipients. Nevertheless, diagnostics assessing the precision of our instruments to detect differences between subjects indicate that several of the scales measuring non-cognitive skills performed poorly in our sample. Moreover, our relatively low survey response rate of 11 percent raises concerns about the representativeness of our sample. Given these issues, we caution that our results are best understood as descriptive and not necessarily conclusive: they do not represent reliable estimates of the causal impact of the LSP on student non-cognitive skills and political tolerance.
17 citations
TL;DR: Hitt et al. as mentioned in this paper studied how much of the within-country and between-country variation in PISA test scores is associated with student effort, rather than academic content knowledge.
Abstract: Policy debates in education are often framed by using international test scores, such as the Programme for International Student Assessment (PISA) The obvious presumption is that observed differences in test scores within and across countries reflect differences in cognitive skills and general content knowledge, the things which achievement tests are designed to measure We challenge this presumption, by studying how much of the within-country and between-country variation in PISA test scores is associated with student effort, rather than true academic content knowledge Drawing heavily on recent literature, we posit that our measures of student effort are actually proxy measures of relevant non-cognitive skills related to conscientiousness Completing surveys and tests takes effort and students may actually reveal something about their conscientiousness by the amount of effort they show during these tasks Our previous work, and that of others validates this claim (eg Boe, May and Boruch, 2002; Borghans and Schils, 2012; Hitt, Trivitt and Cheng, 2016; Hitt, 2016; Zamarro et al, 2016) Using parametrizations of measures of survey and test effort we find that these measures help explain between 32 and 38 percent of the observed variation in test scores across countries, while explaining only a minor share of the observed variation within countries
16 citations
TL;DR: The authors used a student-level panel data set to analyze the impacts of the Louisiana Scholarship Program (LSP) on racial segregation in public and private schools in the state of Louisiana.
Abstract: The question of how school choice programs affect the racial stratification of schools is highly salient in the field of education policy. We use a student-level panel data set to analyze the impacts of the Louisiana Scholarship Program (LSP) on racial segregation in public and private schools. This targeted school voucher program provides funding for low-income, mostly minority students in the lowest-graded public schools to enroll in participating private schools. Our analysis indicates that the vast majority (82%) of LSP transfers have reduced racial segregation in the voucher students’ former public schools. LSP transfers have marginally increased segregation in the participating private schools, however, where just 45% of transfers reduce racial segregation. In those school districts under federal desegregation orders, voucher transfers result in a large reduction in traditional public schools’ racial segregation levels and have no discernible impact on private schools. The results of this analysis provide reliable empirical evidence that parental choice actually has aided desegregation efforts in Louisiana.
15 citations
TL;DR: The authors found that Black students received more severe punishments than White students for the same infractions across the state of Arkansas and these differences were due, in large part, to school-level differences in disciplinary patterns.
Abstract: According to a 2014 report from the US Department of Education’s Office for Civil Rights, Black students represent only 15% of students across the nation, but 35% of students suspended once are Black, 44% of students suspended more than once are Black, and 36% of expelled students are Black. While these disparate disciplinary outcomes at the aggregate level are troubling, more information is needed if we are to develop a full understanding of the problem of disproportionate discipline. In particular, consequences should be attached to specific infractions and school level data are necessary to help us decipher the extent to which students in the same schools are being treated differently for the same ostensible infractions. In this study, we exploit three years of student-level discipline data from Arkansas and find that Black students received more severe punishments than White students for the same infractions across the state. These differences were due, in large part, to school-level differences in disciplinary patterns. However, even after controlling for infraction and school-fixed effects, Black students received slightly longer punishments than their White peers in the same schools.
TL;DR: This article developed a simple, powerful relationship between steady-state (SS) inequity in contributions, the percent of extra contributions to fund prior cohorts, and the SS unfunded ratio and showed that the SS degree of inequity is over 60 percent under an 80 percent funding target and over 50 percent with a one point gap between assumed and true returns.
Abstract: Generational inequity in pension funding is highly sensitive to the lax policies of 80-percent funding targets and high assumed returns to investment. I develop a simple, powerful relationship between steady-state (SS) inequity in contributions – the percent of extra contributions to fund prior cohorts – and the SS unfunded ratio. I then show how the SS unfunded ratio is governed by x-percent funding targets and the gap between assumed and true returns. The SS degree of inequity is over 60 percent under an 80 percent funding target and over 50 percent with a one point gap between assumed and true returns.
TL;DR: In this article, the authors examined the incidence of unit nonresponse in an internet panel, a relatively new, and hence understudied, approach to gathering longitudinal data, and hypothesize that personality traits, which typically remain unobserved and unmeasured in many data sets, affect the likelihood of unit notresponse.
Abstract: Unit nonresponse or attrition in panel data sets is often a source of nonrandom measurement error. Why certain individuals attrite from longitudinal studies and how to minimize this phenomenon have been examined by researchers. However, this research has typically focused on data sets collected via telephone, postal mail, or face-to-face interviews. Moreover, this research usually focuses on using demographic characteristics such as educational attainment or income to explain variation in the incidence of unit nonresponse. We make two contributions to the existing literature. First, we examine the incidence of unit nonresponse in an internet panel, a relatively new, and hence understudied, approach to gathering longitudinal data. Second, we hypothesize that personality traits, which typically remain unobserved and unmeasured in many data sets, affect the likelihood of unit nonresponse. Using data from an internet panel that includes self-reported measures of personality in its baseline survey, we find that conscientiousness and openness to experience predict the incidence of unit nonresponse in subsequent survey waves, even after controlling for cognitive ability and demographic characteristics that are usually available and used by researchers to correct for panel attrition. We also test the potential to use paradata as proxies for personality traits. Although we show that these proxies predict panel attrition in the same way as self-reported measures of personality traits, it is unclear to what extent they capture particular personality traits versus other individual circumstances related to future survey completion. Our results suggest that obtaining explicit measures of personality traits or finding better proxies for them are crucial to more fully address the potential bias that may arise as a result of panel attrition.
TL;DR: The authors found evidence for the need for additional funding in nearly all scenarios in which the program is eliminated unless at least 13.52% of current voucher users stay in private schools and pay tuition out of pocket or through other private means.
Abstract: Eliminating the Louisiana Scholarship Program has been proposed as a way to reduce state education expenditures for the upcoming fiscal year. Drawing upon Louisiana’s education funding formulas, we determine that the overall effect of removing the program will increase state education expenditures. It is true that the state would avoid $41.6 million of spending if the voucher program is eliminated. However, each current voucher student who returns to a public school increases the local district’s necessary education expenditures without increasing the local tax revenue for schools, obligating the state to provide increased funding to the district. While our results depend on which assumptions we use, our analysis generally indicates the net effect of eliminating the program is an increase in state funding to local districts. In particular, we find evidence of the need for additional funding in nearly all scenarios in which the program is eliminated unless at least 13.52% of current voucher users stay in private schools and pay tuition out of pocket or through other private means.
TL;DR: Developmental coursework at the University of Arkansas, while having mixed impacts on students, could be due to the quality of student choosing to attend the state’s flagship institution and the state's policies tied to opting-in and placing-out of developmental courses.
Abstract: High school graduation serves as an important gateway to increased professional opportunities. Not only does a high school graduate improve the national economy, a high school diploma is the key to opening the door to college. However, obtaining a high school degree does not necessarily ensure college readiness. In fact, many high school graduates are not prepared for college coursework, but still apply to and attend college in our college for all system. The class of 2013 saw only 38 percent of students test at a level considered prepared for college on the reading portion of the NAEP, but the problem is 66 percent of these students went on to enroll in college (Petrilli, 2016). In order to rectify this situation of unprepared students entering post-secondary education, colleges have implemented developmental coursework policies to prepare students for college-level coursework. Here, we add to the literature by examining the impacts of developmental coursework on students at Arkansas’s flagship public institution. We use a regression discontinuity design to examine multiple bandwidths of student-level observations for first-time enrollees from 2003-2014. The full sample includes 40,395 first-time enrollees for the time period of interest, 92 percent reported scores for each of the three ACT sections that determine recommendation for remediation. Using marginal effects to predict outcomes, we find that students recommended to developmental math courses are less likely to persist in college and less likely to graduate in 4 years after enrollment. We find that students recommended for developmental English coursework were more likely to persist into the second semester and year of college but were less likely to graduate in 4 years. We conclude that developmental coursework at the University of Arkansas, while having mixed impacts on students, could be due to the quality of student choosing to attend the state’s flagship institution and the state’s policies tied to opting-in and placing-out of developmental courses.
Posted Content•
TL;DR: The authors evaluated the effectiveness of the Arkansas Teacher Corps, an alternative teacher certification program that places teachers in high-needs schools in rural, southern Arkansas and found that ATC teachers are rated significantly higher on constructs of content knowledge, teacherstudent relationships in class, and teacher-student relationships out of class by students.
Abstract: This study is an evaluation of the Arkansas Teacher Corps, an alternative teacher certification program that places teachers in high needs schools in rural, southern Arkansas. This evaluation focuses on an intermediate goal of the organization — effective teaching practices — and uses a matching strategy to determine the effectiveness of Arkansas Teacher Corps Fellows. Data comes from third party observations and student surveys. ATC teachers are rated significantly higher on constructs of content knowledge, teacher-student relationships in class, and teacher-student relationships out of class by students. There are no significant differences between ATC and non-ATC teachers noted by observers or on other constructs measured by student surveys.
TL;DR: In this paper, a comprehensive analysis of cross-subsidies in employer contributions across all entry and exit ages is provided, showing that about two-thirds of all entrants are losers.
Abstract: Under traditional defined benefit pension plans, annual contributions are set at a uniform percentage of pay to fund accruing benefits. That normal cost rate masks wide variation in the cost of individual benefits, generating an extensive and non-transparent pattern of cross-subsidization. We provide a comprehensive analysis of cross-subsidies in employer contributions across all entry and exit ages. The gains and losses of winners and losers must add up to zero, and we explain why they do not in some previous work, which claims that nearly all teachers are winners in the California Teachers Retirement System. To the contrary, we find about two-thirds of all entrants are losers. The losers earn benefits with an average annual employer cost of 0.8 percent of pay, vs. 5.7 percent for the winners. In effect, this is a system of widely varying, but non-transparent employer matches to the employee contribution, unlike a retirement account plan with a uniform match.
TL;DR: For example, the authors found that the first cohort of high school students to experience four years of a universal ESA program in Texas would produce 749 fewer felonies and misdemeanors by the time they become 22 years old, resulting in about $7 million in benefits to society.
Abstract: Decision-makers in Texas have proposed an Education Savings Account (ESA) that would allow all families to take a fraction of their public education financing to a school of their choice. If the ESA funding amount exceeds the school tuition level, families would be able to use these funds for other educational expenses such as tutoring, textbooks, educational therapy, online learning, and college costs. While this is may be viewed as obvious benefits to individual children and their families, the impacts on society overall are less clear.We estimate the impact of the proposed ESA on criminality from 2016 to 2035. We use crime reduction estimates from our previous study of the impact of the longest-standing private school voucher program in the United States, along with existing estimates of the social costs of misdemeanors and felonies, in order to monetize and forecast impacts for the ESA in Texas. We find that a universally-accessible ESA could have large benefits to the state of Texas through reduced crime over the first 20 years of the program. Specifically, we estimate that the first cohort of high school students to experience four years of a universal ESA program in Texas would produce 749 fewer felonies and misdemeanors by the time they become 22 years old, resulting in about $7 million in benefits to society by 2025. The cumulative social benefits would amount to $74 million by the end of 2030 and $194 million by the end of 2035.
TL;DR: In this article, the authors analyze whether or not parents get what they choose for when given the opportunity to choose a private school, using data from the Milwaukee Parental Choice Program (MPCP).
Abstract: Providing parents choices in education has become an increasingly popular instrument for reforming education in the United States. While existing research on parent satisfaction in private school choice programs shows that parents are satisfied with the schools they have chosen, there is not much to explain their satisfaction. Previous research using parent surveys asks parents to rate and/or grade their school of choice, while comparing their response to their thoughts on their previous public school. This paper reports new empirical evidence that looks to offer a possible explanation for parents’ satisfaction. Using data from the Milwaukee Parental Choice Program, we look to analyze whether or not parents get what they choose for when given the opportunity to choose a private school. Our analysis makes use of survey responses from parents that can be matched to students and then matched to principals. In total, there were 7,338 parents who received a survey. Of these, 3,226 parents completed a survey. In total, there were 1,868 students who responded to surveys. Parents were matched to MPCP students using a unique child ID, resulting in 1,856 parents who were matched to students. These were then matched to principals representing 123 schools participating in the MPCP. Our analysis of the MPCP examines the probability of a parent choosing a school that ranked at least above average on the specific characteristic they had listed as most important to their school choice. Since a school having a specific characteristic is a binary variable, we used Logit as the functional form of the regression equation in order to estimate the probability that parents get what they choose for.
Posted Content•
TL;DR: This paper compared the performance of different methods to correct for differential attrition in the district dataset, as well as conducted simulation analyses to assess the sensitivity of bounding methods to their underlying assumptions.
Abstract: In randomized controlled trials, it is common for attrition rates to differ by lottery status, jeopardizing the identification of causal effects. Inverse probability weighting methods (Hirano et al, 2003; Busso et al., 2014) and estimation of informative bounds for the treatment effects (e.g. Lee, 2009; Angrist et al., 2006) have been used frequently to deal with differential attrition bias. This paper studies the performance of various methods by comparing the results using two datasets: a district-sourced dataset subject to considerable differential attrition, and an expanded state-sourced dataset with much less attrition, differential and overall. We compared the performance of different methods to correct for differential attrition in the district dataset, as well as we conducted simulation analyses to assess the sensitivity of bounding methods to their underlying assumptions. In our application, methods to correct differential attrition induced bias, whereas the unadjusted district level results were closer and more substantively similar to the estimated effects in the benchmark state dataset. Our simulation exercises showed that even small deviations from the underlying assumptions in bounding methods proposed by Angrist et al. (2006) increased bias in the estimates. In practice, researchers often do not have enough information to verify the extent to which these underlying assumptions are met, so we recommend using these methods with caution.
01 Jan 2016
TL;DR: An overview of school psychology from both professional and practice perspectives can be found in this paper, where the roles, functions, and practices of effective school psychologists are described and discussed in the context of effective schooling.
Abstract: While there are currently 14 specialty areas recognized by the American Psychological Association (APA), School Psychology was one of the four specialties originally categorized within the “profession of psychology” by APA (the others were Organizational Psychology, Clinical Psychology, and Counseling Psychology). While there is a common science of psychology undergirding all four of these original professional specialty areas, school psychology focuses on the application of social and organizational, educational and learning, child and adolescent, normal and abnormal, and biological and ecological psychology to the school and schooling process. This entry provides an overview of school psychology from both professional and practice perspectives. In the context of effective school and schooling, the roles, functions, and practices of effective school psychologists are described and discussed.
TL;DR: This study draws upon prior research, Texas demographics, and conservative assumptions to forecast that the launch of a universal private school choice program in the form of Education Savings Accounts in the fall of 2017 would generate 11,809 additional high school graduates in the Lone Star State by 2022.
Abstract: Graduating from high school is a crucial outcome for young people. Unfortunately, 12 percent of Texas students fail to earn the vital credential of a high school diploma. Private school choice has a proven track record of increasing graduation rates. In this study I draw upon prior research, Texas demographics, and conservative assumptions to forecast that the launch of a universal private school choice program in the form of Education Savings Accounts in the fall of 2017 would generate 11,809 additional high school graduates in the Lone Star State by 2022. In other words, of those students attending high school today, or entering high school within a year, an additional 11,809 students will graduate with the passage of ESAs in Texas. This is a conservative projection and the numbers will also grow over time.