This article presents findings from a meta-analysis of 213 school-based, universal social and emotional learning (SEL) programs involving 270,034 kindergarten through high school students. Compared to controls, SEL participants demonstrated significantly improved social and emotional skills, attitudes, behavior, and academic performance that reflected an 11-percentile-point gain in achievement. School teaching staff successfully conducted SEL programs. The use of 4 recommended practices for developing skills and the presence of implementation problems moderated program outcomes. The findings add to the growing empirical evidence regarding the positive impact of SEL programs. Policy makers, educators, and the public can contribute to healthy development of children by supporting the incorporation of evidence-based SEL programming into standard educational practice.

http://explorefhp.com/wp-content/uploads/2013/03/Durlak-Weissberg-Child-Development-Meta-Analysis.pdf

The Impact of Enhancing Students’ Social and Emotional Learning: A Meta‐Analysis of School‐Based Universal Interventions

The view that the returns to educational investments are highest for early childhood interventions is widely held and stems primarily from several influential randomized trials—Abecedarian, Perry, and the Early Training Project—that point to super-normal returns to early interventions. This article presents a de novo analysis of these experiments, focusing on two core issues that have received limited attention in previous analyses: treatment effect heterogeneity by gender and overrejection of the null hypothesis due to multiple inference. To address the latter issue, a statistical framework that combines summary index tests with familywise error rate and false discovery rate corrections is implemented. The first technique reduces the number of tests conducted; the latter two techniques adjust the p values for multiple inference. The primary finding of the reanalysis is that girls garnered substantial short- and long-term benefits from the interventions, but there were no significant long-term benefits fo...

/pdf/multiple-inference-and-gender-differences-in-the-effects-of-3023a6wijx.pdf

Multiple Inference and Gender Differences in the Effects of Early Intervention: A Reevaluation of the Abecedarian, Perry Preschool, and Early Training Projects

The objective of this article is to offer guidelines regarding the selection, calculation, and interpretation of effect sizes (ESs). To accomplish this goal, ESs are first defined and their important contribution to research is emphasized. Then different types of ESs commonly used in group and correlational studies are discussed. Several useful resources are provided for distinguishing among different types of effects and what modifications might be required in their calculation depending on a study's purpose and methods. This article should assist producers and consumers of research in understanding the role, importance, and meaning of ESs in research reports.

How to Select, Calculate, and Interpret Effect Sizes

This meta-analysis reviewed 82 school-based, universal social and emotional learning (SEL) interventions involving 97,406 kindergarten to high school students (Mage = 11.09 years; mean percent low socioeconomic status = 41.1; mean percent students of color = 45.9). Thirty-eight interventions took place outside the United States. Follow-up outcomes (collected 6 months to 18 years postintervention) demonstrate SEL's enhancement of positive youth development. Participants fared significantly better than controls in social-emotional skills, attitudes, and indicators of well-being. Benefits were similar regardless of students’ race, socioeconomic background, or school location. Postintervention social-emotional skill development was the strongest predictor of well-being at follow-up. Infrequently assessed but notable outcomes (e.g., graduation and safe sexual behaviors) illustrate SEL's improvement of critical aspects of students’ developmental trajectories.

Promoting Positive Youth Development through School-Based Social and Emotional Learning Interventions: A Meta-Analysis of Follow-Up Effects.

The calculation and use of effect sizes—such as d for mean differences and r for correlations—has increased dramatically in second language (L2) research in the last decade. Interpretations of these effects, however, have been rare and, when present, have largely defaulted to Cohen's levels of small (d = .2, r = .1), medium (.5, .3), and large (.8, .5), which were never intended as prescriptions but rather as a general guide. As Cohen himself and many others have argued, effect sizes are best understood when interpreted within a particular discipline or domain. This article seeks to promote more informed and field-specific interpretations of d and r by presenting a description of L2 effects from 346 primary studies and 91 meta-analyses (N > 604,000). Results reveal that Cohen's benchmarks generally underestimate the effects obtained in L2 research. Based on our analysis, we propose a field-specific scale for interpreting effect sizes, and we outline eight key considerations for gauging relative magnitude and practical significance in primary and secondary studies, such as theoretical maturity in the domain, the degree of experimental manipulation, and the presence of publication bias.

How Big Is “Big”? Interpreting Effect Sizes in L2 Research

/pdf/the-challenge-of-scaling-up-educational-reform-findings-and-223fvh4lok.pdf

The Challenge of Scaling Up Educational Reform. Findings and Lessons from First Things First. Final Report.

/pdf/using-covariates-to-improve-precision-empirical-guidance-for-108osajyhf.pdf

Using Covariates to Improve Precision Empirical Guidance for Studies That Randomize Schools to Measure the Impacts of Educational Interventions

The Youth Transition Demonstration (YTD), led by Mathematica Policy Research, MDRC, and TransCen, Inc., is a large-scale demonstration and evaluation sponsored by the Social Security Administration (SSA) to improve understanding of how to help youth with disabilities reach their full economic potential. In particular, SSA is interested in developing and testing promising approaches for helping young people with disabilities become more self-sufficient and less reliant on disability benefits. The YTD conceptual framework, which is based on best practices in facilitating youth transition, specifies that the six projects participating in the evaluation provide employment services (emphasizing paid competitive employment), benefits counseling, links to services available in the community, and other assistance to youth with disabilities and their families. Additionally, participating youth are eligible for SSA waivers of certain benefit program rules, which allow them to retain more of their disability benefits and health insurance while they work for pay. Using a rigorous random assignment methodology, the YTD evaluation team is assessing whether these services and incentives are effective in helping youth with disabilities achieve greater independence and economic self-sufficiency. The earliest of the evaluation projects began operations in 2006 and ended in 2009. The latest started in 2008 and will end in 2012. This report presents first-year evaluation findings for the City University of New York (CUNY) Youth Transition Demonstration Project (YTDP), which served youth ages 14 through 19 in Bronx County, New York, from August 2006 to May 201'3 While it will take several more years to fully observe the transitions that youth participants make to adult life, early data from the evaluation provide rich information on how the YTDP operated and the differences it made in key outcomes for youth. Specifically, the report includes findings from a process analysis of the YTDP, including a description of the program model, how the project was implemented and services were delivered, and the project’s fidelity to the YTD model. The report also includes impact findings, based on data collected 12 months after youth entered the evaluation, on the use of services, paid employment, participation in education, income from earnings and benefits, and attitudes and expectations. In brief, the YTDP was well implemented and had statistically significant impacts on several important outcomes during the year following random assignment. The project as implemented closely conformed to the local design for the intervention and included all major components of the YTD program model. The process analysis found that the project enrolled 79 percent of eligible youth and provided all enrollees with at least some services, many of which were delivered through Saturday morning workshops. Approximately half of the enrollees had paid summer work experiences through the project. The impact analysis found that youth who had been given the opportunity to participate in the YTDP were more likely to have used services to promote employment and been employed for pay than in the absence of the intervention. However, the project had no impacts on income, expectations, or a composite measure of school enrollment or high school completion.

/pdf/the-social-security-administration-s-youth-transition-1ut1jkqlh1.pdf

The Social Security Administration’s Youth Transition Demonstration ProjectsInterim Report on the City University of New York’s Project

The Youth Transition Demonstration (YTD) is a large-scale demonstration and evaluation sponsored by the Social Security Administration (SSA) to improve understanding of how to help youth with disabilities reach their full economic potential. In particular, SSA is interested in developing and testing promising approaches for helping young people with disabilities become more self-sufficient and less reliant on disability benefits. The YTD program model, which is based on best practices in facilitating youth transition, specifies that the six projects participating in the evaluation provide employment services (emphasizing paid competitive employment), benefits counseling, links to services available in the community, and other assistance to youth with disabilities and their families. Additionally, participating youth are eligible for SSA waivers of certain benefit program rules, which allow them to retain more of their disability benefits and health insurance while they work for pay. Using a rigorous random assignment methodology, the YTD evaluation team is assessing whether these services and incentives are effective in helping youth with disabilities achieve greater independence and economic self-sufficiency.2 The earliest of the evaluation projects began operations in 2006 and ended in 2009. The latest started in 2008 and will end in 2012.In this report, we present first-year evaluation findings for the Colorado Youth WINS (Work Incentive Network of Supports) YTD project, which served youth in four counties from August 2006 to December 2009. While it will take several more years before we fully observe the transitions that youth participants make to adult life, early data from the evaluation provide rich information on how the Youth WINS project operated and the difference it made during the initial stages of the transitions. Specifically, the report includes findings from our process analysis of Youth WINS, including a description of the program model, how the project was implemented and services were delivered, and the project’s fidelity to the YTD program model. The report also includes impact findings, based on data collected 12 months after youth entered the evaluation, on the use of services, paid employment, participation in education, income from earnings and benefits, and attitudes and expectations.In brief, we learned through the process analysis that Youth WINS, as implemented, deviated from the YTD program model in ways that may have reduced its potential to achieve certain critical YTD objectives. In particular, the project focused more on case management and less on the delivery of employment services directly to youth than indicated by the YTD program model. Notwithstanding that focus, the impact analysis revealed that youth who had been given the opportunity to participate in Youth WINS were more likely to have used services to promote employment than youth in a randomly selected control group. However, we found no impacts on key measures of youth employment, income, and expectations during the year following random assignment.

/pdf/the-social-security-administration-s-youth-transition-4j79atavqj.pdf

The Social Security Administration's Youth Transition Demonstration Projects: Interim Report on Colorado Youth Wins

The Youth Transition Demonstration (YTD) is a large-scale demonstration and evaluation sponsored by the Social Security Administration (SSA) to improve understanding of how to help youth with disabilities reach their full economic potential. In particular, SSA is interested in developing and testing promising approaches for helping young people with disabilities become more self-sufficient and less reliant on disability benefits. The YTD conceptual framework, which is based on best practices in facilitating youth transition, specifies that the six projects participating in the evaluation provide employment services (emphasizing paid competitive employment), benefits counseling, links to services available in the community, and other assistance to youth with disabilities and their families. Additionally, participating youth are eligible for SSA waivers of certain benefit program rules, which allow them to retain more of their disability benefits and health insurance while they work for pay. Using a rigorous random assignment methodology, the YTD evaluation team is assessing whether these services and incentives are effective in helping youth with disabilities achieve greater independence and economic self-sufficiency. The earliest of the evaluation projects began operations in 2006 and ended in 2009. The latest started in 2008 and will end in 2012.This report presents first-year evaluation findings for the Transition WORKS YTD project, which served youth disability beneficiaries in Erie County, New York, including the city of Buffalo, from February 2007 to December 2009. While it will take several more years to fully observe the transitions that youth participants make to adult life, early data from the evaluation provide rich information on how Transition WORKS operated and the differences it made in key outcomes for youth. Specifically, the report includes findings from a process analysis of Transition WORKS, including a description of the program model, how the project was implemented and services were delivered, and the project’s fidelity to the YTD model. The report also includes impact findings, based on data collected 12 months after youth entered the evaluation, on the use of services, paid employment, participation in education, income from earnings and benefits, and attitudes and expectations.In brief, Transition WORKS was a well-organized, cohesive project that broadly conformed to the YTD program model and focused on self-determination, benefits planning, employment, education, and case management. The project enrolled 83 percent of eligible youth and provided most of them with some services in each of these components. The median duration of services directly delivered to participating youth was lower for the employment component than for several of the other components; however, the impact analysis found that youth who had been given the opportunity to participate in Transition WORKS were more likely to have used services to promote employment than youth in a randomly selected control group. Nevertheless, the project had no impacts on youth employment during the year following random assignment — nor on income, expectations for the future, and a composite measure of school enrollment or high school completion.

/pdf/the-social-security-administration-s-youth-transition-1rc7a01423.pdf

Alison Black

Papers

The Challenge of Scaling Up Educational Reform. Findings and Lessons from First Things First. Final Report.

Using Covariates to Improve Precision Empirical Guidance for Studies That Randomize Schools to Measure the Impacts of Educational Interventions

The Social Security Administration’s Youth Transition Demonstration ProjectsInterim Report on the City University of New York’s Project

The Social Security Administration's Youth Transition Demonstration Projects: Interim Report on Colorado Youth Wins

The Social Security Administration's Youth Transition Demonstration Projects: Interim Report on Transition WORKS