Journal of Research on Educational Effectiveness 

About: Journal of Research on Educational Effectiveness is an academic journal published by Taylor & Francis. The journal publishes majorly in the area(s): Academic achievement & Reading comprehension. It has an ISSN identifier of 1934-5747. Over the lifetime, 409 publications have been published receiving 12217 citations. The journal is also known as: JREE.

Why we (usually) don't have to worry about multiple comparisons

Abstract: Applied researchers often find themselves making statistical inferences in settings that would seem to require multiple comparisons adjustments. We challenge the Type I error paradigm that underlies these corrections. Moreover we posit that the problem of multiple comparisons can disappear entirely when viewed from a hierarchical Bayesian perspective. We propose building multilevel models in the settings where multiple comparisons arise. Multilevel models perform partial pooling (shifting estimates toward each other), whereas classical procedures typically keep the centers of intervals stationary, adjusting for multiple comparisons by making the intervals wider (or, equivalently, adjusting the p values corresponding to intervals of fixed width). Thus, multilevel models address the multiple comparisons problem and also yield more efficient estimates, especially in settings with low group-level variation, which is where multiple comparisons are a particular concern.

The Impact of Vocabulary Instruction on Passage-Level Comprehension of School-Age Children: A Meta-Analysis

Abstract: A meta-analysis of vocabulary interventions in grades pre-K to 12 was conducted with 37 studies to better understand the impact of vocabulary on comprehension. Vocabulary instruction was found to be effective at increasing students' ability to comprehend text with custom measures (d = 0.50), but was less effective for standardized measures (d = 0.10). When considering only custom measures, and controlling for method variables, students with reading difficulties (d = 1.23) benefited more than three times as much as students without reading problems (d = 0.39) on comprehension measures. Gains on vocabulary measures, however, were comparable across reading ability. In addition, the correlation of vocabulary and comprehension effects from studies reporting both outcomes was modest (r = .43).

Persistence and Fadeout in the Impacts of Child and Adolescent Interventions

Abstract: Many interventions targeting cognitive skills or socioemotional skills and behaviors demonstrate initially promising but then quickly disappearing impacts. Our paper seeks to identify the key features of interventions, as well as the characteristics and environments of the children and adolescents who participate in them, that can be expected to sustain persistently beneficial program impacts. We describe three such processes: skill-building, foot-in-the-door and sustaining environments. We argue that skill-building interventions should target "trifecta" skills - ones that are malleable, fundamental, and would not have developed eventually in the absence of the intervention. Successful foot-in-the-door interventions equip a child with the right skills or capacities at the right time to avoid imminent risks (e.g., grade failure or teen drinking) or seize emerging opportunities (e.g., entry into honors classes). The sustaining environments perspective views high quality of environments subsequent to the completion of the intervention as crucial for sustaining early skill gains. These three perspectives generate both complementary and competing hypotheses regarding the nature, timing and targeting of interventions that generate enduring impacts.

Performance Trajectories and Performance Gaps as Achievement Effect-Size Benchmarks for Educational Interventions

Abstract: Two complementary approaches to developing empirical benchmarks for achievement effect sizes in educational interventions are explored. The first approach characterizes the natural developmental progress in achievement made by students from one year to the next as effect sizes. Data for seven nationally standardized achievement tests show large annual gains in the early elementary grades followed by gradually declining gains in later grades. A given intervention effect will therefore look quite different when compared to the annual progress for different grade levels. The second approach explores achievement gaps for policy-relevant subgroups of students or schools. Data from national- and district-level achievement tests show that, when represented as effect sizes, student gaps are relatively small for gender and much larger for economic disadvantage and race/ethnicity. For schools, the differences between weak schools and average schools are surprisingly modest when expressed as student-level e...

Moving from the Lab to the Field: The Role of Fidelity and Achieved Relative Intervention Strength.

Abstract: As an example of how the effectiveness of well-designed laboratory interventions is often diffused in the field, we examined the effects of a motivation intervention on students' perceptions and learning. The intervention proved to be more effective in the laboratory (g = 0.45) than the field (g = 0.05) in enhancing subsequent motivation. We explored this reduction in treatment effectiveness through a fidelity analysis that examined the extent to which participants responded to the treatment. We calculated fidelity as three indices of achieved relative treatment strength (Cordray & Pion, 2006), and found that, regardless of how fidelity was calculated, achieved relative strength was about 1 standard deviation less in the classroom than the laboratory. In addition, greater levels of achieved relative strength were associated with greater differences in the motivational outcome—indicating that intervention was more effective for participants who actually received the treatment than those who did no...