Preface Chapter 1 The challenge Chapter 2 The nature of the evidence: A synthesis of meta-analyses Chapter 3 The argument: Visible teaching and visible learning Chapter 4: The contributions from the student Chapter 5 The contributions from the home Chapter 6 The contributions from the school Chapter 7 The contributions from the teacher Chapter 8 The contributions from the curricula Chapter 9 The contributions from teaching approaches - I Chapter 10 The contributions from teaching approaches - II Chapter 11: Bringing it all together Appendix A: The 800 meta-analyses Appendix B: The meta-analyses by rank order References

Visible Learning: A Synthesis of Over 800 Meta-Analyses Relating to Achievement

A systematic search of the research literature from 1996 through July 2008 identified more than a thousand empirical studies of online learning. Analysts screened these studies to find those that (a) contrasted an online to a face-to-face condition, (b) measured student learning outcomes, (c) used a rigorous research design, and (d) provided adequate information to calculate an effect size. As a result of this screening, 51 independent effects were identified that could be subjected to meta-analysis. The meta-analysis found that, on average, students in online learning conditions performed better than those receiving face-to-face instruction. The difference between student outcomes for online and face-to-face classes—measured as the difference between treatment and control means, divided by the pooled standard deviation—was larger in those studies contrasting conditions that blended elements of online and face-to-face instruction with conditions taught entirely face-to-face. Analysts noted that these blended conditions often included additional learning time and instructional elements not received by students in control conditions. This finding suggests that the positive effects associated with blended learning should not be attributed to the media, per se. An unexpected finding was the small number of rigorous published studies contrasting online and face-to-face learning conditions for K–12 students. In light of this small corpus, caution is required in generalizing to the K–12 population because the results are derived for the most part from studies in other settings (e.g., medical training, higher education).

Evaluation of Evidence-Based Practices in Online Learning: A Meta-Analysis and Review of Online Learning Studies

A meta-analysis of the comparative distance education (DE) literature between 1985 and 2002 was conducted. In total, 232 studies containing 688 independent achievement, attitude, and retention outcomes were analyzed. Overall results indicated effect sizes of essentially zero on all three measures and wide variability. This suggests that many applications of DE outperform their classroom counterparts and that many perform more poorly. Dividing achievement outcomes into synchronous and asynchronous forms of DE produced a somewhat different impression. In general, mean achievement effect sizes for synchronous applications favored classroom instruction, while effect sizes for asynchronous applications favored DE. However, significant heterogeneity remained in each subset.

/pdf/how-does-distance-education-compare-with-classroom-dh5xmepwak.pdf

How Does Distance Education Compare With Classroom Instruction? A Meta-Analysis of the Empirical Literature

Background/ContextEarlier research on various forms of distance learning concluded that these technologies do not differ significantly from regular classroom instruction in terms of learning outcom...

https://journals.sagepub.com/doi/pdf/10.1177/016146811311500307

The Effectiveness of Online and Blended Learning: A Meta-Analysis of the Empirical Literature

This meta-analysis of the experimental literature of distance education (DE) compares different types of interaction treatments (ITs) with other DE instructional treatments. ITs are the instructional and/or media conditions designed into DE courses, which are intended to facilitate student–student (SS), student– teacher (ST), or student–content (SC) interactions. Seventy-four DE versus DE studies that contained at least one IT are included in the meta-analysis, which yield 74 achievement effects. The effect size valences are structured so that the IT or the stronger IT (i.e., in the case of two ITs) serve as the experimental condition and the other treatment, the control condition. Effects are categorized as SS, ST, or SC. After adjustment for methodological quality, the overall weighted average effect size for achievement is 0.38 and is heterogeneous. Overall, the results support the importance of the three types of ITs and strength of ITs is found to be associated with increasing achievement outcomes. A strong association is found between strength and achievement for asynchronous DE courses compared to courses containing mediated synchronous or face-to-face interaction. The results are interpreted in terms of increased cognitive engagement that is presumed to be promoted by strengthening ITs in DE courses.

https://www.jstor.org/stable/pdfplus/40469094.pdf

A Meta-Analysis of Three Types of Interaction Treatments in Distance Education

This article summarizes a quantitative synthesis of studies of the effectiveness of interactive distance education using videoconferencing and telecommunications for K-12 academic achievement. Effect sizes for 19 experimental and quasi-experimental studies including 929 student participants were analyzed across sample characteristics, study methods, learning environment, learner attributes, and technological characteristics. The overall mean effect size was 0.147, a small positive effect in favor of distance education. Effect sizes were more positive for interactive distance education programs that combine an individualized approach with traditional classroom instruction. Programs including instruction delivered through telecommunications, enhancement of classroom learning, short duration, and small groups yielded larger effect sizes than programs using videoconferencing, primary instruction through distance, long duration, and large groups. Studies of distance education for all academic content areas except foreign language resulted in positive effect sizes. This synthesis supports the use of interactive distance education to complement, enhance, and expand education options because distance education can be expected to result in achievement at least comparable to traditional instruction in most academic circumstances.

The Effectiveness of Interactive Distance Education Technologies in K-12 Learning:

Online, Blended, and Distance Education in Schools

The current study examines both student self-regulated learning (SRL) and teacher orchestration in a virtual learning environment (VLE), with respect to student achievement. The study used SRL indicators derived from the log data on how students used the VLE system, survey data on how teachers made use of the VLE for Algebra instruction, as well as formative and summative Algebra assessment scores. The sample included 6,174 students being taught by 93 teachers in 49 schools in the 2018–2019 academic year. Multilevel structural equation modeling (SEM) analysis revealed the relationship between SRL indicators and student achievement. We found a positive relationship between student SRL and student achievements. Specifically, three SRL indicators, “Watched Recommended Videos”, “Watched Videos” and “Answered Quizzes after Watching a Video”, were significantly associated with student achievement. We did not observe direct associations between teacher orchestration and student achievement, but we found an indirect association between teacher orchestration and student achievement via one SRL indicator, “Watched Recommended Videos”. The results show that a positive relationship between EOC scores and student reviewing incorrect questions increased as the use of instructional videos by the teacher increased. This study supports the critical role of SRL and indicates that teachers should have flexibility in adapting learning activities for online learning. • A structural equation model showed relationships between student SLR of a VLE and achievement in Algebra. • Student watching of videos, recommended videos, taking quizzes and reviewing incorrect answers predicted Algebra scores. • Watching recommended videos mediated the relationship between preparation for high-stakes testing and formative quizzes. • Teacher instructional video use moderated the relationship between review of incorrect questions and high-stakes tests. 

The relationship between self-regulated student use of a virtual learning environment for algebra and student achievement: An examination of the role of teacher orchestration

Digital Transformation in Education: Critical Components for Leaders of System Change

Advanced learning technologies (ALT) have become increasingly available to teachers for classroom use. Research has suggested that many factors can influence teacher adoption and fidelity of use of ALT in the classroom, including teacher beliefs, knowledge and experience, technological factors, and instructional factors. However, there has been scarce research linking teacher factors to student adoption of ALTs. This study examined the relationships between teacher characteristics and practices and student adoption and learning gains with a video recommendation system embedded within a virtual learning environment (VLE) for Algebra. Secondary data was obtained from an experimental study conducted over one academic semester in middle and high schools in a southeastern state of the United States. The sample included 52 teachers and 2936 students. The data included teacher responses to three surveys, and student demographic and achievement variables. A random forest was used to predict the rate that the students followed video recommendations in the VLE. The results show that the recommendation followed rate is related to the teachers' fidelity of use, frequency of student monitoring, and experience with the VLE. Most of the survey items specifically evaluating teachers' beliefs about the recommender were important predictors of students' following video recommendations. Teacher monitoring through a dashboard was the most important predictor. The analysis of treatment effect heterogeneity of the video recommendation system was performed using the generic machine learning inference (GenericML) method paired with random forests. Results show that teachers of students who benefitted most reported spending more time using the videos of the VLE and following student progress through the dashboard, but less time on the VLE than teachers of students who benefit the least. Teachers of students who benefitted the least had larger classrooms, struggled more with the challenges due to the Covid-19 pandemic, and spent less time with classroom planning. The results support the recommendation that teacher professional development for ALT should engage groups of educators in increasing their experience with the application so that they build comfort and confidence in its use in ways in which students are most likely to benefit.

Catherine S. Cavanaugh

Papers

The Effectiveness of Interactive Distance Education Technologies in K-12 Learning:

Online, Blended, and Distance Education in Schools

The relationship between self-regulated student use of a virtual learning environment for algebra and student achievement: An examination of the role of teacher orchestration

Digital Transformation in Education: Critical Components for Leaders of System Change

How Teachers Influence Student Adoption and Effectiveness of a Recommendation System for Algebra