No Comparison: Distance Education Finds a New Use for "No Significant Difference."

TLDR: In this paper, the authors present an analysis of the research design of media comparison studies for measuring the effectiveness of instructional technology in distance education and show that such a methodology is an inappropriate approach to such an investigation.

Abstract: Media comparison studies have long been criticized as an inappropriate research design for measuring the effectiveness of instructional technology. However, a resurgence in their use has recently been noted in distance education for program evaluation purposes. An analysis of the research design will detail why such a methodology is an inappropriate approach to such an investigation. Increased access to such programming does not seem to serve as a satisfactory benefit for the implementation of distance education efforts. Stakeholders desire to prove that participants in distance-delivered courses receive the same quality of instruction off-campus as those involved in the “traditional” classroom setting. However, the desire to prove that the quality of such distributed offerings is equal to the quality of on-campus programming often results in comparisons of achievement between the two groups of student participants. Statistically, such a research design almost guarantees that the desired outcome will be attained—that indeed distance learners perform as well as campus-based students.