Showing papers in "Educational Evaluation and Policy Analysis in 1998"
TL;DR: The authors examined how performance-based assessment changed mathematics teaching under conditions of moderate and low stakes, and found that the effects of state testing on teaching may be overrated by both advocates and opponents of such policies.
Abstract: To examine how performance-based assessment changed mathematics teaching under conditions of moderate and low stakes, we studied middle school teachers in five districts in Maine and Maryland. Our observations suggest that the effects of state testing on teaching may be overrated by both advocates and opponents of such policies. When combined with moderately high stakes and other conditions, such assessments generate considerable activity focused on the test itself. This activity can promote certain changes, like aligning subjects taught with the test. It appears to be less successful, however, in changing basic instructional strategies.
272 citations
TL;DR: This paper investigated the relationship between students' opportunity to learn (OIL) and their science achievement and found that OTL variables were significant predictors of both written and hands-on test scores even after students' general ability level, ethnicity, and gender were controlled.
Abstract: This study investigated the relationship between students’ opportunity to learn (OIL) and their science achievement. The data are of 623 8th-graders enrolled in five public schools in Los Angeles, California. Hierarchical linear modeling was used to analyze OTL variables at two levels of instructional processes: the classroom level and the student level. Students’ science test scores are based on a written test and a hands-on test. OTL effects on these two test scores were studied to see whether the effects differ depending on how science achievement is measured. It was found that OTL variables were significant predictors of both written and hands-on test scores even after students’ general ability level, ethnicity, and gender were controlled. The OTL effects varied by test format (written test and hands-on test). Content exposure was the most significant predictor of students’ written test scores, and quality of instructional delivery was the most significant predictor of the hands-on test scores. In con...
173 citations
TL;DR: In this paper, the authors describe case studies of five high-performing public schools that have organized professional resources in innovative ways and quantify objectively the ways in which these schools use resources differently depending on their instructional goals and strategies.
Abstract: Although a great deal of debate surrounds the level and allocation of resources to public schools, very little research addresses how schools might organize teaching resources more effectively at the school level. This article describes case studies of five high-performing public schools that have organized professional resources in innovative ways. The study sought to detail alternative ways of deploying instructional resources in order to provide concrete alternatives to traditional organization of teachers and to quantify objectively the ways in which these schools use resources differently depending on their instructional goals and strategies. Although the schools studied looked very different from one another, they shared six principles of resource allocation that are outlined in this article. The article develops a framework for examining the use of resources and a methodology that may be used to measure the extent to which schools use their resources in focused ways to support teaching and learning.
165 citations
TL;DR: The first five years of the Milwaukee Voucher Program were summarized in this paper, and the results in terms of the effects on families and students and on schools were discussed in detail.
Abstract: The Milwaukee voucher program was enacted by the Wisconsin State Legislature in the summer of 1990. Beginning that August, it allowed students to attend private schools with public vouchers for the first time in the United States. This article provides a summary of the results of the first five years of that program. I begin with a brief discussion of the theoretical and research issues—brief because these issues have been thoroughly aired many times (Cookson, 1994; Henig, 1994; Smith & Meier, 1995; Wells, 1993; Witte & Rigdon, 1993). Following a description of the initial program and subsequent changes, I outline who participated in the program—including characteristics of students and families and schools. I then describe the results in terms of the effects on families and students and on schools. I conclude with a discussion of the implications of this type of program and more openended voucher programs.
117 citations
TL;DR: In this article, a longitudinal investigation of 23 colleges and universities sought to estimate the impacts of on-and off-campus work on standardized measures of student cognitive development across three years of college.
Abstract: This longitudinal investigation of 23 colleges and universities sought to estimate the impacts of on- and off-campus work on standardized measures of student cognitive development across three years of college. With controls made for student background characteristics and other experiences of college, there was little evidence to suggest that either form of work inhibited cognitive development in the first year of college. In the second year of college, on-campus work had small negative total and direct influences on science reasoning, but neither form of work inhibited students’ writing skills. Both forms of work had a significant curvilinear relationship with a composite measure of end-of-third-year cognitive development consisting of reading comprehension and critical thinking. Part-time on- or off-campus work had a positive influence, but on-campus work in excess of 15 hours per week or off-campus work in excess of 20 hours per week had a negative impact. Finally, across all years of the study, the co...
109 citations
TL;DR: In this article, the authors relate various types of educational spending to mean mathematics achievement and its social distribution among students, using a nationally representative database of 7,217 12th graders.
Abstract: This study relates various types of educational spending to mean mathematics achievement and its social distribution among students. A nationally representative database of 7,217 12th graders was d...
97 citations
TL;DR: In this article, social and ethnic inequality in access to resources for mathematics learning in eighth grade was considered, including favorable school disciplinary climate, advanced course offerings, teacher sub-classes, and teacher diversity.
Abstract: This article considers social and ethnic inequality in access to resources for mathematics learning in eighth grade: favorable school disciplinary climate, advanced course offerings, teacher subjec...
95 citations
TL;DR: In this article, the authors examined the impact of SSS on retention and found that SSS had a positive impact for all three measures of retention that were used, but the impact varied depending on which services students used and how much they participated.
Abstract: Student Support Services (SSS) is one of the largest federal TRIO programs designed to help disadvantaged students stay in and complete college. Through a longitudinal study of participants and comparable non participants, we examined the impact of SSS on retention. Data were collected through student questionnaires, institutional and program data, and students’ postsecondary transcripts. We found that SSS had a positive impact for all three measures of retention that were used, but the impact varied depending on which services students used and how much they participated. The results confirm that retention programs should address both academic and social integration on campus.
79 citations
TL;DR: In this paper, the authors examined the effects of high school grades on long-term productivity as measured by earnings and found that high school grade has a strong and significant effect on earnings 9 years after high school for both men and women.
Abstract: Research shows that employers are dissatisfied with their ability to hire good workers out of high school (Barton, 1990; Cappelli & Rogovsky, 1993). This article considers whether employers could benefit from using high school grades to identify workers whom they will value more in the long run. Using the High School and Beyond data on the sophomore cohort, this article examines the effects of high school grades on long-term productivity as measured by earnings. It finds that high school grades do have a strong and significant effect on earnings 9 years after high school for both men and women, those with and without bachelor’s degrees, and controlling for race/ethnicity, SES, region of the country, and whether the school is public or private. Using a fixed-effect model, it also demonstrates that these findings are robust even after controlling for school-level differences. The article further confirms other researchers’ findings of no or negative short-term effects of high school grades on earnings. It a...
79 citations
TL;DR: The authors examined whether middle and high school algebra students who were taught in a manner consistent with the National Council for Teachers of Mathematics Professional Standards performed differently on three standardized algebra assessments than students taught in traditional classrooms.
Abstract: As almost every state attempts to reform mathematics instruction by implementing new teaching standards, state testing practices remain largely unchanged. Do these new standards undermine student performance on old tests? This question is investigated by examining whether middle and high school algebra students taught in a manner consistent with the National Council for Teachers of Mathematics Professional Standards performed differently on three standardized algebra assessments than students taught in traditional classrooms. The data come from 94 teachers, 2,369 students, and 40 schools in one of the nation’s largest school districts. Results indicate that the Standards do not undermine performance on the old tests. In fact, middle school algebra students whose teachers spent more time using the NCTM teaching approach had higher growth rates than students whose teachers spent less time using the approach. However, students with higher ability levels benefited more. The growth rates of the lowest achievin...
64 citations
TL;DR: In this article, the authors present a synthesis of what is known about Title I schoolwide programs, focusing on three aspects: characteristics of schools and districts implementing school wide programs, programmatic and organizational characteristics of schoolwide program schools, and evidence of the effectiveness of school wide program schools particularly in terms of student performance.
Abstract: Recent federal legislation, including the 1994 Improving America’s Schools Act, has enabled broad expansion of Title I schoolwide programs to over 8,000 schools across the nation. These regulatory changes are intended to reduce the historically fragmented or categorical character of title I programs and improve the effectiveness of entire schools rather than targeting services to meet the needs of the most disadvantaged subpopulations. Despite the dramatic increase in the number of schoolwide programs, there is little comprehensive information about them and their effectiveness relative to traditional Title I programming. This article presents a synthesis of what is known about Title I schoolwide programs, focusing on three aspects: characteristics of schools and districts implementing schoolwide programs, programmatic and organizational characteristics of schoolwide program schools and districts, and evidence of the effectiveness of schoolwide program schools, particularly in terms of student performance...
TL;DR: In this article, the authors argue that traditional school finance systems are aging structures in need of dramatic change to make them more supportive of the goals and strategies of standards and school-based education reform.
Abstract: This article argues that traditional school finance systems are aging structures in need of dramatic change to make them more supportive of the goals and strategies of standards-and school-based education reform. This article reviews the shortcomings of current finance structures, provides several suggestions for changing school finance structures, including a shift from fiscal equity toward educational adequacy, and also suggests several performance enhancement elements that could enhance such new school finance systems.
TL;DR: Gender differences on the NELS:88 multiple-choice and constructed-response science tests were explored through a combination of statistical analyses and interviews as discussed by the authors, and the results reveal the value of studying the validity of the outcome measure and suggest that conclusions about group differences and about correlates of achievement depend heavily on specific features of the items that make up the test.
Abstract: Gender differences on the NELS:88 multiple-choice and constructed-response science tests were explored through a combination of statistical analyses and interviews. Performance gaps between males and females varied across formats (multiple-choice versus constructed-response) and across items within a format. Differences were largest for items that involved visual content and called on application of knowledge commonly acquired through extracurricular activities. Large-scale surveys such as NELS:88 are widely used by researchers to study the effects of various student and school characteristics on achievement. The results of this investigation reveal the value of studying the validity of the outcome measure and suggest that conclusions about group differences and about correlates of achievement depend heavily on specific features of the items that make up the test.
TL;DR: This article explored the historical evolution of the school-state relationship, how educational reform movements of the past 30 years may undermine the alliance of school and state, and the challenges this development poses for analysts of education policy.
Abstract: The American school originated as an extension of fundamental social units: families, churches, and communities. Repeated waves of reform succeeded infully institutionalizing public schooling, and schools came to operate under a formal system of public authority. Founded as an institution independent from state andfederal control, the school has been transformed into an institution wedded to multiple layers of government-local, state, and federal. This essay explores the historical evolution of the school-state relationship, how educational reform movements of the past 30 years may undermine the alliance of school and state, and the challenges this development poses for analysts of education policy. The 19th-century school system struggled into existence by gaining authority over its core activities and fending off rival institutions attempting to usurp its authority. Because of the success of these efforts, the educational system was able to monopolize the set of experiences that we associate with becoming a learned adult. Virtually all schools in the United States, even religious and private nonsectarian schools, attend to the institutional forms established and maintained by the public school system-e.g., graded classes of students and teachers, curriculum organized by subject matter, diplomas signifying matriculation from one level to the next (Meyer & Rowan, 1978). Learning and formal schooling are not the same thing, but they are considered synonymous in modem thinking and frequently presented as guarantors of prosperity for both individuals and nations (Reich, 1991). Schools won this remarkable position in society through an alliance with the state: They became public schools. This article examines the evolution of the American school's close relationship with government. It may seem strange to think about the school and the state as separate entities, but doing so allows for a sharper analysis of the two institutions' historical interactions. The benefits of their partnership are impressive-schooling for those who could not otherwise afford it, a steady supply of skilled workers and informed voters, a polity socialized by common experiences with
TL;DR: Work-based learning experiences in connection with school activities are proliferating, yet we know very little about their quality as learning experiences for young children as mentioned in this paper, and we know little about the quality of these experiences as a learning experience for adults.
Abstract: Programs that incorporate work-based learning (WBL) experiences in connection with school activities are proliferating, yet we know very little about their quality as learning experiences for young...
TL;DR: The fall 1997 issue of Educational Evaluation and Policy Analysis contained an article by Jianping Shen of Western Michigan University titled "Has the Alternative Certification Policy Materialized Its Promise? A Comparison Between Traditionally and Alternatively certified Teachers in Public Schools" Shen's research used data from the 1993-94 Schools and Staffing Survey to contrast teachers who held alternative certification with those who entered through traditional education programs as discussed by the authors.
Abstract: The fall 1997 issue of Educational Evaluation and Policy Analysis contained an article by Jianping Shen of Western Michigan University titled "Has the Alternative Certification Policy Materialized Its Promise? A Comparison Between Traditionally and Alternatively Certified Teachers in Public Schools" Shen's research used data from the 1993-94 Schools and Staffing Survey to contrast teachers who held alternative certification with those who entered through traditional teacher education programs The findings, generally unfavorable to alternative certification, have attracted attention in education policy circles and were the subject of an article in the September 10, 1997, issue of Education Week (Sandham, 1997) The purpose of this note is to caution readers against acceptance of these findings I do not argue that the conclusions are necessarily wrong, but rather that they are based on data that are badly contaminated with measurement and response error, raising grave questions about the reliability of any research based on them The key survey question--on which all the rest of the analysis depends-was an item that asked teachers to identify the type of certificate they held There were eight possible responses, given below verbatim:
TL;DR: This paper argued that Ballou offered some interesting and creative ideas to check the quality of the data set, but gave no conclusive evidence to demonstrate that the findings are invalid, and further analyses of the dataset were discussed.
Abstract: This rejoinder is a response to the three Ballou critiques. It argues that Ballou offered some interesting and creative ideas to check the quality of the data set, but gave no conclusive evidence to demonstrate that the findings are invalid. Further analyses of the data set are discussed.
TL;DR: This paper investigated the usefulness of the 1992 TSA in reading for state-level administrators of assessment, and found that it involved the International Reading Association and more teachers, continue but modify descriptors, disseminate results quicker, and make results more useful.
Abstract: This study investigated the usefulness of the 1992 TSA in reading for state-level administrators of assessment. Data regarding their perceptions about the credibility and orientation of various components of the TSA were collected (using phone interviews) from 26 state directors of assessment, or their designees, and analyzed. Additionally, data about the utility of the manner of reporting TSA results, how results are currently used and could be improved, were collected and analyzed. The following four conclusions were drawn from the study: involve the International Reading Association and more teachers, continue but modify descriptors, disseminate results quicker, and make results more useful.
TL;DR: In this paper, the authors found that current national data sets do not contain questions about family background or rank of school attended that are detailed enough to permit a significant analysis of the impact offamily background on educational attainment.
Abstract: In reviewing educational trends in Japan during the post-World War II era, the authors found that current national data sets do not contain questions aboutfamily background or rank of school attended that are detailed enough to permit a significant analysis of the impact offamily background on educational attainment. Given Japan's stature in the current community of nations, the continued government aversion to collecting family income or educational background data, along with precise data on rank of school attended and/or types of cram school, appears negligent. The authors discuss both cultural and political reasons that may create a reluctance to collect such data.
TL;DR: In this paper, the authors describe two systems: quinquennial program-based licensing at a California university and annual performance-based license at a British polytechnic, focusing on the quality of the program preparing the candidates; the latter, on the qualifications of individual candidates.
Abstract: How should the state assure itself that those entrusted with the formal education of its young are qualified? With what investigation can it be satisfied that it has acquired sufficient knowledge about teacher candidate qualifications to grant a license?' Although the basic authority of the state to license teachers is not at issue in current educational reform, there are in the United States and in Great Britain divergent licensing practices This article describes two systems: quinquennial programbased licensing at a California university and annual performance-based licensing at a British polytechnic2 The former focuses on the quality of the program preparing the candidates; the latter, on the qualifications of individual candidates The former is the dominant mode of teacher licensing in the United States; the latter, an emerging model3 The systems represent, in Maurice Kogan's terms (1986), a "public control" model of accountability They have in common (a) a basic objective, the assurance of competence in program graduates; (b) a basic procedure, review by professional colleagues acting as agents of the state;4 (c) a set of standards to be used in judging quality; and (d) the availability of sanctions to force institutional compliance with review team findings The fact that, in both systems, professional colleagues function as agents of the state and in common utilize such procedures as observation and interview should not obscure the fundamental difference in immediate objective--determining the quality of a teacherpreparation program versus determining the teaching qualifications of individual students