Showing papers in "Education Policy Analysis Archives in 2002"

High-stakes testing, uncertainty, and student learning

Abstract: A brief history of high-stakes testing is followed by an analysis of eighteen states with severe consequences attached to their testing programs. These 18 states were examined to see if their high-stakes testing programs were affecting student learning, the intended outcome of high-stakes testing policies promoted throughout the nation. Scores on the individual tests that states use were not analyzed for evidence of learning. Such scores are easily manipulated through test-preparation

The Link Between Teacher Classroom Practices and Student Academic Performance

Abstract: Quantitative studies of school effects have generally supported the notion that the problems of U.S. education lie outside of the school. Yet such studies neglect the primary venue through which students learn, the classroom. The current study explores the link between classroom practices and student academic performance by applying multilevel modeling to the 1996 National Assessment of Educational Progress in mathematics. The study finds that the effects of classroom practices, when added to those of other teacher characteristics, are comparable in size to those of student background, suggesting that teachers can contribute as much to student learning as the students themselves.

High-Stakes Testing & Student Learning

Abstract: A brief history of high-stakes testing is followed by an analysis of eighteen states with severe consequences attached to their testing programs. These 18 states were examined to see if their high-stakes testing programs were affecting student learning, the intended outcome of high-stakes testing policies promoted throughout the nation. Scores on the individual tests that states use were not analyzed for evidence of learning. Such scores are easily manipulated through test-preparation programs, narrow curricula focus, exclusion of certain students, and so forth. Student learning was measured by means of additional tests covering some of the same domain as each state's own high-stakes test. The question asked was whether transfer to these domains occurs as a function of a state's high-stakes testing program.

State-Mandated Testing and Teachers' Beliefs and Practice

Abstract: In this article, I examine the relationship between state-mandated testing and teachers' beliefs and practice. The studies reviewed suggest that while state testing does matter by influencing what teachers say and do, so, too, do other things, such as teachers' knowledge of subject matter, their approaches to teaching, their views of learning, and the amalgam of experience and status they possess in the school organization. As a result, the influence state-mandated testing has (or not) on teachers and teaching would seem to depend on how teachers interpret state testing and use it to guide their actions. Moreover, the influence state testing may or may not have on teachers and teaching expands beyond individual perceptions and actions to include the network of constructed meanings and significance extant within particular educational contexts. Consequently, although a relationship between the state-mandated testing and teachers' beliefs and practice does exist, testing does not appear to be an exclusive or a primary lever of change. There is further suggestion that it might not be a substantive one either. Studies that provide a richer, more in-depth understanding of the relationship between state-mandated testing and teaching in actual school settings, therefore, not only point toward important directions for future research in this area, but are greatly needed.

Research and Rhetoric on Teacher Certification

Abstract: In October, 2001, the Baltimore-based Abell Foundation issued a report purporting to prove that there is "no credible research that supports the use of teacher certification as a regulatory barrier to teaching" and urging the discontinuation of certification in Maryland. The report argued that large inequities in access to certified teachers for poor and minority students are not a problem because research linking teacher education to student achievement is flawed. In July, 2002, the U.S. Secretary of Education cited the Abell Foundation paper in his Annual Report on Teacher Quality as the sole source for concluding that teacher education does not contribute to teacher effectiveness. The Secretary's report then recommended that requirements for education coursework be eliminated from certification standards, and attendance at schools of education and student teaching be made optional. This article documents the many inaccuracies in the Abell Foundation paper and describes the actual findings of many of the studies it purports to review, as well as the findings of other studies it ignores. It details misrepresentations of a number of studies, including inaccurate statements about their methods and findings, false claims about their authors' views, and distortions of their data and conclusions. The article addresses methodological issues regarding the validity and interpretation of research. Finally, the article presents data challenging the Abell Foundation's unfounded claims that uncertified teachers are as effective as certified teachers, that teacher education makes no difference to teacher effectiveness, that verbal ability is the most important determinant of teaching effectiveness, that private schools staffed by uncertified teachers are more effective than public schools, and that untrained teachers are more qualified than prepared teachers. It concludes with a discussion of the policy issues that need to be addressed if all students are to be provided with highly qualified teachers.

The Effectiveness of "Teach for America" and Other Under-certified Teachers

Abstract: The academic achievements of students taught by under-certified primary school teachers were compared to the academic achievements of students taught by regularly certified primary school teachers. This sample of under-certified teachers included three types of under-qualified personnel: emergency, temporary and provisionally certified teachers. One subset of these under-certified teachers was from the national program "Teach For America (TFA)." Recent college graduates are placed by TFA where other under-qualified under-certified teachers are often called upon to work, namely, low-income urban and rural school districts. Certified teachers in this study were from accredited universities and all met state requirements for receiving the regular initial certificate to teach. Recently hired under-certified and certified teachers (N=293) from five low-income school districts were matched on a number of variables, resulting in 109 pairs of teachers whose students all took the mandated state achievement test. Results indicate 1) that students of TFA teachers did not perform significantly different from students of other under-certified teachers, and 2) that students of certified teachers out-performed students of teachers who were under-certified. This was true on all three subtests of the SAT 9reading, mathematics and language arts. Effect sizes favoring the students of certified teachers were substantial. In reading, mathematics, and language, the students of certified teachers outperformed students of under-certified teachers, including the students of the TFA teachers, by about 2 months on a grade equivalent scale. Students of under-certified teachers make about 20% less academic growth per year than do students of teachers with regular certification. Traditional programs of teacher preparation apparently result in positive effects on the academic achievement of low-income primary school children. Present policies allowing under-certified teachers, including those from the TFA program, to work with our most difficult to teach children appear harmful. Such policies increase differences in achievement between the performance of poor children, often immigrant and minority children, and those children who are more advantaged.

The Distribution of Emergency Permit Teachers in California

Abstract: There is a significant negative relationship between the percentage of teachers on emergency permits and student achievement at the school level in California schools, after controlling for other student and school characteristics. Generally, the more emergency permit teachers there are in a school, the lower the school's achievement. This phenomenon is examined in the context of other contributors to student achievement such as socio-economic status and school size. The effects of teacher distribution and school selection as contributing factors are considered. In addition, policy and legislative initiatives related to emergency permit teachers that have been recently debated in California will be discussed. Finally, a set of initiatives is proposed that attempt to decrease the need for emergency permit teachers and ensure that those that must be hired due to shortage conditions have the support they need to become credentialed teachers.

Testing and Diversity in Postsecondary Education: The Case of California

Abstract: The past several years have seen numerous efforts to scale back or eliminate affirmative action in postsecondary admissions. In response, policymakers and postsecondary institutions in many states are searching for ways to maintain the diversity of student populations without resorting to a prohibited focus on race. In response to these changes, this study used data from California and a simplified model of the University of California admissions process to explore how various approaches to admissions affect the diversity of the admitted student population. "Race-neutral" admissions based solely on test scores and grades were compared with the results of actual admissions before and after the elimination of affirmative action. A final set of analyses explored the effects on diversity of alternative approaches that take into account factors other than grades and scores, but not race or ethnicity. Replacing the former admissions process that included preferences with a race-neutral model based solely on GPA and SAT-I scores substantially reduced minority representation at the two most selective UC campuses but had much smaller effects at the other six, less selective campuses. SAT-I scores contributed to but were not the sole cause of the underrepresentation of African American and Hispanic students. A race-neutral model based solely on GPA also produced an underrepresentation of minorities, albeit a less severe one. None of the alternative admissions models analyzed could replicate the composition of the student population that was in place before the termination of affirmative action in California. The only approach that substantially increased the representation of minority students was accepting most students on the basis of within-school rather than statewide rankings, and this approach caused a sizable drop in both the average SAT scores and the average GPA of admitted applicants, particularly among African American and Hispanic students. Although admissions systems differ, the basic findings of this study are likely to apply at a general level to many universities and underscore the difficulty of providing proportional representation for underserved minority students at highly selective institutions without explicit preferences.

Basic Education Reform in China: Untangling the Story of Success

Abstract: China's recent basic education reform followed and, in a certain way, imitated its economic reform. The economic reform merged the experimental dual (planned and market) price systems into a free market economy and yielded phenomenal success. Basic education reform, however, has not succeeded in transforming the introductory dual-track (key school and regular school) systems into a universal one. This article briefly examines the general process and outcomes of basic education reform. It discusses the following questions: Is basic education reform also a story of success? What significant lessons can the Chinese reform experience offer to other comparable developing countries?

Home Schooling in the United States

Abstract: Home schooling is a subject of great fascination, but little solid knowledge. Despite its importance, it has received less research attention than some other recent changes in the educational system, such as the growth of charter schools. It could be argued that home schooling may have a much larger impact on educational system, both in the short and long run. This report uses the 1994 October CPS, and the National Household Education Survey of 1996 and 1999 to examine popular characterizations of the home school population. The article assembles evidence from several sources to confirm that home schooling is growing. It finds home-schooled children more likely to be middle income, white, from larger families, and from two-parent families with one parent not working. While some authors have described a division between religiously-motivated and academically-motivated home schoolers, this research finds more support for a divide based on attitude towards regular schools.

A validity critique of language program evaluations and analysis of English learner test scores

Abstract: Several states have recently faced ballot initiatives that propose to functionally eliminate bilingual education in favor of English-only approaches. Proponents of these initiatives have argued an overall rise in standardized achievement scores of California's limited English proficient (LEP) students is largely due to the implementation of English immersion programs mandated by Proposition 227 in 1998, hence, they claim Exito en California (Success in California). However, many such arguments presented in the media were based on flawed summaries of these data. We first discuss the background, media coverage, and previous research associated with California's Proposition 227. We then present a series of validity concerns regarding use of Stanford-9 achievement data to address policy for educating LEP students; these concerns include the language of the test, alternative explanations, sample selection, and data analysis decisions. Finally, we present a comprehensive summary of scaled-score achievement means and trajectories for California's LEP and non-LEP students for 1998-2000. Our analyses indicate that although scores have risen overall, the achievement gap between LEP and EP students does not appear to be narrowing.

The Geographical Distribution of Teacher Absenteeism in Large Urban School District Settings: Implications for School Reform Efforts Aimed at Promoting Equity and Excellence in Education.

Abstract: School reform efforts aimed at promoting equity and excellence at urban school settings are heavily dependent upon the quality of teaching personnel that are used to deliver the instructional program. Social Justice and other public policy issues related to equity and excellence at urban schools have begun to examine the impact that teacher absenteeism, and by extension the reliance on substitute teachers to deliver instructional might have on educational attainment. This study

Teacher Inequality: New Evidence on Disparities in Teachers' Academic Skills.

Abstract: When discussing the teacher quality gap, policy makers have tended to focus on teacher certification, degrees, and experience. These indicators have become key benchmarks for progress toward equality of educational opportunity, in part for lack of additional teacher quality indicators. This article turns attention to teachers' academic skills. National data on teachers' entrance examination scores and college selectivity reveal substantial disparities by school poverty level. The findings commend attention to the gap in academic skills in the formulation of future policy and research on the teacher quality gap.

Technical and Ethical Issues in Indicator Systems.

Abstract: Most indicator systems are top-down, published, management systems, addressing primarily the issue of public accountability. In contrast we describe here a university-based suite of "grass-roots," research-oriented indicator systems that are now subscribed to, voluntarily, by about 1 in 3 secondary schools and over 4,000 primary schools in England. The systems are also being used by groups in New Zealand, Australia and Hong Kong, and with international schools in 30 countries. These systems would not have grown had they not been cost-effective for schools. This demanded the technical excellence that makes possible the provision of one hundred percent accurate data in a very timely fashion. An infrastructure of powerful hardware and ever-improving software is needed, along with extensive programming to provide carefully chosen graphical and tabular presentations of data, giving at-a-glance comparative information. Highly skilled staff, always learning new techniques, have been essential, especially as we move into computer-based data collection. It has been important to adopt transparent, readily understood methods of data analysis where we are satisfied that these are accurate, and to model the processes that produce the data. This can mean, for example, modelling separate regression lines for 85 different examination syllabuses for one age group, because any aggregation can be shown to represent unfair comparisons. Ethical issues are surprisingly often lurking in technical decisions. For example, reporting outcomes from a continuous measure in terms of the percent of students who surpassed a certain level, produces unethical behavior: a concentration of teaching on borderline students. Distortion of behavior and data corruption are ever-present concerns in indicator systems. The systems we describe would have probably failed to thrive had they not addressed schools' on-going concerns about education. Moreover, data interpretation can only be completed in the schools, by those who know all the factors involved. Thus the commitment to working closely and collaboratively with schools in "distributed research" is important, along with "measuring what matters"... not only achievement. In particular the too-facile interpretation of correlation as causation that characterized much school effectiveness research had to be avoided and the need for experimentation promoted and demonstrated. Reasons for the exceptionally warm welcome from the teaching profession may include both threats (such as the unvalidated inspection regime run by the Office for Standards in Education) and opportunities (such as site based management).

Quantifying Quality: What Can the U.S. News and World Report Rankings Tell us About the Quality of Higher Education?

Abstract: Since their first appearance in 1983, the U.S. News and World Report rankings of colleges and graduate schools have generated much discussion and debate, from some declaring them among the best rankings ever published to others describing them as shallow, inaccurate, and even dangerous. The research presented here addresses two of the most common criticisms of the methodology used to produce these rankings. In particular, this study answers the following questions: What is the extent of change in U.S. News' ranking formulas across years and what are the implications for interpreting shifts in a school's rank over time? How precise is the overall score that U.S. News uses to rank schools and what are the implications for assigning schools to discrete ranks? Findings confirm critic's concerns in each of these areas, particularly in relation to the ranking of graduate schools of education. Based on these results, five recommendations are made for improving the interpretability and usefulness of the rankings.

¿Exito en California? A Validity Critique of Language Program Evaluations and Analysis of English Learner Test Scores

Abstract: Several states have recently faced ballot initiatives that propose to functionally eliminate bilingual education in favor of English-only approaches. Proponents of these initiatives have argued an overall rise in standardized achievement scores of California's limited English proficient (LEP) students is largely due to the implementation of English immersion programs mandated by Proposition 227 in 1998, hence, they claim Exito en California (Success in California). However, many such arguments presented in the media were based on flawed summaries of these data. We first discuss the background, media coverage, and previous research associated with California's Proposition 227. We then present a series of validity concerns regarding use of Stanford-9 achievement data to address policy for educating LEP students; these concerns include the language of the test, alternative explanations, sample selection, and data analysis decisions. Finally, we present a comprehensive summary of scaled-score achievement means and trajectories for California's LEP and non-LEP students for 1998-2000. Our analyses indicate that although scores have risen overall, the achievement gap between LEP and EP students does not appear to be narrowing.

Lake Woebe guaranteed: Misuse of Test Scores in Massachusetts, Part I

Abstract: Misuse of test results in Massachusetts largely guarantees woes for both students and schools. Analysis of annual test score averages for close to 1000 Massachusetts schools for four years (1998—2001) shows that test score gains in one testing period tend to be followed by losses in the next. School averages are especially volatile in relatively small schools (with less than 150 students tested per grade). One of the reasons why scores fluctuate is that the Massachusetts state test has been developed using norm-referenced test construction procedures so that items which all students tend to answer correctly (or incorrectly) are excluded from operational versions of the test. This article concludes with a summary of other reasons why results from state tests, like that in Massachusetts, ought not be used in isolation to make high-stakes decisions about students or schools.

Charter School Funding Issues.

Abstract: Although a great deal has been written about charter schools, rather little attention has been given to their funding. The first part of this article raises four current issues in the funding of regular public schools across the U.S. and shows how these issues carry over to the funding of charter schools. The second part explores four additional issues that have arisen in the funding of charter schools that go to the core identity of charter schools and the nature of the students they enroll. In both parts, extra attention is paid to developments in California, one of the most active charter school states.

The Varieties of Knowledge and Skill-Based Pay Design: A Comparison of Seven New Pay Systems for K-12 Teachers.

Abstract: A number of lines of research (e.g., National Commission on Teaching and America's Future, 1996; Slavin & Fashola, 1998; Wright, Horn, & Sanders, 1997; Bembry, Jordan, Gomez, Anderson, & Mendro, 1998; Ferguson & Ladd, 1996) have identified teacher instructional capacity as a key variable in the success of educational reforms in improving student achievement. Since 2000, the Consortium for Policy Research in Education has been studying a new form of teacher compensation that may have the potential to support improvements in the capacity of teachers to deliver instruction that would enable all children to achieve to high academic standards, as well as to respond to the growing public concern that there be some link between teacher salaries and teacher performance. This innovation -knowledge and skill-based pay -rewards teachers with base pay increases and/or bonuses for acquiring and demonstrating specific knowledge and skills needed to meet educational goals, such as improving student achievement. The application of this pay concept to K-12 education has been suggested by Conley and Odden (1995), Mohrman, Morhman, and Odden (1996), and Odden and Kelley (1997). This report examines a study of seven knowledge and skill-based pay systems for teachers that have been developed by U.S. schools or districts. Disciplines Curriculum and Instruction | Teacher Education and Professional Development Comments View on the CPRE website. This report is available at ScholarlyCommons: http://repository.upenn.edu/cpre_researchreports/29 The Varieties of Knowledge and Skill-based Pay Design: A Comparison of Seven New Pay Systems for K-12 Teachers

The Politics of School-Based Management: Understanding the Process of Devolving Authority in Urban School Districts.

Abstract: Since the late 1970s the problem of urban education has been cast as partially a problem of governance and authority structures. This focus mirrors a larger preoccupation by educational reformers with democratizing the decision-making process in public schools, a preoccupation that is evident not only in this country but also many nations throughout the world. Borrowing from the private sector, the underlying assumption behind decentralization is that educational improvement is only possible if those closest to the point at which decision are enacted become the architects of these decisions. Thus,

The Question of the Student In Educational Reform

Abstract: In pursuing the goals of educational reform over the past several decades, educational policy makers have focused on teachers, administrators, and school structures as keys to higher educational achievement. As the would-be beneficiaries of reform, students, and their interaction with the educational system, have been almost entirely overlooked in the pursuit of educational excellence. Yet, as we argue, students are as causally central as educators in bringing about higher educational achievement. In what follows, we examine rational student interaction with the educational system and show why a large number of students have incentives to undercut the intent of the reforms. These are incentives created by our development of an educationally-based, meritocratic social and economic system. No one, apparently, is asking what exactly is in the reforms from the point of view of quite rational, if sometimes irresponsible, student self-interest. Indeed, the eduationally-based, meritocratic social and economic system may be actually forming student preferences guaranteed to result in educational mediocrity rather than excellence. Finally, we comment upon the meaning of "educational excellence" and show why the educational reformers' understanding of the purpose of public education—to compete in the global economic system—can only fail to capture it. In doing so, we point to the kinds of educational structures and policies that create multiple pathways to competent adulthood that do have a chance of bringing about the reformers' stated goal of excellence in the educational system. But these are structures and policies that challenge the entire conceptual framework of the current educational reform movement.

Are Increasing Test Scores in Texas Really a Myth

Abstract: Pass rates by Texas tenth-graders on the high school exit exam improved from 52 percent in 1994 to 72 percent in 1998. In his article "The Myth of the Texas Miracle in Education" (EPAA, August 2000) Professor Walt Haney argued that some part of this increased pass rate was, as he put it, an illusion. Haney contended that the combined effects of students dropping out of school prior to taking the 10th grade TAAS and special education exemptions accounted for much of the increase in TAAS pass rates. Relying on the same methodology and data that Haney used, we demonstrate that his conclusion is incorrect. None of the 20 percent improvement in the TAAS exit test pass rate between 1994 and 1998 is explained by combined increases in dropout rates or special education exemptions.

Overview of Private Education Development in Modern China

Abstract: It is not surprising that private education is gaining importance in China given the overall context of huge national efforts toward building up a “socialist market economy.” However, the fast growth rate in both the quantities and the qualities of profitable private schools in a socialist society is beyond what people usually expect. This paper looked into the modern history of private education in China and found that such a huge resurgence of private education is rooted in the heritage of private education in the Chinese society. Private schools were the precursor of modern Chinese education. They played an important role in the country for most of the time. When the government policy became more flexible and household income increased substantially, such a heritage revived and becomes a stimulating factor in the education sector.

The Amalgamation of Chinese Higher Education Institutions

Abstract: The 1990s witnessed revolutionary change in China's higher education system, particularly through radical mergers. The reform process and its background are detailed here, with a case study focusing on Zhejiang University. After nearly 15 years of painstaking effort, the reform goals for the higher education system have been met, and a decentralized, two-tiered administrative system has been installed. However, the most hotly debated reform has been the amalgamation of universities. The need to optimize China's system of higher education has a background dating back about 50 years, when the first reordering of higher education took place. The reordering and its results are described, and the causes and after effects of this reform are detailed.

Failing Georgia—The Case Against the Ban on Social Promotion

Abstract: Our analysis begins with an examination of the state of Georgia's rationale for the decision regarding social promotion that was based on the perceived views that teachers have on the issue. Research suggests, however, that teachers hold contradictory opinions concerning the use of standardized tests for high stakes decisions, such as promotion, and are not aware of the consequences most children suffer when they fail a grade. Following a discussion that challenges the claims of success in Chicago, Baltimore, and Texas, we explore the viability of choosing litigation as a strategy to stop the use of high stakes tests given the adverse impact they have on protected minorities. From a study of the thirty-nine poorest counties in rural Georgia, the relationships between poverty, race and the Georgia Criterion Referenced Competency Test Results suggest that these tests do have an enormously disparate impact on impoverished African American children. Because chances for educational attainment will be severely limited by this test, most African American children will be discouraged from achieving a high school diploma. As a way to put a face on the data, a case study of a young girl who would probably fail her grade in school if the law was enforced is presented followed by recommendations that argue for changes in education policy and teaching. Rather than mandate a ban on social promotion, the state of Georgia should pursue improvement of socio-economic conditions, education policy and pedagogy.

Power-discourse Relationship in a Croatian Higher Education Setting

Abstract: Croatian higher education system's public space is researched through a critical analysis of a Croatian faculty's discourse. Representing a typical faculty social situation, two council meetings recorded in minutesare critiqued. Both meetings' minutes provide evidence of discourse strategies of deception used by faculty power holders to create an illusion of consent. We attribute the success of the deception to council members' ideas about the Faculty's groups/individuals, relations and issues related to the Faculty's hierarchy, their rank within that hierarchy, and their position within the Faculty's social network. To support our argument, we explore how the Faculty power holders' discourse is built on a power/ideology/language formation. We conclude that, failing to critique the faculty's discourse, council members neglected their historical task of paving the way to democracy.

Mentoring Narratives ON-LINE:Teaching the Principalship

Abstract: The need to develop new models for preparation of school administrators has been a prominent concern in educational discourse in the last decade. Having been criticized for the inadequate preparation of the school leadership cadre, academic departments responsible for training future school administrators have had to revisit their approaches and to reframe their teaching philosophies to ensure the readiness of their graduates for the challenges and complexities of school leadership. This article reports on the new model of principals' training that has been used in York University's Principals' Qualification Program (PQP) from the late 1990s onward. One component of the program brings traditional case methodology into a computer-mediated/on-line environment. The on-line cases are narratives from the everyday lives of the Ontario school administrators who serve as mentors in the on-line environment. Situating our discussion within the context of the rapidly changing educational landscape of Ontario, we focus on the PQP model to explore experientially generated case narratives as one method for teaching and learning the work of the local school administrator. We focus particularly on the teaching and learning embedded in computer-mediated or on-line case narratives used in training teachers for school leadership. We argue that the complexities of school leadershipthe social, cultural, relational, ethical and moral context of school leadershipcan be taught effectively through the reflective processes of on-line case narratives. We seek to contribute to the ongoing dialogue on the potential of new pedagogies and new technologies to help prepare the competent and responsible leaders for tomorrow's schools.