Showing papers in "Educational Evaluation and Policy Analysis in 2003"

Resources, Instruction, and Research:

Abstract: Many researchers who study the relations between school resources and student achievement have worked from a causal model, which typically is implicit. In this model, some resource or set of resources is the causal variable and student achievement is the outcome. In a few recent, more nuanced versions, resource effects depend on intervening influences on their use. We argue for a model in which the key causal agents are situated in instruction; achievement is their outcome. Conventional resources can enable or constrain the causal agents in instruction, thus moderating their impact on student achievement. Because these causal agents interact in ways that are unlikely to be sorted out by multivariate analysis of naturalistic data, experimental trials of distinctive instructional systems are more likely to offer solid evidence on instructional effects.

Leadership of Inquiry: Building and Sustaining Capacity for School Improvement:

Abstract: This article reports on findings from a longitudinal study of leadership in the context of a region-wide school renewal effort entitled the Bay Area School Reform Collaborative (BASRC). BASRC's the...

Distributed Leadership in Schools: The Case of Elementary Schools Adopting Comprehensive School Reform Models:

Abstract: This is a study of distributed leadership in the context of elementary schools' adoption of comprehensive school reforms (CSR). Most CSRs are designed to configure school leadership by defining formal roles, and we hypothesized that such programs activate those roles by defining expectations for and socializing (e.g., through professional development) role incumbents. Configuration and activation were further hypothesized to influence the performance of leadership functions in schools. Using data from a study of three of the most widely adopted CSR models, support was found for the configuration and activation hypotheses. Leadership configuration in CSR schools differed from that of nonCSR schools in part because of the addition of model-specific roles. Model participation was also related to the performance of leadership functions as principals in CSR schools and CSR-related role incumbents were found to provide significant amounts of instructional leadership. Further support for the activation hypothesi...

Relational Networks, Social Trust, and Norms: A Social Capital Perspective on Students’ Chances of Academic Success

Abstract: This study elaborates a theoretical rationale for relational networks, norms, and trust as structural and functional forms of social capital that can facilitate student achievement. The results of hierarchical generalized linear modeling show that 4th-grade students’ odds of passing state-mandated mathematics and writing assessments are modestly increased in urban schools characterized by high levels of social capital. The results suggest the need for more research investigating the extent to which social capital is independent from socioeconomic status and whether social capital may be developed in schools serving high concentrations of poor and minority students.

Leadership Content Knowledge.

Abstract: Drawing inspiration from Shulman’s (1986) construct of pedagogical content knowledge, we propose that leadership content knowledge is a missing paradigm in the analysis of school and district leadership. After defining leadership content knowledge as that knowledge of academic subjects that is used by administrators when they function as instructional leaders, we present three cases of instructional leadership—situated at different school and district levels—and examine each for evidence of leadership content knowledge in use. Based on a cross-case analysis, we argue that as administrative levels increase and functions become broader, leadership content knowledge becomes less fine-grained, though always anchored in knowledge of the subject, how it is learned (by adults as well as students), and how it is taught. We go on to suggest that all administrators have solid mastery of at least one subject (and the learning and teaching of it) and that they develop expertise in other subjects by “postholing,” that...

Teacher Effects and Teacher Effectiveness: A Validity Investigation of the Tennessee Value Added Assessment System

Abstract: This article addresses the validity of teacher evaluation measures produced by the Tennessee Value Added Assessment System (TVAAS). The system analyzes student test score data and estimates the effects of individual teachers on score gains. These effects are used to construct teacher value-added measures of teaching effectiveness. We describe the process of generating teacher effectiveness estimates in TVAAS and discuss policy implications of using these estimates for accountability purposes. Specifically, the article examines the TVAAS definition of teacher effectiveness, the mechanism employed in calculating numerical estimates of teacher effectiveness, and the relationships between these estimates and student ability and socioeconomic background characteristics. Our validity analyses point to several logical and empirical weaknesses of the system, and underscore the need for a strong validation research program on TVAAS.

Studying Large-Scale Reforms of Instructional Practice: An Example from Mathematics and Science

Abstract: A number of challenges are encountered when evaluating a large-scale, multisite educational reform aimed at changing classroom practice. The challenges include substantial variability in implementa...

Making Sense of Distributed Leadership: The Case of Peer Assistance and Review.

Abstract: This article explores a case of shifting leadership responsibility for teacher evaluation. Peer Assistance and Review (PAR) formally involves teachers in the summative evaluation of other teachers—although the boundaries of the involvement are often vague. Since teacher evaluation has traditionally been the domain of school principals, involving teachers in teacher evaluation raises questions about how those faced with the new role make sense of it and enact it. The article draws on theories of professions, organizations, and institutions to examine the implementation of PAR in one large urban school district. Findings suggest that, despite positive sentiments about the policy across stakeholder groups, those involved wanted principals to remain a central figure in the evaluation of teachers in PAR. Education’s hierarchical norms, the difficulty of conducting evaluations, district leadership, and program ambiguity are identified as challenges to distributing leadership.

Counterfeit Social Capital and Mexican-American Underachievement:

Abstract: A critical issue facing U.S. schools and one with broad social implication is the persistent disparity in educational achievement between racial/ethnic groups. The achievement gap may be particular...

Comprehensive School Reform in Culturally and Linguistically Diverse Contexts: Implementation and Outcomes from a Four-Year Study:

Abstract: This article presents findings from a 4-year study of 13 culturally and linguistically diverse elementary schools implementing comprehensive school reform (CSR) models. The study focused on: (a) the actions at the state and district levels that facilitated or inhibited reform implementation; (b) the adaptability of the various reforms in multicultural, multilingual contexts; and (c) the student achievement outcomes associated with reform, for schools as a whole and for language minority students in particular. Some schools implemented reforms and bilingual education programs in mutually supportive ways; others had difficulty adapting reforms to suit the needs of Limited English Proficient (LEP) students. Reforms generally helped educators meet goals for multicultural education, but in some cases, educators’ beliefs about student ability, race, and language served as constraints to reform. Students from CSR schools had achievement outcomes that were generally equivalent to those for students from matched c...

Does Competition Improve Teaching and Learning? Evidence from New Zealand

Abstract: Central to the argument for more competition in education is that it will induce schools to provide higher quality education at no greater cost. This article sheds new light on this issue by measuring how competition among New Zealand’s schools affected student learning as perceived by teachers and principals. The analysis builds on the fact that New Zealand’s introduction of full parental choice in 1992 increased competitive pressures more for some schools than for others. With careful attention to various potential threats to validity, we conclude that competition—as perceived by teachers—generated negative effects on the quality of student learning and other aspects of schooling in New Zealand’s elementary schools.

Raising the Bar and Equity? Effects of State High School Graduation Requirements and Accountability Policies on Students' Mathematics Course Taking

Abstract: In response to the national push to raise academic perfornance of all students, most states have adopted policies designed to raise academic standards, monitor progress toward those standards, and hold schools and students responsible for attaining them. Given the complex nature of the educational process, these policies are likely to have mixed effects on both general levels of attainment and stratification based on race or ethnicity and social class. Using nationally representative longitudinal data and hierarchical linear modeling, this article explored the association between students' mathematics course work and states' high school graduation requirements and assessment or accountability policies. We found that students in states with more graduation requirements tended to enroll in higher level mathematics courses as freshmen and persist to take more advanced level courses. Similar trends were also found for students in states that link test perfornance to consequences for schools. Extensive testing, however, had little effect on course taking except to increase differences based on socioeconomic status. In contrast, differences between racial or ethnic groups tended to be smaller in states where test perfornance was linked to consequences for students.

Schools in Transition: Reform Efforts and School Capacity in Washington State:

Abstract: This article features case studies of two elementary schools, identified as exemplary by Washington educators, as they worked to enact Washington’s Essential Academic Learning Requirements and help...

Examining Relationships between Where Students Start and How Rapidly They Progress: Using New Developments in Growth Modeling to Gain Insight into the Distribution of Achievement within Schools.

Abstract: Studying change in student achievement is of central importance in numerous areas of educational research, including efforts to monitor school performance, investigations of the effects of educational interventions over time, and school effects studies focusing on how differences in school policies and practices relate to differences in student progress. In this article, we argue that in studying patterns of change, it is often important to consider the relationship between where students start (i.e., their initial status) and how rapidly they progress (i.e., their rates of change). Drawing on recent advances in growth modeling methodology, we illustrate the potential value of such an approach in the context of monitoring school performance. In particular, we highlight the ways in which attending to initial status in analyses of student progress can help draw attention to possible concerns regarding the distribution of achievement within schools. To convey the logic of our approach and illustrate various ...

A Nation at Rest: The American Way of Homework

Abstract: We use several national surveys to provide a 50-year perspective on time spent on homework. The great majority of American children at all grade levels now spend less than one hour studying on a typical day—an amount that has not changed substantially in at least 20 years. Moreover, high school students in the late 1940s and early 1950s studied no more than their counterparts did in the 1970s, 1980s, and 1990s. Changes in educational opinion on homework over the last half century have had little effect on student behavior, with only two notable exceptions: a temporary increase in homework time in the decade following Sputnik, and a new willingness in the last two decades to assign small amounts to primary-grade students.

The Influence of High School Graduation Requirement Policies in Mathematics and Science on Student Course-Taking Patterns and Achievement:

Abstract: One of the most widely implemented educational reform efforts of the last two decades has been the adoption of increased high school graduation requirements, especially in mathematics and science. The present study examines the effect of that reform. More particularly, this article first investigates the extent to which a commonly adopted requirement that students complete three credits in mathematics and science was implemented at schools. Using multilevel regression analysis, the study then examines the relationship between three-course requirements in mathematics and science and three expected outcomes: increases in the number of credits students earned in mathematics and science, increases in the level of mathematics and science classes completed by students, and increases in student achievement in math and science, as measured by 8th- to 12th-grade test score gains. To test this relationship, the author drew a nationally representative sample of 1992 public high school graduates from the National Edu...

The Effect of Early Childhood Intervention and Subsequent Special Education Services: Findings from the Chicago Child-Parent Centers

Abstract: This article explores patterns of special education services during the elementary grades among children who participated in either the Child-Parent Center (CPC) Preschool Program or other early childhood programs in the Chicago Public Schools. The study sample included 1,377 low-income, racial minority children in the Chicago Longitudinal Study. Controlling for family background characteristics that might affect educational performance, children who participated in Child-Parent Center preschool had a significantly lower rate of special education placement (12.5%) than the comparison group (18.4%), who participated in an alternative all-day kindergarten program. The estimated impact of CPC preschool intervention was best explained by the cognitive advantage hypothesis. This article provides support for the long-term impact of the CPC preschool intervention on special education outcomes.

Leaders as Learners: An Inquiry into the Formation and Transformation of Principals' Professional Perspectives.

Abstract: This article reports on a research project that explored the manner in which veteran school principals become aware of, and committed to, new perspectives and practices. Personal stories of formative and transformative change from the careers of 23 school principals were collected and analyzed comparatively. Four analytic tools were devised that can support future studies of principal learning. The findings indicate that these principals’ professional learning is influenced by (a) impressions of teachers and schooling formed in early childhood, (b) progression through a common perspective development sequence with patterned influences, (c) a personal orientation to learning, which can be seen as a combination of four dynamics—connections, context, characteristics, and concepts—with one dominating the others, and (d) a story about oneself as a learner—the learning story—that serves as both a guide for approaching novelty and a catalyst for change. Implications for scholars and policymakers are discussed.

Methodological Thoughts on Measuring the Impact of Private Sector Competition on the Educational Marketplace

Abstract: Numerous researchers have speculated on and attempted to quantify the impact ofpolicies designed to increase school choice and competition between schools, and they have reached widely differing conclusions. In this article, we provide an overview of the research that focuses on the potential achievement effects of greater private sector competition on K-12 schooling. In particular, we explore the various methodologies that have been used to assess the relative effectiveness of public and private schools. We discuss the strengths and shortcomings of different approaches, and based on this, we ponder what is known and not known about the impact of increased choice and competition. In conclusion, we maintain that although many of the methodologies used to assess the effects of particular interventions, such as educational vouchers, are sound, they likely fail to capture the general equilibrium consequences of enhanced choice. As a result, many of the questions we would want answered to inform policymaking remain unknown.

Making the Grade: Comparing DC Charter Schools to Other DC Public Schools:

Abstract: Across the United States, charter schools have become one of the most frequently used means of increasing choice among educational alternatives In this article we use data from a recent telephone survey of Washington DC parents to evaluate the success of the District’s large and growing charter school program We find that parents with children in charter schools rate their teachers, principals, facilities and schools higher than their traditional public counterparts This finding is robust even when controlling for self-selection into charter schools Based on these empirical results, we argue that the greater satisfaction with charter schools reflected in these differences in grades is not simply the result of the act of choosing

Transition Leadership in a Shifting Policy Environment

Abstract: While transformational leadership has received much attention in recent years, the purpose of this article is to explicate the concept of transition leadership and its centrality to understanding p...