Showing papers in "Assessment in Education: Principles, Policy & Practice in 2010"
TL;DR: The authors examined assessment education at one pre-service teacher education program in Ontario, Canada and identified teacher candidates' perceived confidence levels in assessment practice, theory, and philosophy, and these teacher candidates also provided their views towards assessment topics that it is important to include in a pre-'service educational assessment course.
Abstract: As a result of the standards‐based movement in education there is an increased need for teacher competency in the area of student assessment and evaluation. This study examines assessment education at one pre‐service teacher education program in Ontario, Canada. Through a questionnaire administered to 288 teacher candidates, this study identified teacher candidates’ perceived confidence levels in assessment practice, theory, and philosophy. These teacher candidates also provided their views towards assessment topics that it is important to include in a pre‐service educational assessment course. Descriptive statistics, factor analysis, and analysis of variance were used to analyse questionnaire responses. In general, findings support the need for direct instruction in assessment with specific topics identified (e.g. reporting achievement, modifying assessments, developing constructed‐response items, item reliability, validity, articulating a philosophy of assessment, etc.) as important to developing teache...
226 citations
TL;DR: The authors discuss and analyse recorded talk in teacher moderation meetings showing the processes that teachers use as they work with stated standards to award grades (A to E) and show how they move to and fro between supplied textual artefacts, including stated standards and samples of student responses, drawing into the moderation, and social processes of dialogue and negotiation.
Abstract: There is a strong quest in several countries including Australia for greater national consistency in education and intensifying interest in standards for reporting. Given this, it is important to make explicit the intended and unintended consequences of assessment reform strategies and the pressures to pervert and conform. In a policy context that values standardisation, the great danger is that the technical, rationalist approaches that generalise and make superficial assessment practices, will emerge. In this article, the authors contend that the centrality and complexity of teacher judgement practice in such a policy context need to be understood. To this end, we discuss and analyse recorded talk in teacher moderation meetings showing the processes that teachers use as they work with stated standards to award grades (A to E). We show how they move to and fro between (1) supplied textual artefacts, including stated standards and samples of student responses, (2) tacit knowledge of different types, drawing into the moderation, and (3) social processes of dialogue and negotiation. While the stated standards play a part in judgement processes, in and of themselves they are shown to be insufficient to account for how the teachers ascribe value and award a grade to student work in moderation. At issue is the nature of judgement as cognitive and social practice in moderation and the legitimacy (or otherwise) of the mix of factors that shape how judgement occurs.
148 citations
TL;DR: This article investigated the relationship between middle school students' scores for a written assignment (N = 162) and a process that involved students in generating criteria and self-assessing with a rubric.
Abstract: This study investigated the relationship between middle school students’ scores for a written assignment (N = 162) and a process that involved students in generating criteria and self‐assessing with a rubric. Gender, time spent writing, grade level, prior rubric use, and previous achievement in English were also examined. The treatment involved using a model essay to scaffold the process of generating a list of criteria for an effective essay, reviewing a written rubric, and using the rubric to self‐assess first drafts. The comparison condition involved generating a list of criteria and reviewing first drafts. Findings include a main effect of treatment, gender, grade level, writing time, and previous achievement on total essay scores, as well as main effects on scores for every criterion on the scoring rubric. The results suggested that reading a model, generating criteria, and using a rubric to self‐assess can help middle school students produce more effective writing.
120 citations
TL;DR: This article explored and developed teachers' understanding and practices in their summative assessments, which are used on a regular basis within schools for guiding the progress of pupils and for internal accountability, and found that teachers understand validity and formulate their classroom assessment practices in the light of that understanding, and also consider how that understanding might be challenged and developed.
Abstract: This paper describes some of the findings of a project which set out to explore and develop teachers’ understanding and practices in their summative assessments. The focus was on those summative assessments that are used on a regular basis within schools for guiding the progress of pupils and for internal accountability. The project combined both intervention and research elements. The intervention aimed both to explore how teachers might improve those practices in the light of their re‐examination of their validity, and to engage them in moderation exercises within and between schools to audit examples of students’ work and to discuss their appraisals of these examples. This paper reports findings, arising from this work, of the research that aimed to study how teachers understand validity, and how they formulate their classroom assessment practices in the light of that understanding. The paper also considers how that understanding might be challenged and developed. It was found that teachers’ attention ...
112 citations
TL;DR: In this article, an achievement test score can be viewed as a joint function of skill and will, of knowledge and motivation, but when interpreting and using test scores, the 'will' part is not always acknowled
Abstract: An achievement test score can be viewed as a joint function of skill and will, of knowledge and motivation. However, when interpreting and using test scores, the 'will' part is not always acknowled ...
97 citations
TL;DR: In this paper, the impact of formative assessment on student learning in physical education has been investigated in a post-primary physical education planning framework with assessment instruments for use by teachers, and a number of teachers were engaged in the development of assessment and planning materials, the trialling of these in school settings and their subsequent refinement based on the feedback received from the teaching and learning setting.
Abstract: This study arose in response to a perceived need for additional teacher support for assessment in physical education and the limited focus in physical education pedagogy literature on the impact of Assessment for Learning (AfL), in particular the impact of formative assessment on student learning. The study involved the refinement and evaluation of a post‐primary physical education planning framework with assessment instruments for use by teachers. A number of teachers were engaged in the development of assessment and planning materials, the trialling of these in school settings and their subsequent refinement based on the feedback received from the teaching and learning setting. The study was contingent on teachers cultivating a learning culture within their class. Students’ and teachers’ experiences of AfL are reported before highlighting some of the challenges that remain in investigating formative assessment.
86 citations
TL;DR: This paper examined the implementation of reform in mathematics in grades 7-10 in Ontario, Canada and found that teachers use a variety of forms of assessment and using assessment to improve student learning.
Abstract: This article responds to calls that have been made for research into how teachers incorporate new assessment ideas into their practice. We draw on a large‐scale study that examined the implementation of reform in mathematics in grades 7–10 in Ontario, Canada. We present teacher questionnaire data, and focus on data gathered from case studies for details of what new assessment practices look like in classrooms. We show teachers using a variety of forms of assessment and using assessment to improve student learning. Observed assessment practices went beyond tests to include the use of journals, observation, questioning, self‐assessment and unique forms of ‘quizzes’. These practices allowed teachers to examine students’ mathematical thinking and provided feedback to students and teachers to improve student learning. We also observed the important role of collaboration, coherence, and the teachers’ beliefs in supporting new assessment practices.
75 citations
TL;DR: In this article, six remedial reading teachers in a large, rural school district participated in a form of professional development called Teaching as Intentional Learning, based on an inquiry process.
Abstract: Six remedial reading teachers in a large, rural school district participated in a form of professional development called Teaching as Intentional Learning, based on an inquiry process. Their topic of inquiry was formative assessment. Professional development comprised both direct instruction and inquiry learning in teachers’ own classrooms. This study describes the strategies they experimented with, their professional growth in formative assessment, and effects on students. All six teachers showed important professional growth, as indicated by their own reflections and also by their supervisor’s observations. In First Grade, at‐risk students assigned to these project teachers had increased reading readiness scores on one measure (DIBELS PSF) compared with at‐risk students assigned to non‐project teachers.
68 citations
TL;DR: The authors survey secondary teachers' beliefs of classroom and large-scale tests for providing information about students' learning processes, influencing meaningful student learning, and eliciting learning or test-taking strategies for successful test performance.
Abstract: Classroom teachers are in the front line of introducing students to formal learning, including assessments, which can be assumed to continue for students should they extend their schooling past the expected mandatory 12 years. The purpose of the present investigation was to survey secondary teachers’ beliefs of classroom and large‐scale tests for (a) providing information about students’ learning processes, (b) influencing meaningful student learning, and (c) eliciting learning or test‐taking strategies for successful test performance. Secondary teachers were surveyed because a majority of large‐scale tests are developed for secondary students (e.g., PISA, TIMSS). Results suggested that in comparison to large‐scale tests teachers believe classroom tests provide more information about student learning processes, are more likely to influence meaningful student learning, and are more likely to require learning over test‐taking strategies. The implications of these results for assessment literacy are explored.
55 citations
TL;DR: In this paper, the authors compare the theoretical development of the concept of validity with the methodology adopted in validity studies over time, and conclude that practices seem to have followed theory when it comes to how th...
Abstract: The concept of validity, as described in the literature, has changed over time to become a broad and rather complex issue. The purpose of this paper is to investigate if practice has followed theory, or if there is a gap between validity in theory and validity in practice. It compares the theoretical development of the concept of validity with the methodology adopted in validity studies over time. Important phases in the history of validity, and also common arguments for and against traditional and modern validity perspectives, are presented and discussed. Thereafter, three Swedish research projects aiming to validate instruments used for selection to higher education are described. The idea is to use these projects as examples of contemporary practice, and to compare their designs, research questions and outcomes with how validity was theoretically described during their specific period of time. The conclusions from these comparisons are that practices seem to have followed theory when it comes to how th...
55 citations
TL;DR: The Chilean national learning outcome assessment system Sistema de Medicio de Calidad de la Educación (SIMCE) has carried out census-based assessments since 1988 and publishes the results at both t...
Abstract: The Chilean national learning outcome assessment system Sistema de Medicio de Calidad de la Educacion (SIMCE) has carried out census‐based assessments since 1988 and publishes the results at both t...
TL;DR: In this paper, the experiences and opinions of year 6 teachers, primary headteachers and primary science coordinators as a contribution to the debate about the effect of statutory national testing in England and its abolition in Wales were analysed.
Abstract: The research aims to articulate the experiences and opinions of Year 6 (Y6) teachers, primary headteachers and primary science coordinators as a contribution to the debate about the effect of statutory national testing in England and its abolition in Wales A multi‐faceted approach of quantitative and qualitative data collection and analysis was used in England and Wales including a telephone survey of 600 respondents and eight focus group interviews involving a total of 74 participants The findings of this research provide insights into the strategies adopted by Y6 teachers in England in ensuring pupils gain the highest possible scores in high‐stakes national tests, used as the basis of published Schools and Colleges Assessment and Attainment Tables at Key Stage 2 (KS2) Test preparation is perceived by teachers to narrow the curriculum, and for science in particular, as the inclusion of only those aspects of science likely to be included in paper and pencil tests is said to have reduced many aspects of
TL;DR: In this paper, the authors present a profile of the assessment system in England, focusing on those aspects that are related to government policy and argue that successive governments have increasingly intervened directly in curriculum and qualifications development, and observes the growing governmental policy influence over assessment issues.
Abstract: This profile explains the assessment system in England, concentrating on those aspects that are related to government policy. It begins by putting the system in context; it then describes the national educational structure, curriculum and assessment arrangements. The government agencies responsible for carrying out education policies are introduced. In order to illustrate the intersection of curriculum and assessment development and political policy, the profile describes and discusses assessment issues that highlight the role of government and its agencies in the development, implementation and monitoring of England's national curriculum tests and 14‐to‐19 qualifications. It argues that successive governments have increasingly intervened directly in curriculum and qualifications development, and observes the growing governmental policy influence over assessment issues and the repercussions of that influence.
TL;DR: In this article, an evidence-based approach to targeting primary school students' developmental reading comprehension levels for effective learning is described, and the centrality of a professional learning team approach to change for the student, teacher and school is discussed.
Abstract: Outcomes and findings from an evidence‐based approach to targeting primary school students’ developmental reading comprehension levels for effective learning are described. Nineteen schools participated in a literacy assessment project designed to monitor and improve the reading comprehension achievement levels of their students. The project integrated a developmental approach to learning and teaching, information derived from standardised reading comprehension assessments, and professional development for teachers. Reading comprehension achievement across the schools increased at a higher rate than typically expected. Teacher discourse about teaching and learning changed from discrete skill and resource focused, to developmentally focused. The centrality of a professional learning team approach to change for the student, teacher and school is discussed.
TL;DR: In this paper, the authors synthesize findings regarding the development of competence and learned helplessness and factors influencing persistence and intrinsic motivation, suggests the process through which small differences in early achievement are magnified by the current structure of schools, and reviews evidence suggesting that the characteristics of a specific type of individualized instruction and assessment system may be especially suited to remediate these differences.
Abstract: This article synthesizes findings regarding the development of competence and learned helplessness and factors influencing persistence and intrinsic motivation, suggests the process through which small differences in early achievement are magnified by the current structure of schools, and reviews evidence suggesting that the characteristics of a specific type of individualized instruction and assessment system may be especially suited to remediate these differences. Age‐graded schools and group tests label students as ‘below’ and ‘above’ average, inadvertently demoralizing below‐average students, depressing effort and achievement, and perpetuating the gap in achievement between poor students and their more affluent peers. Analysis of the research literature suggests that the psychological experience of school for both high and low achieving students may be altered through a structure where instruction is individualized, students are challenged at their own levels, and each student receives objective asses...
TL;DR: This article used the interactive potential of computers to measure how much help a student needs to complete a task, where the computer can offer several kinds of support, such as help with understanding a question, hints on the meanings of key concepts, and examples or analogies.
Abstract: The two most common models for assessment involve measuring how well students perform on a task (the quality model), and how difficult a task students can succeed on (the difficulty model). By exploiting the interactive potential of computers we may be able to use a third model: measuring how much help a student needs to complete a task. We assume that every student can complete it, but some need more support than others. This kind of tailored support will give students a positive experience of assessment, and a learning experience, while allowing us to differentiate them by ability. The computer can offer several kinds of support, such as help with understanding a question, hints on the meanings of key concepts, and examples or analogies. A further type of support has particular importance for test validity: the computer can probe students for a deeper explanation than they have so far given. In subjects like geography or science, markers often would like to ask ‘yes, but why?’, suspecting that students ...
TL;DR: In this paper, the authors investigated three aspects of transparency in relation to an examination methodology for student teachers: self-assessment criteria, a scoring rubric, and exemplars.
Abstract: If the aim of education is for all students to learn and improve, then the expectations must be transparent to the students. In this study, three aspects of transparency are investigated in relation to an examination methodology for student teachers: self‐assessment criteria, a scoring rubric, and exemplars. The examinations studied were carried out in 2004, 2005, and 2006 respectively, all with a cohort of first year student teachers (n = 170, 154, and 138). There was a large difference in scores between the 2004 and 2005 cohorts (d = 3.21), when changes in the examination were implemented. The comparison between 2005 and 2006, when no changes were made, does not show a corresponding difference (d = .27). These results suggest that, by making the assessment more transparent, students’ performances could be greatly improved.
TL;DR: Teacher assessment, particularly in the shape of formative assessment, has received a great deal of research attention over the past couple of decades Much of the literature has had strong prescriber as mentioned in this paper.
Abstract: Teacher assessment, particularly in the shape of formative assessment, has received a great deal of research attention over the past couple of decades Much of the literature has had strong prescri
TL;DR: This article explored children's understanding of formative assessment and its effects on their learning, in order to establish what learners perceive as important to help them improve their learning and found that children have a good understanding of the concepts of feedback, and can articulate clearly the elements most useful for helping them to improve learning.
Abstract: This study was designed to explore children's understanding of formative assessment and its effects on their learning, in order to establish what learners perceive as important to help them improve their learning. The study found that children have a good understanding of the concepts of feedback, and can articulate clearly the elements most useful for helping them to improve their learning. A key finding of the study was that although there is consistency about the factors involved in good assessment, there is variation in the type of feedback seen as effective in demonstrating those factors. Issues relating to possible gender differences in the reception of feedback, and in what is perceived as helpful, emerged during the study. Recommendations for future research include the need to explore these, followed by the identification of implications for classroom use if a gender‐based link relating to formative assessment is found.
TL;DR: The authors investigated the role questioning occupies in an oral examination as it relates to King and Kitchener's theory of critical, reflective thinking and found that the relationship between the examiners' probing questions and the assessment of reflective thinking in pre-service teachers.
Abstract: This paper investigates the role questioning occupies in an oral examination as it relates to King and Kitchener’s theory of critical, reflective thinking. Pre‐service teachers’ experience of sitting an oral examination and professors’ reflections on conducting the assessment in a small liberal arts university are considered. Findings from this research reflect upon the learning process in an oral examination and review the relationship between the examiners’ probing questions and the assessment of reflective thinking in pre‐service teachers.
TL;DR: In this article, the authors used Item Response Theory number-correct score equating to examine the relative standards that are set between tiers on certain GCSE assessments and found evidence to suggest that candidates on the foundation tier are being over-rewarded, while those on the higher tier are underrewarding.
Abstract: Tiering is a multi‐stage test design whereby teachers allocate students to a particular difficulty level (tier) of a test. This approach to the challenge of delivering assessments to students with a heterogeneous ability distribution is normal practice in UK public examinations at the age of 16. This study uses Item Response Theory number‐correct score equating to examine the relative standards that are set between tiers on certain GCSE assessments. It finds evidence to suggest that candidates on the foundation tier are being over‐rewarded, while those on the higher tier are being under‐rewarded. It concludes that the use of IRT test equating could help improve standard setting on tiered tests and that the issue of restricted grade ranges on these tests may need to be reconsidered.
TL;DR: Theory practice gaps can be problematical for a number of reasons as mentioned in this paper, such as: theory might not have been worked through to its practical implications; impractical and therefore not implemented; it c...
Abstract: Theory–practice gaps can be problematical for a number of reasons. Theory might be too new to have been worked through to its practical implications; impractical and therefore not implemented; it c...
TL;DR: Stobart and Stobart as mentioned in this paper discuss the testing times for assessment with annual uproar in the UK about national examinings and their effect on the UK's education system.
Abstract: by G. Stobart, Abingdon, Routledge, 2008, 224 pp., £22.99 (paperback), ISBN 978‐0‐415‐40475‐4 We are undoubtedly in ‘testing times’ for assessment with annual uproar in the UK about national examin...
TL;DR: The authors investigated classification consistency by analysing five years of examination and assessment data in the subject areas of English and mathematics, and creating simulated parallel test observed scores at varying reliabilities (based on classical test theory assumptions).
Abstract: In educational systems, concern has been expressed about the accuracy of classification when marks are aligned to grades or levels. In particular, it has been claimed that a school assessment‐based grading would have much greater levels of accuracy than one based on examination scores. This paper investigates classification consistency by analysing five years of examination and assessment data in the subject areas of English and mathematics, and creating simulated parallel‐test observed scores at varying reliabilities (based on classical test theory assumptions). While grades created from moderated school assessments did show greater agreement than those from examination scores, the improvement was only modest.
TL;DR: In this paper, the authors used interviews of pupils and their teachers to identify persons whose test results "misfit" and are considered statistically "aberrant" or "unexpected" and whose measures are consequently potentially invalid, drawing the test validity into question.
Abstract: Researchers have developed indices to identify persons whose test results ‘misfit’ and are considered statistically ‘aberrant’ or ‘unexpected’ and whose measures are consequently potentially invalid, drawing the test’s validity into question. This study draws on interviews of pupils and their teachers, using a sample of 31 10‐year‐olds who were flagged as most ‘aberrant’ in a standardised mathematics test. The children’s and their teachers’ explanations were analysed and attributed: (i) to item‐, person‐ (self/other) and classroom‐levels; and ii) according to causal dimensions. Children’s and teachers’ explanations were mostly in agreement in relation to unexpected negative results and they included references to previously well‐cited sources of construct‐irrelevant variance (e.g. ineffective test‐taking strategies, careless mistakes) as well as construct‐relevant variance (e.g. misconceptions, weaknesses in particular topics). Findings of this exploratory study are discussed from a test validity and attr...
TL;DR: The Assessment in Education journal has been published in three issues annually since 1994 as discussed by the authors and the fourth issue is the fourth in 2010, following our decision last year to publish the journal for the first time.
Abstract: Since the first issue in 1994, Assessment in Education has been published in three issues annually. This regular issue is the fourth in 2010, following our decision last year to publish the journal...
TL;DR: In this article, assessment in education: Principles, Policy & Practice, Volume 17, Number 3, August 2010, pp. 251-254, DOI: 10.1080/0969594X.2010.496216.
Abstract: Published in Assessment in Education: Principles, Policy & Practice, Volume 17, Number 3, August 2010, pp. 251–254, DOI: 10.1080/0969594X.2010.496216. Robert MacCann’s name was spelt incorrectly th...
TL;DR: In this paper, van Blerkom sets himself the task of assisting teachers with practical aspects of learning, and the task is completed by assisting them with a practical aspect of learning.
Abstract: by Malcolm van Blerkom, Abingdon, Routledge, 2008, 312 pp., £52.50 (paperback), ISBN 978‐0‐8058‐6457‐1 In this text van Blerkom sets himself the task of assisting teachers with practical aspects of...