Showing papers on "Judgement published in 2012"

A model for programmatic assessment fit for purpose

Abstract: We propose a model for programmatic assessment in action, which simultaneously optimises assessment for learning and assessment for decision making about learner progress. This model is based on a set of assessment principles that are interpreted from empirical research. It specifies cycles of training, assessment and learner support activities that are complemented by intermediate and final moments of evaluation on aggregated assessment data points. A key principle is that individual data points are maximised for learning and feedback value, whereas high-stake decisions are based on the aggregation of many data points. Expert judgement plays an important role in the programme. Fundamental is the notion of sampling and bias reduction to deal with the inevitable subjectivity of this type of judgement. Bias reduction is further sought in procedural assessment strategies derived from criteria for qualitative research. We discuss a number of challenges and opportunities around the proposed model. One of its prime virtues is that it enables assessment to move, beyond the dominant psychometric discourse with its focus on individual instruments, towards a systems approach to assessment design underpinned by empirically grounded theory.

Climate change and moral judgement

Abstract: Converging evidence from the behavioural and brain sciences suggests that climate change fails to generate strong moral intuitions and therefore it does not stimulate an urgent need for action. However, adequate communication strategies could enhance moral intuitions about climate change and therefore motivate greater support for ameliorative actions and policies. Converging evidence from the behavioural and brain sciences suggests that the human moral judgement system is not well equipped to identify climate change — a complex, large-scale and unintentionally caused phenomenon — as an important moral imperative. As climate change fails to generate strong moral intuitions, it does not motivate an urgent need for action in the way that other moral imperatives do. We review six reasons why climate change poses significant challenges to our moral judgement system and describe six strategies that communicators might use to confront these challenges. Enhancing moral intuitions about climate change may motivate greater support for ameliorative actions and policies.

Learning from clinical work: the roles of learning cues and credibility judgements.

Abstract: CONTEXT: How learners interpret their clinical experiences to create meaningful learning has not been well studied. We explored experiences considered by doctors to be influential in their learning in order to better understand this process. METHODS: Using a grounded theory approach, we interviewed 22 academic doctors who had been in practice for

Making sense of work‐based assessment: ask the right questions, in the right way, about the right things, of the right people

Abstract: Medical Education 2012: 46: 28–37 Context Historically, assessments have often measured the measurable rather than the important. Over the last 30 years, however, we have witnessed a gradual shift of focus in medical education. We now attempt to teach and assess what matters most. In addition, the component parts of a competence must be marshalled together and integrated to deal with real workplace problems. Workplace-based assessment (WBA) is complex, and has relied on a number of recently developed methods and instruments, of which some involve checklists and others use judgements made on rating scales. Given that judgements are subjective, how can we optimise their validity and reliability? Methods This paper gleans psychometric data from a range of evaluations in order to highlight features of judgement-based assessments that are associated with better validity and reliability. It offers some issues for discussion and research around WBA. It refers to literature in a selective way. It does not purport to represent a systematic review, but it does attempt to offer some serious analyses of why some observations occur in studies of WBA and what we need to do about them. Results and Discussion Four general principles emerge: the response scale should be aligned to the reality map of the judges; judgements rather than objective observations should be sought; the assessment should focus on competencies that are central to the activity observed, and the assessors who are best-placed to judge performance should be asked to participate.

The method of Adaptive Comparative Judgement

Abstract: Adaptive Comparative Judgement (ACJ) is a modification of Thurstone’s method of comparative judgement that exploits the power of adaptivity, but in scoring rather than testing Professional judgement by teachers replaces the marking of tests; a judge is asked to compare the work of two students and simply to decide which of them is the better From many such comparisons a measurement scale is created showing the relative quality of students’ work; this can then be referenced in familiar ways to generate test results The judges are asked only to make a valid decision about quality, yet ACJ achieves extremely high levels of reliability, often considerably higher than practicable operational marking can achieve It therefore offers a radical alternative to the pursuit of reliability through detailed marking schemes ACJ is clearly appropriate for performances like writing or art, and for complex portfolios or reports, but may be useful in other contexts too ACJ offers a new way to involve all teachers in s

Values-led Participatory Design

Abstract: The widespread use of participatory design (PD) has meant that different approaches and conceptualisations exist in this field today. In this article, it is argued that one fruitful approach is to rekindle a concern for values in design, focusing upon values as the engine that drives activities in PD. Drawing from the authors‘ own PD projects, this article shows how this can be accomplished: through designers enacting their appreciative judgement of values by engaging in a dynamic and dialogical process of cultivating the emergence of values, developing them and supporting their grounding.

The future of teacher education: Evidence, competence or wisdom?

Abstract: In recent years policymakers and politicians in many countries have become increasingly interested in teacher education. In most cases, however, the interest in teacher education is not informed by a desire to enhance the professionalism of teachers but rather is part of ongoing attempts to control the educational 'enterprise.' In this chapter I analyse these developments, particularly with regard to a focus on the alleged need for 'evidence' to form the basis for teaching or the idea that teaching can be adequalty captured in terms of competences. Against these tendencies I argue for the important role teacher judgement plays in education, make clear why such judgement is needed, and what this would require for teacher education.

Service co-creation and value realisation

Abstract: The notion of value co-creation is central to the discourse of Service-Dominant Logic (S-D logic) yet there remains little agreement among academics seeking to explain or research the value co-creation process. We distinguish service co-creation from the S-D logic notion of value co-creation, and conceptualise service co-creation as a process comprising value potential, resource integration, and resource modification. Value, being a personal evaluative judgement, cannot be co-created; rather it is realised by actors as an outcome of service co-creation. We define service co-creation as planned resource integration behaviours by actors intended to realise a value proposition. We provide guidance to assist practitioners seeking to enhance the value their customers might realise.

Practitioner Reflection and Judgement as Phronesis

Abstract: Dominant conceptions of professional knowledge appear to have largely forgotten Aristotle’s conception of phronesis and its place in considerations of what it means to know in professional life. Aristotle draws attention to phronesis as a form of reflective practical wisdom that complements techne, technically oriented approaches, and episteme, scientifically oriented approaches, in considerations of what it might mean to develop and enact professional knowledge.

What Does Kant Mean by ‘Power of Judgement’ in his Critique of the Power of Judgement?

Abstract: The notion of ‘power of judgement’ in the title of Kant's Critique of the Power of Judgement is commonly taken to refer to a cognitive power inclusive of both determining judgement and reflecting judgement. I argue, first, that this seemingly innocuous view is in conflict both with the textual fact that Kant attempts a Critical justification of the reflecting power of judgement – only – and with the systematic impossibility of a transcendentally grounded determining power of judgement. The conventional response to these difficulties is to point out that, Kant's systematic ambitions in the third Critique notwithstanding, reflection, qua concept-forming synthesis, is too closely tied to determination to be a cognitive power in its own right. I argue, second, that this response is question-begging, since the notion of reflection it employs is not only not one central to the third Critique but one antecedently tied to the understanding. I argue, third, that Kant's discussion, in the pivotal § § 76–7, of our cognitive relation to sensible particularity addresses an epistemic problem present (but not raised) in the Critique of Pure Reason. This is the problem of the synthesizability, qua absolute unity, of unsynthesized intuitions. Solving this problem requires Critical justification of a principle of reflection. It follows that Kant's systematic ambitions in the third Critique are appropriate. Given the problem Kant seeks to address, he must offer what he takes himself to be offering: a Critique of the (Reflecting) Power of Judgement.

Comparative judgement for assessment

Abstract: Historically speaking, students were judged long before they were marked. The tradition of marking, or scoring, pieces of work students offer for assessment is little more than two centuries old, and was introduced mainly to cope with specific problems arising from the growth in the numbers graduating from universities as the industrial revolution progressed. This paper describes the principles behind the method of Comparative Judgement, and in particular Adaptive Comparative Judgement, a technique borrowed from psychophysics which is able to generate extremely reliable results for educational assessment, and which is based on the kind of holistic evaluation that we assume was the basis for judgement in pre-marking days, and that the users of assessment results expect our assessment schemes to capture.

Enterprise education: For all, or just some?

Abstract: Purpose The purpose of this paper is to provocatively enter four imagined worlds of enterprise education with the express aim of contemplating an emerging future. The authors do so not to expressly determine what positioning is most appropriate for enterprise/entrepreneurship education, but rather to consider the issues associated with each of the four imagined worlds. Design/methodology/approach The authors’ approach is built around a combination of cycles of reflective practice and the use of scenario development processes. The authors seek to suspend their collective judgement whilst entering the four imagined worlds, but ultimately do not claim to have hidden their personal biases. Findings It is concluded that enterprise/entrepreneurship education should be shared across the university and not owned by any school or faculty. While the authors find it difficult to dismiss the underlying purpose of each scenario, they sense an opportunity to unite their common focus on the development of a transformative student experience. Practical implications This process has provided unexpected insights into the potential of scenario planning as a tool that could conceivably be employed more often to tackle complex issues, such as the positioning of enterprise/entrepreneurship education in Higher Education. Originality/value This paper, despite its inherent biases, offers the reader an opportunity to gain a sense of the various roles forced upon enterprise/entrepreneurship education by its various key stakeholders. In doing so, the shortcomings of the current situation are highlighted.

Standards and quality in English primary schools over time: the national evidence

Abstract: As the title indicates, this chapter is concerned with standards in primary education and it is as well at the outset to be clear about what we mean by that. The word can take on more than one interpretation. On the one hand, standards can be thought of as a level against which one tries to make a judgement. In high jumping terms that would be the height of the bar and the observations of individuals trying to jump over it. In educational terms it would involve ascertaining how many people met or exceeded a certain criterion; but another way of looking at standards is to take them as the level that one has reached. How high can an individual jump? It is that way of thinking about standards which is used in this chapter. Educational standards and judgements about the quality of those standards can beapproached by starting with a description of children. Such an approach might begin with what pupils know and can do when they start primary education, and then be followed with a description of how this progresses as they age. It might also include how their perceptions and attitudes change, as well as other variables. The educationalist can then be left to come to his or her own judgements about quality. Some may argue that the process of schooling is a vital part of quality and that standards and quality should be judged against processes. They would be quite right in pointing to processes as being at the heart of education but in the last analysis, in order to assess that quality, one must look at impact and therefore at change in children. And, of course, anyone’s judgement about quality must involve making decisions, explicitly or implicitly, about what standards are expected or appropriate in order to decide whether what is seen is good or poor. In fact, whenever one makes judgements about quality or standards one makes them against a reference point. There will always be a comparison. This might be a comparison of change over time, or against other countries, other schools or against personal expectations, but the judgements are always comparative. What might a description of what children know and can do when they start schoollook like? Figure 17.1 is taken from a study of Scottish children starting school (Merrell and Tymms 2007) and provides a graphical illustration of what we have in mind. It shows both the distribution of pupils and their abilities on the same scale. The enormous variation is apparent and differences between sexes, in social background, and other groupings readily follow. In the longer term it also becomes possible to take such a description and show how what children know and can do changes as they progress through their primary years. The parallel chart for England is almost identical once the halfyear difference in the age of starting school is taken into account. However, thestatutory Foundation Stage Profile, which monitors the progress of children as they move through the Foundation Stage in England, could not be used to generate such detailed objectively based information.

Scylla or Charybdis? Can we navigate between objectification and judgement in assessment?

Abstract: Kevin Eva: I’ve stolen the title for this first ‘Dialogue’ from a paper you wrote a few years ago in which you used the journey between Scylla and Charybdis, the dangers Odysseus had to negotiate on his voyage, as a metaphor for the difficulty of navigating between ‘excessive examination and naı̈ve reliance on self-assessment’. The metaphor seems equally fitting for describing the difficulties we face in navigating between objectification and judgement in assessment. Do we choose to crash up against the rocky shoal of checklists and the atomisation of medicine they promote or to be sucked down into the whirlpool that is subjectivity and the concerns about fairness and defensibility that go with it?

Synthesizing study results in a systematic review.

Abstract: A single study rarely suffices to underpin treatment or policy decisions. This creates a strong imperative for systematic reviews. Authors of reviews need a method to synthesize the results of several studies, regardless of whether or which statistical method is used. In this article, we provide arguments for combining studies in a review. To combine studies authors should judge the similarity of studies. This judgement should be based on the working mechanism of the intervention or exposure. It should also be assessed if this mechanism is similar for various populations and follow-up times. The same judgement applies to the control interventions. Similar studies can be combined in either a meta-analysis or narrative synthesis. Other methods such as vote counting, levels of evidence synthesis, or best evidence synthesis are better avoided because they may produce biased results. We support our arguments by re-analysing a systematic review. In its original form, the review showed strong evidence of no effect, but our re-analysis concluded there is evidence of an effect. We provide a flow-chart to guide authors through the synthesis and assessment process.

Models for professional judgement in social work

Abstract: In order to further research on decision-making in social work, a richer theoretical underpinning of models of decision-making is required. There has been significant theoretical advance in other disciplines from which we might learn and develop supportive theoretical constructs. This article outlines some major theoretical approaches to modelling individual judgement including expected utility, fuzzy set theory, signal detection, heuristics and biases, judgement analysis and bounded rationality models. Models are drawn from diverse fields such as computing, economics, psychology and operations research and are illustrated with social work examples. The models are sequenced from those that are more prescriptive (based on mathematical models of how a rational person should act) through to those that are more descriptive (creating models from studies of how people make decisions in real life). Implications are drawn out for future research on judgement and decision-making in social work. It is time for research on social work decision-making to be taken to a new level by creating and adapting models to inform empirical studies. This will require interdisciplinary collaboration and clarity in selecting, adapting and creating models appropriate to the complex environment of social work decision-making.

Dealing with customer misbehaviour : Employees’ tactics, practical judgement and implicit knowledge

Abstract: Much current research fails to provide in-depth explanations as to how and with what resources frontline employees deal with incidents where customers display dysfunctional behaviour. By drawing on theory of implicit knowledge and practical judgement this paper aims to explain this and conceptualize inherent structures and sub-mechanisms, central to service marketing. The analysis is based on in-depth interviews and narratives from four different industries, each representing service provision wherein customer misbehaviour is found to be frequent. The results display linkages between the central dimensions of dealing with customer misbehaviour. When incidents of misbehaviour occur they are met by tactics ranging from routinized action to more analytical and strategic approaches. These tactics are guided by underlying mechanisms in the form of practical judgements based on rules, balanced adjustment or reflection, with the judgements in turn being informed by implicit knowledge based on norms, schemes, or multi-perspective thinking. The study reveals patterns of linkages between these.

A matter of judgement: The effect of information and deliberation on public attitudes to punishment:

Abstract: The idea of reducing public punitiveness through providing information and encouraging deliberation has attracted considerable interest. However, there remains no solid evidence of durable changes in attitude. The study presented here provides a test of the hypothesis that information combined with deliberation can affect general measures of punitiveness, confidence in the courts and acceptance of alternatives to imprisonment (the three dependent variables). The study involved a pre-test, post-test experimental design. Participants were randomly allocated to either an intervention group or a control condition. Statistically significant changes in the dependent variables were observed immediately following the intervention but these changes were not sustained when measured at follow-up nine months later. Further, at the time of the follow-up the differences between the control group scores and the intervention group scores were not significantly different. The observed changes immediately following the intervention are seen to be a function of the changed relationship of the respondent to the task. The implications of the results for integrating public perspectives into policy are discussed. It is argued that rather than a focus on public education, a more productive direction is to focus on the way the public is engaged on matters concerning punishment.

The judgement process in evidence-based medicine and health technology assessment

Abstract: This article describes the judgements used to interpret evidence in evidence-based medicine (EBM) and health technology assessment (HTA). It outlines the methods and processes of EBM and HTA. Respectively, EBM and HTA are approaches to medical clinical decision making and efficient allocation of scarce health resources. At the heart of both is a concern to review and synthesise evidence, especially evidence derived from randomised controlled trials (RCTs) of clinical effectiveness. The driver of the approach of both is a desire to eliminate, or at least reduce, bias. The hierarchy of evidence, which is used as an indicator of the likelihood of bias, features heavily in the process and methods of EBM and HTA. The epistemological underpinnings of EBM and HTA are explored with particular reference to the distinction between rationalism and empiricism, developed by the philosopher David Hume and elaborated by Immanuel Kant in the Critique of Pure Reason. The importance of Humian and Kantian principles for understanding the projects of EBM and HTA is considered and the ways in which decisions are made in both, within a judgemental framework originally outlined by Kant, are explored.

Values in science

Abstract: In this essay, which was his presidential address to the Philosophy of Science Association, Ernan McMullin argued that the watershed between “classic” philosophy of science (by this meaning, not just logical positivism but the logicist tradition in theory of science stretching back through Kant and Descartes to Aristotle) and the “new” philosophy of science can best be understood by analyzing the change in our perception of the role played by values in science. He begins with some general remarks about the nature of value, goes on to explore some of the historical sources for the claim that judgement in science is value-laden, and concludes by reflecting on the implications of this claim for traditional views of the objectivity of scientific knowledge-claims.

A Longitudinal Study of Accounting Students' Ethical Judgement Making Ability

Abstract: This paper reports the effect of a moral education programme on the ethical judgement making ability of university students. The programme comprises two forms of intervention: a dedicated ethics course and subsequent practical training. A total of 113 accounting students from six Malaysian universities participated in a longitudinal study including three points of data collection, prior to an ethics course, after an ethics course, and following six-months' practical training. James Rest's short version of the Defining Issues Test (DIT) instrument (Rest, 1986, Moral development: Advances in research and theory, Praeger, NY) was employed and P-scores calculated at each data collection point. General Linear Model Repeated Measure analysis was employed to examine the within-subjects effect of the accounting programme on ethical judgement making ability. The findings highlight that the accounting programme is able to elevate levels of ethical judgement making ability and that practical training contributed sig...

A formal representation of the WHO and UNICEF estimates of national immunization coverage: a computational logic approach.

Abstract: Production of official statistics frequently requires expert judgement to evaluate and reconcile data of unknown and varying quality from multiple and potentially conflicting sources. Moreover, exceptional events may be difficult to incorporate in modelled estimates. Computational logic provides a methodology and tools for incorporating analyst's judgement, integrating multiple data sources and modelling methods, ensuring transparency and replicability, and making documentation computationally accessible. Representations using computational logic can be implemented in a variety of computer-based languages for automated production. Computational logic complements standard mathematical and statistical techniques and extends the flexibility of mathematical and statistical modelling. A basic overview of computational logic is presented and its application to official statistics is illustrated with the WHO & UNICEF estimates of national immunization coverage.

An integrated professionalism in further education: a time for phronesis?

Abstract: The aim of this article is to examine the role taken by the Institute for Learning (IfL) in England to promote the nature of professionalism in the lifelong learning sector. It raises the possibility that the decisions taken by the IfL, since its inception in 2002, are leading to the de-professionalisation of teachers. It is argued that what is now needed is a new professionalism that is driven by the practice of phronesis: wise practical reasoning, based on judgement and wisdom, and that accords with the centrality of context and the reflective nature of the activity of teaching. It will be informed by values that enable practitioners to mediate the confrontational forces of managerialism which might otherwise threaten to undermine their professionalism.

Conflicting obligations, moral dilemmas and the development of judgement through business ethics education

Abstract: Learning to address moral dilemmas is important for participants on courses in business ethics and corporate social responsibility (CSR). While modern, rule-based ethical theory often provides the normative input here, this has faced criticism in its application. In response, post-modern and Aristotelian perspectives have found favour. This paper follows a similar line, presenting an approach based initially on a critical interpretation of Ross's theory of prima facie duties, which emphasises moral judgement in actual situations. However, the retention of a modern element is suggested, with an invitation to hypothetically universalise detailed proposals for action, following Hare. This further step may reassure students/managers who seek additional closure in their decision-making. Some pedagogical implications are also briefly raised regarding the use of case material and the need to recognise questions of social interaction and personal character as features of dilemma resolution in actual organisational contexts.

Planning and professionalism: Knowledge, judgement and expertise in English planning:

Abstract: Planning has struggled with its identity as a profession. This has made planning systems vulnerable in the face of attacks on their utility. Sue Hendler was a passionate advocate of planning and its claims to professional status. Building on her writing, I argue that planning’s claims to professional status and expertise have validity. Judged against other professions, planners’ claims are of equal status, grounded as they are in the generation and melding of knowledge and knowledge claims and in the exercising of non-routinised judgement in issues pertaining to place and space. Planning practice and education needs to recognise this reality more explicitly.

Moderation and consistency of teacher judgement: teachers’ views

Abstract: Major curriculum and assessment reforms in Australia have generated research interest in issues related to standards, teacher judgement and moderation. This article is based on one related inquiry of a large-scale Australian Research Council Linkage project conducted in Queensland. This qualitative study analysed interview data to identify teachers’ views on standards and moderation as a means to achieving consistency of teacher judgement. A complementary aspect of the research involved a blind review that was conducted to determine the degree of teacher consistency without the experience of moderation. Empirical evidence was gained that most teachers, of the total interviewed, articulated a positive attitude towards the use of standards in moderation and perceived that this process produces consistency in teachers’ judgements. Context was identified as an important influential factor in teachers’ judgements and it was concluded that teachers’ assessment beliefs, attitudes and practices impact on their perceptions of the value of moderation practice and the extent to which consistency can be achieved.

Hawk Eye: A Logical Innovative Technology Use in Sports for Effective Decision Making

Abstract: The present study aims at explaining hawk eye technology in sports with regard to their latest trends in the sporting arena. This is one of the most commonly used technologies in the various sports. It has been put to a variety of uses, such as providing a way to collect interesting statistics, construct very suggestive visual representations of the game play and even helping viewers to better understand the umpiring decisions. The Hawk-eye is one such technology which is considered to be really top indentation in various sports. The necessary idea is to monitor the trajectory of the ball during the entire duration of play. We will see how the Hawk-eye technology successfully treats each of these issues and provides a robust system to be used in practice. The Hawk-eye system was developed as a replay system, originally for TV Broadcast coverage. We have thus seen that the Hawk-eye is a great innovation, which puts technology to good use in the field of sports. The international scenario of hawk eye has been discussed while analyzing the data of hawk eye software for reliable decision. The technology is used widely these days, in sports such as Tennis and Cricket. The accuracy which can be achieved with the use of the system is making the authorities think seriously about reducing the human error component involved in important decisions. Hawk eye (artificial judgement) is very excellent and superb decision making power as compared to human being (natural judgement). Hawk-Eye software team is already working on the implementing system for basketball, football, badminton and snooker.

Moderation and consistency of teacher judgement : teachers’ views

Abstract: Major curriculum and assessment reforms in Australia have generated research interest in issues related to standards, teacher judgement and moderation. This article is based on one related inquiry of a large-scale Australian Research Council Linkage project conducted in Queensland. This qualitative study analysed interview data to identify teachers’ views on standards and moderation as a means to achieving consistency of teacher judgement. A complementary aspect of the research involved a blind review that was conducted to determine the degree of teacher consistency without the experience of moderation. Empirical evidence was gained that most teachers, of the total interviewed, articulated a positive attitude towards the use of standards in moderation and perceived that this process produces consistency in teachers’ judgements. Context was identified as an important influential factor in teachers’ judgements and it was concluded that teachers’ assessment beliefs, attitudes and practices impact on their perceptions of the value of moderation practice and the extent to which consistency can be achieved.