In 1974 an article appeared in Science magazine with the dry-sounding title “Judgment Under Uncertainty: Heuristics and Biases” by a pair of psychologists who were not well known outside their discipline of decision theory. In it Amos Tversky and Daniel Kahneman introduced the world to Prospect Theory, which mapped out how humans actually behave when faced with decisions about gains and losses, in contrast to how economists assumed that people behave. Prospect Theory turned Economics on its head by demonstrating through a series of ingenious experiments that people are much more concerned with losses than they are with gains, and that framing a choice from one perspective or the other will result in decisions that are exactly the opposite of each other, even if the outcomes are monetarily the same. Prospect Theory led cognitive psychology in a new direction that began to uncover other human biases in thinking that are probably not learned but are part of our brain’s wiring.

Thinking fast and slow.

Pedagosko i psiholosko testiranje i procjenjivanje spadaju među najvažnije doprinose znanosti o ponasanju nasem drustvu i pružaju temeljna i znacajna poboljsanja u odnosu na ranije postupke. Iako se ne može ustvrditi da su svi testovi dovoljno usavrseni niti da su sva testiranja razborita i korisna, postoji velika kolicina informacija koje ukazuju na ucinkovitost kvalitetnih instrumenata u onim situacijama u kojima je njihovo koristenje potkrijepljeno validacijskim podacima. Pravilna upotreba testova može dovesti do boljih odluka o pojedincima i programima nego sto bi to bio slucaj bez njihovog koristenja, a također i ukazati na put za siri i pravedniji pristup obrazovanju i zaposljavanju. Međutim, losa upotreba testova može dovesti do zamjetne stete nanesene ispitanicima i drugim sudionicima u procesu donosenja odluka na temelju testovnih podataka. Cilj Standarda je promoviranje kvalitetne i eticne upotrebe testova te uspostavljanje osnovice za ocjenu kvalitete postupaka testiranja. Svrha objavljivanja Standarda je uspostavljanje kriterija za evaluaciju testova, provedbe testiranja i posljedica upotrebe testova. Iako bi evaluacija prikladnosti testa ili njegove primjene trebala ovisiti prvenstveno o strucnim misljenjima, Standardi pružaju okvir koji osigurava obuhvacanje svih relevantnih pitanja. Bilo bi poželjno da svi autori, sponzori, nakladnici i korisnici profesionalnih testova usvoje Standarde te da poticu druge da ih također prihvate.

Standards for educational and psychological testing

미국의 “전국 수학 교사 협의회”(National Council of Teachers of Mathematics, NCTM)는 1989년부터 〈학교 수학의 교육과정과 평가 규준〉(1989), 〈수학 가르침(교수)의 전문성 규준〉(1991), 〈학교 수학의 평가(시험) 규준〉(NCTM, 1995), 〈학교 수학의 원리와 규준〉(2000)을 출판하여 미국의 수학 교육 의 전망(목표, 나아갈 길)과 규준(실행 지침)을 제시하였다. 수학 교사들로 구성된 미국의 NCTM은 학생, 학부모, 학교 행정가 등 많은 사람들과 힘을 합하여 모든 학생들에게 수준 높은 수학 교육을 받을 수 있는 여건(환경, 기회)을 조성하는 데 구심점의 역할을 하였다. 한편 많은 관련 단체들은 여러 배경과 능력을 가진 학생들이 전문성을 지닌 교사(특수 교사를 일컫는 밀이 아니다. 수학 교과를 이해하고 수학의 전문성과 특수성을 가르칠 수 있는 일반 교사를 일컫는 말이다.)로부터 미래를 대비해 평등하고, 진취적이며, 지원이 잘 이루어지고, 공학 도구(IT)가 잘 갖춰진 환경에서 중요한 수학적 아이디어를 이해하면서 학습할 수 있는 수학 교실(미국에서는 우리나라처럼 수학 교사가 수학 시간에 학생의 방(교실: Homeroom)에 찾아가지 않고 학생들이 선생의 방(수학 교실: Classroom)을 찾아온다. 전형적인 수학 교실의 사진은 2쪽에 나와 있다.)을 만들기 위해 함께 힘썼다. NCTM에서 출간한 여러 규준들은 우리나라의 제6차와 제7차 교육과정에도 큰 영향을 미쳤다. 이 글에서는 NCTM(2000)에서 제시한 학습 원리를 간단히 살펴본 다음 이를 중심으로 현재 미국 수학 교육의 교수ㆍ학습 이론의 동향을 살펴본다.

미국 NCTM의 Principles and Standards for School Mathematics에 나타난 수학과 교수,학습의 이론

(1998). Human cognitive abilities: A survey of factor analytic studies. Gifted and Talented International: Vol. 13, No. 2, pp. 97-98.

Human cognitive abilities: A survey of factor analytic studies

This study investigates teachers’ pedagogical content knowledge, professional beliefs, work-related motivation, and self-regulation as aspects of their professional competence. Specifically, it examines how these aspects impact instruction and, in turn, student outcomes. In a nationally representative sample of 194 German secondary school mathematics classes, multiple measures were used to assess teacher competence, instructional quality, and students’ achievement and motivation. The effect of teachers’ professional competence on student outcomes was estimated in a 1-year repeated-measures design. Two-level structural equation models revealed positive effects of teachers’ pedagogical content knowledge, enthusiasm for teaching, and self-regulatory skills on instructional quality, which in turn affected student outcomes. In contrast, teachers’ general academic ability did not affect their instruction. The multidimensional model of teachers’ professional competence introduced in this article seems suited to stimulate further research on the personal indicators of teacher quality.

Professional Competence of Teachers: Effects on Instructional Quality and Student Development

The goals of education and qualification in modern industrial societies can no longer be described by a fixed set of specialized skills that are transferable from one generation to the next. Nowadays, knowledge must be applicable to different, new, and complex situations and contexts. It is against this background that the concept of competence has attracted increased research attention. Competencies are conceptualized as complex ability constructs that are context-specific, trainable, and closely related to real life. The theoretical modeling of competencies, their assessment, and the usage of assessment results in practice present new challenges for psychological and educational research. This article reviews current issues in competence modeling, outlining research questions and the current state of research, and identifying the need for more interdisciplinary research. Finally, a research program recently initiated by the German Research Foundation (DFG) to address these questions and demands is prese...

Current Issues in Competence Modeling and Assessment

Der Begriff der Kompetenz ist seit vielen Jahren ein Modebegriff der Sozial- und Erziehungswissenschaften, er ist in vielfaltiger unterschiedlicher Weise genutzt und immer wieder neu gefasst worden. Der vorliegende Aufsatz versucht, die sozialwissenschaftlichen Wurzeln des Kompetenzbegriffs und den darauf aufbauenden Diskurs in der Erziehungswissenschaft nachzuzeichnen. Damit wird der Kontext skizziert, in dem die aktuelle empirische Forschung zur Modellierung und Messung von Kompetenzen angesiedelt ist. Die Bedeutung des Kompetenzbegriffs in sprachwissenschaftlichen sowie sozialisationstheoretischen Zusammenhangen muss hierbei ebenso berucksichtigt werden wie funktional-pragmatische Kompetenzkonzepte in der Psychologie. Schlieslich ist der Kompetenzbegriff in den Erziehungswissenschaften zentral zur Beschreibung der Ziele von schulischer und beruflicher Bildung. Die Komplexitat des Kompetenzbegriffs stellt die empirische Forschung vor hohe Herausforderungen. Interessierende Kompetenzkonstrukte mussen prazisiert werden und in angemessene Messmodelle und -instrumente umgesetzt werden. Eine angemessene Modellierung von Kompetenzen kann sowohl Auswertungsroutinen fur Messverfahren bereitstellen als auch die Zusammenhange zwischen individuellen Fahigkeiten und Fertigkeiten und erfolgreichem Handeln in spezifischen Kontexten beschreiben.

Kompetenzkonzepte in den Sozialwissenschaften und im erziehungswissenschaftlichen Diskurs

The concept of competence in educational contexts.

In most large-scale assessments of student achievement, several broad content domains are tested. Because more items are needed to cover the content domains than can be presented in the limited testing time to each individual student, multiple test forms or booklets are utilized to distribute the items to the students. The construction of an appropriate booklet design is a complex and challenging endeavor that has far-reaching implications for data calibration and score reporting. This module describes the construction of booklet designs as the task of allocating items to booklets under context-specific constraints. Several types of experimental designs are presented that can be used as booklet designs. The theoretical properties and construction principles for each type of design are discussed and illustrated with examples. Finally, the evaluation of booklet designs is described and future directions for researching, teaching, and reporting on booklet designs for large-scale assessments of student achievement are identified.

An NCME Instructional Module on Booklet Designs in Large‐Scale Assessments of Student Achievement: Theory and Practice

Preface Contributors Theoretical Perspectives and Developmental Models 1. The Concept of Competence in Educational Contexts, Eckhard Klieme, Johannes Hartig, and Dominique Rauch 2. Competencies for Successful Learning: Developmental Changes and Constraints, Marcus Hasselhorn 3. A Model-Based Test of Competence Profile and Competence Level in Deductive Reasoning, Christiane Spiel and Judith Gluck Psychometric Modeling 4. Psychometric Models for the Assessment of Competencies, Johannes Hartig 5. Explanatory Item Response Models: A Brief Introduction, Mark Wilson, Paul De Boeck, and Claus H. Carstensen 6. Linking Competencies in Horizontal, Vertical, and Longitudinal Settings and Measuring Growth, Alina A. von Davier, Claus H. Carstensen, and Matthias von Davier 7. Reporting Test Outcomes Using Models for Cognitive Diagnosis, Matthias von Davier, Lou DiBello, and Kentaro Yamamoto Assessment Methods and Technology 8. Measuring Competencies: Introduction to Concepts and Questions of Assessment in Education, Detlev Leutner, Johannes Hartig, and Nina Jude 9. Introduction to the Computer-Based Assessment of Competencies, Astrid Jurecka 10. Adaptive Testing and Item Banking, Theo J. H. M. Eggen 11. Computer-Based Tests: Alternatives for Test and Item Design, Joachim Wirth 12. Computer-Based Assessment in Support of Distance Learning, Gregory K. W. K. Chung, Harold F. O'Neil, William L. Bewley, and Eva L. Baker Large-Scale Assessment for the Monitoring of Educational Quality 13. Assessment in Large-Scale Studies, Tina Seidel and Manfred Prenzel 14. Introduction of Educational Standards in German-Speaking Countries, Eckhard Klieme and Katharina Maag Merki 15. Causal Effects and Fair Comparison: Considering the Influence of Context Variables on Student Competencies, Christof Nachtigall, Ulf Krohne, Ulrike Enders, and Rolf Steyer 16. Monitoring and Assurance of School Quality: Principles of Assessment and Internet-Based Feedback of Test Results, Ingmar Hosenfeld.

/pdf/assessment-of-competencies-in-educational-contexts-oj17wxl7dz.pdf

Johannes Hartig

Papers

Current Issues in Competence Modeling and Assessment

Kompetenzkonzepte in den Sozialwissenschaften und im erziehungswissenschaftlichen Diskurs

The concept of competence in educational contexts.

An NCME Instructional Module on Booklet Designs in Large‐Scale Assessments of Student Achievement: Theory and Practice

Assessment of Competencies in Educational Contexts