Journal ArticleDOI
Validating the Interpretations and Uses of Test Scores
Reads0
Chats0
TLDR
In this article, an argument-based approach to validate an interpretation or use of test scores is proposed, where the claims based on the test scores are outlined as an argument that specifies the inferences and supporting assumptions needed to get from test responses to score-based interpretations and uses.Abstract:
To validate an interpretation or use of test scores is to evaluate the plausibility of the claims based on the scores. An argument-based approach to validation suggests that the claims based on the test scores be outlined as an argument that specifies the inferences and supporting assumptions needed to get from test responses to score-based interpretations and uses. Validation then can be thought of as an evaluation of the coherence and completeness of this interpretation/use argument and of the plausibility of its inferences and assumptions. In outlining the argument-based approach to validation, this paper makes eight general points. First, it is the proposed score interpretations and uses that are validated and not the test or the test scores. Second, the validity of a proposed interpretation or use depends on how well the evidence supports the claims being made. Third, more-ambitious claims require more support than less-ambitious claims. Fourth, more-ambitious claims (e.g., construct interpretations) tend to be more useful than less-ambitious claims, but they are also harder to validate. Fifth, interpretations and uses can change over time in response to new needs and new understandings leading to changes in the evidence needed for validation. Sixth, the evaluation of score uses requires an evaluation of the consequences of the proposed uses; negative consequences can render a score use unacceptable. Seventh, the rejection of a score use does not necessarily invalidate a prior, underlying score interpretation. Eighth, the validation of the score interpretation on which a score use is based does not validate the score use.read more
Citations
More filters
Journal ArticleDOI
Stephen Jay Gould’s Analysis of the Army Beta Test in The Mismeasure of Man: Distortions and Misconceptions Regarding a Pioneering Mental Test
TL;DR: The Army Beta test for illiterate American draftees in World War I was a well-designed test by the standards of the time, and all evidence indicates that it measured intelligence a century ago and can, to some extent, do so today.
Journal ArticleDOI
Ercikan, K., & Pellegrino, J. W. Eds. (2017). Validation of Score Meaning for the Next Generation of Assessments: The Use of Response Processes. New York, NY: Routledge.
Journal ArticleDOI
Exploring Validity Evidence Associated With Questionnaire-Based Tools for Assessing the Professional Performance of Physicians: A Systematic Review.
Mirja W. van der Meulen,Alina Smirnova,Sylvia Heeneman,Mirjam G.A. oude Egbrink,Cees P. M. van der Vleuten,Kiki M. J. M. H. Lombarts +5 more
TL;DR: Based on the argument-based approach to validity, not all questionnaire-based tools seem to support their intended use, thus weakening the argument to use these tools for formative and, especially, for summative assessments of physicians' clinical and teaching performance.
Journal ArticleDOI
The fairness of internal assessment in the GCSE: the value of students' accounts
TL;DR: This article conducted a study which surveyed 1600 GCSE students and held focus groups with 128 others in Northern Ireland and Wales, in addition to data from the WISERDEducation multi-cohort study.
Journal ArticleDOI
Análise da qualidade de uma prova de matemática do Exame Nacional do Ensino Médio
TL;DR: In this article, the authors analyzed the results of the National Secondary Education exam (Enem) 2015 by means of two models from the item response theory (IRT), the logistics of three parameters with studies on discrimination, difficulty and probability of casual hits and the model of nominal response, and studies on the difficulty and behavior of each response option in every item.
References
More filters
Journal ArticleDOI
Convergent and discriminant validation by the multitrait-multimethod matrix.
TL;DR: This transmutability of the validation matrix argues for the comparisons within the heteromethod block as the most generally relevant validation data, and illustrates the potential interchangeability of trait and method components.
Book
Experimental and Quasi-Experimental Designs for Generalized Causal Inference
TL;DR: In this article, the authors present experiments and generalized Causal inference methods for single and multiple studies, using both control groups and pretest observations on the outcome of the experiment, and a critical assessment of their assumptions.
Journal ArticleDOI
Construct validity in psychological tests.
Lee J. Cronbach,Paul E. Meehl +1 more
TL;DR: The present interpretation of construct validity is not "official" and deals with some areas where the Committee would probably not be unanimous, but the present writers are solely responsible for this attempt to explain the concept and elaborate its implications.
Book
Statistical Theories of Mental Test Scores
TL;DR: In this paper, the authors present a survey of test theory models and their application in the field of mental test analysis. But the focus of the survey is on test-score theories and models, and not the practical applications and limitations of each model studied.