Pedagosko i psiholosko testiranje i procjenjivanje spadaju među najvažnije doprinose znanosti o ponasanju nasem drustvu i pružaju temeljna i znacajna poboljsanja u odnosu na ranije postupke. Iako se ne može ustvrditi da su svi testovi dovoljno usavrseni niti da su sva testiranja razborita i korisna, postoji velika kolicina informacija koje ukazuju na ucinkovitost kvalitetnih instrumenata u onim situacijama u kojima je njihovo koristenje potkrijepljeno validacijskim podacima. Pravilna upotreba testova može dovesti do boljih odluka o pojedincima i programima nego sto bi to bio slucaj bez njihovog koristenja, a također i ukazati na put za siri i pravedniji pristup obrazovanju i zaposljavanju. Međutim, losa upotreba testova može dovesti do zamjetne stete nanesene ispitanicima i drugim sudionicima u procesu donosenja odluka na temelju testovnih podataka. Cilj Standarda je promoviranje kvalitetne i eticne upotrebe testova te uspostavljanje osnovice za ocjenu kvalitete postupaka testiranja. Svrha objavljivanja Standarda je uspostavljanje kriterija za evaluaciju testova, provedbe testiranja i posljedica upotrebe testova. Iako bi evaluacija prikladnosti testa ili njegove primjene trebala ovisiti prvenstveno o strucnim misljenjima, Standardi pružaju okvir koji osigurava obuhvacanje svih relevantnih pitanja. Bilo bi poželjno da svi autori, sponzori, nakladnici i korisnici profesionalnih testova usvoje Standarde te da poticu druge da ih također prihvate.

Standards for educational and psychological testing

The traditional conception of validity divides it into three separate and substitutable types—namely, content, criterion, and construct validities. This view is fragmented and incomplete, especially because it fails to take into account both evidence of the value implications of score meaning as a basis for action and the social consequences of score use. The new unified concept of validity interrelates these issues as fundamental aspects of a more comprehensive theory of construct validity that addresses both score meaning and social values in test interpretation and test use. That is, unified validity integrates considerations of content, criteria, and consequences into a construct framework for the empirical testing of rational hypotheses about score meaning and theoretically relevant relationships, including those of an applied and a scientific nature. Six distinguishable aspects of construct validity are highlighted as a means of addressing central issues implicit in the notion of validity as a unified concept. These are content, substantive, structural, generalizability, external, and consequential aspects of construct validity. In effect, these six aspects function as general validity criteria or standards for all educational and psychological measurement, including performance assessments, which are discussed in some detail because of their increasing emphasis in educational and employment settings.

/pdf/validity-of-psychological-assessment-validation-of-2zgxanmue5.pdf

Validity of Psychological Assessment: Validation of Inferences from Persons' Responses and Performances as Scientific Inquiry into Score Meaning.

Self-report measures of adult attachment are typically scored in ways (e.g., averaging or summing items) that can lead to erroneous inferences about important theoretical issues, such as the degree of continuity in attachment security and the differential stability of insecure attachment patterns. To determine whether existing attachment scales suffer from scaling problems, the authors conducted an item response theory (IRT) analysis of 4 commonly used self-report inventories: Experiences in Close Relationships scales (K. A. Brennan, C. L. Clark, & P. R. Shaver, 1998), Adult Attachment Scales (N. L. Collins & S. J. Read, 1990), Relationship Styles Questionnaire (D. W. Griffin & K. Bartholomew, 1994) and J. Simpson's (1990) attachment scales. Data from 1,085 individuals were analyzed using F. Samejima's (1969) graded response model. The authors' findings indicate that commonly used attachment scales can be improved in a number of important ways. Accordingly, the authors show how IRT techniques can be used to develop new attachment scales with desirable psychometric properties.

An item response theory analysis of self-report measures of adult attachment.

Background Although insomnia is a prevalent complaint with significant morbidity, it often remains unrecognized and untreated. Brief and valid instruments are needed both for screening and outcome assessment. This study examined psychometric indices of the Insomnia Severity Index (ISI) to detect cases of insomnia in a population-based sample and to evaluate treatment response in a clinical sample. Methods Participants were 959 individuals selected from the community for an epidemiological study of insomnia (Community sample) and 183 individuals evaluated for insomnia treatment and 62 controls without insomnia (Clinical sample). They completed the ISI and several measures of sleep quality, fatigue, psychological symptoms, and quality of life; those in the Clinical sample also completed sleep diaries, polysomnography, and interviews to validate their insomnia/good sleep status and assess treatment response. In addition to standard psychometric indices of reliability and validity, item response theory analyses were computed to examine ISI item response patterns. Receiver operating curves were used to derive optimal cutoff scores for case identification and to quantify the minimally important changes in relation to global improvement ratings obtained by an independent assessor. Results ISI internal consistency was excellent for both samples (Cronbach α of 0.90 and 0.91). Item response analyses revealed adequate discriminatory capacity for 5 of the 7 items. Convergent validity was supported by significant correlations between total ISI score and measures of fatigue, quality of life, anxiety, and depression. A cutoff score of 10 was optimal (86.1% sensitivity and 87.7% specificity) for detecting insomnia cases in the community sample. In the clinical sample, a change score of -8.4 points (95% CI: -7.1, -9.4) was associated with moderate improvement as rated by an independent assessor after treatment. Conclusion These findings provide further evidence that the ISI is a reliable and valid instrument to detect cases of insomnia in the population and is sensitive to treatment response in clinical patients.

/pdf/the-insomnia-severity-index-psychometric-indicators-to-18yvovderi.pdf

The Insomnia Severity Index: psychometric indicators to detect insomnia cases and evaluate treatment response.

(1998). Human cognitive abilities: A survey of factor analytic studies. Gifted and Talented International: Vol. 13, No. 2, pp. 97-98.

Human cognitive abilities: A survey of factor analytic studies

The Trait Level Measurement Scale: Meaning, Interpretations, and Measurement-Scale Properties

Binary IRT Models

Item response theory (IRT) has had a major impact on instrument design and analysis, both in academic and clinical settings. This entry provides an introduction to the common dichotomous and polytomous IRT models and motivates the use of IRT in the test design process by comparing it with the previously dominant classical test theory. This entry provides a brief introduction to IRT to provide a foundation for the reader to develop a functional instrument within the IRT framework.


Keywords:

individual differences;
measurement in psychology;
psychometric testing;
testing in psychology;
test construction

Item Response Theory, Approach to Test Construction

We propose a generalization of the linear logistic test model that reframes item discriminations and difficul-ties as functions of relevant item features and accounts for variance in strategy use across latent clusters. The result is a mixture 2PL model with linear constraints (Mix2PLC). We demonstrate how the Mix2PLC may be applied in dynamic testing contexts to account for within cluster variance in strategy use.

A Mixture Explanatory IRT Model for Dynamic Processes

The Woodcock Johnson III (WJ III) Tests of Cognitive Abilities is widely used to make decisions regarding educational programing for school children. The Concept Formation subtest requires the examinee to employ deductive reasoning and rule-based categorization skills to identify common geometric features across a diverse set of shapes. Items differ in perceptual complexity, rule complexity, and the availability of performance feedback. Feedback is provided contemporaneously with testing, producing a dynamic test structure. However, the original WJ III scoring paradigm does not consider this dynamic process in modeling examinee ability and response probability. Neglecting this dynamic structure may: a) bias parameter estimates and b) produce scores that fail to utilize important information regarding individual differences in responsiveness to feedback. To evaluate the effect of item characteristics on performance, a linear logistic trait model (LLTM; Fischer, 1973) of item difficulty was estimated using the responses from the 8,372 individuals from the original norming study. Number of shapes in the item stem and rule complexity were modeled as predictors in the LLTM. Secondly, a multidimensional Rasch model for learning and change (MRMLC; Embretson, 1991) was estimated to capture the dynamic component of the subtest. All items loaded onto the first dimension, while items loaded onto a secondary (modifiability) dimension only if feedback was provided on the immediately preceding item. Thus, this second dimension represented the efficiency with which an examinee updated their problem solving strategies. The LLTM analysis produced a large comparative fit index (.835), indicating that response process validity for

Susan E. Embretson

Papers

The Trait Level Measurement Scale: Meaning, Interpretations, and Measurement-Scale Properties

Binary IRT Models

Item Response Theory, Approach to Test Construction

A Mixture Explanatory IRT Model for Dynamic Processes

The Impact of Item Features and Performance Feedback on the WJ III Concept Formation Subtest.