Pedagosko i psiholosko testiranje i procjenjivanje spadaju među najvažnije doprinose znanosti o ponasanju nasem drustvu i pružaju temeljna i znacajna poboljsanja u odnosu na ranije postupke. Iako se ne može ustvrditi da su svi testovi dovoljno usavrseni niti da su sva testiranja razborita i korisna, postoji velika kolicina informacija koje ukazuju na ucinkovitost kvalitetnih instrumenata u onim situacijama u kojima je njihovo koristenje potkrijepljeno validacijskim podacima. Pravilna upotreba testova može dovesti do boljih odluka o pojedincima i programima nego sto bi to bio slucaj bez njihovog koristenja, a također i ukazati na put za siri i pravedniji pristup obrazovanju i zaposljavanju. Međutim, losa upotreba testova može dovesti do zamjetne stete nanesene ispitanicima i drugim sudionicima u procesu donosenja odluka na temelju testovnih podataka. Cilj Standarda je promoviranje kvalitetne i eticne upotrebe testova te uspostavljanje osnovice za ocjenu kvalitete postupaka testiranja. Svrha objavljivanja Standarda je uspostavljanje kriterija za evaluaciju testova, provedbe testiranja i posljedica upotrebe testova. Iako bi evaluacija prikladnosti testa ili njegove primjene trebala ovisiti prvenstveno o strucnim misljenjima, Standardi pružaju okvir koji osigurava obuhvacanje svih relevantnih pitanja. Bilo bi poželjno da svi autori, sponzori, nakladnici i korisnici profesionalnih testova usvoje Standarde te da poticu druge da ih također prihvate.

Standards for educational and psychological testing

The traditional conception of validity divides it into three separate and substitutable types—namely, content, criterion, and construct validities. This view is fragmented and incomplete, especially because it fails to take into account both evidence of the value implications of score meaning as a basis for action and the social consequences of score use. The new unified concept of validity interrelates these issues as fundamental aspects of a more comprehensive theory of construct validity that addresses both score meaning and social values in test interpretation and test use. That is, unified validity integrates considerations of content, criteria, and consequences into a construct framework for the empirical testing of rational hypotheses about score meaning and theoretically relevant relationships, including those of an applied and a scientific nature. Six distinguishable aspects of construct validity are highlighted as a means of addressing central issues implicit in the notion of validity as a unified concept. These are content, substantive, structural, generalizability, external, and consequential aspects of construct validity. In effect, these six aspects function as general validity criteria or standards for all educational and psychological measurement, including performance assessments, which are discussed in some detail because of their increasing emphasis in educational and employment settings.

/pdf/validity-of-psychological-assessment-validation-of-2zgxanmue5.pdf

Validity of Psychological Assessment: Validation of Inferences from Persons' Responses and Performances as Scientific Inquiry into Score Meaning.

Self-report measures of adult attachment are typically scored in ways (e.g., averaging or summing items) that can lead to erroneous inferences about important theoretical issues, such as the degree of continuity in attachment security and the differential stability of insecure attachment patterns. To determine whether existing attachment scales suffer from scaling problems, the authors conducted an item response theory (IRT) analysis of 4 commonly used self-report inventories: Experiences in Close Relationships scales (K. A. Brennan, C. L. Clark, & P. R. Shaver, 1998), Adult Attachment Scales (N. L. Collins & S. J. Read, 1990), Relationship Styles Questionnaire (D. W. Griffin & K. Bartholomew, 1994) and J. Simpson's (1990) attachment scales. Data from 1,085 individuals were analyzed using F. Samejima's (1969) graded response model. The authors' findings indicate that commonly used attachment scales can be improved in a number of important ways. Accordingly, the authors show how IRT techniques can be used to develop new attachment scales with desirable psychometric properties.

An item response theory analysis of self-report measures of adult attachment.

Background Although insomnia is a prevalent complaint with significant morbidity, it often remains unrecognized and untreated. Brief and valid instruments are needed both for screening and outcome assessment. This study examined psychometric indices of the Insomnia Severity Index (ISI) to detect cases of insomnia in a population-based sample and to evaluate treatment response in a clinical sample. Methods Participants were 959 individuals selected from the community for an epidemiological study of insomnia (Community sample) and 183 individuals evaluated for insomnia treatment and 62 controls without insomnia (Clinical sample). They completed the ISI and several measures of sleep quality, fatigue, psychological symptoms, and quality of life; those in the Clinical sample also completed sleep diaries, polysomnography, and interviews to validate their insomnia/good sleep status and assess treatment response. In addition to standard psychometric indices of reliability and validity, item response theory analyses were computed to examine ISI item response patterns. Receiver operating curves were used to derive optimal cutoff scores for case identification and to quantify the minimally important changes in relation to global improvement ratings obtained by an independent assessor. Results ISI internal consistency was excellent for both samples (Cronbach α of 0.90 and 0.91). Item response analyses revealed adequate discriminatory capacity for 5 of the 7 items. Convergent validity was supported by significant correlations between total ISI score and measures of fatigue, quality of life, anxiety, and depression. A cutoff score of 10 was optimal (86.1% sensitivity and 87.7% specificity) for detecting insomnia cases in the community sample. In the clinical sample, a change score of -8.4 points (95% CI: -7.1, -9.4) was associated with moderate improvement as rated by an independent assessor after treatment. Conclusion These findings provide further evidence that the ISI is a reliable and valid instrument to detect cases of insomnia in the population and is sensitive to treatment response in clinical patients.

/pdf/the-insomnia-severity-index-psychometric-indicators-to-18yvovderi.pdf

The Insomnia Severity Index: psychometric indicators to detect insomnia cases and evaluate treatment response.

(1998). Human cognitive abilities: A survey of factor analytic studies. Gifted and Talented International: Vol. 13, No. 2, pp. 97-98.

Human cognitive abilities: A survey of factor analytic studies

Personality Measurement Issues Viewed Through the Eyes of IRT

Since the Handbook of Human Intelligence appeared in 1982, the “psychometric approach” has changed dramatically. Traditionally, the psychometric approach was synonymous with the factor-analytic approach. Exploratory factor analysis was applied to discover the number and nature of the factors that underlie performance on cognitive tasks. Carroll's (1993) three-stratum model of intellect synthesizes the factors supported across hundreds of studies. Although the studies reported somewhat inconsistent factor patterns, Carroll found consistent support for several factors by reanalyzing their data with common methods of factor analysis. However, the contemporary psychometric approach differs in three major ways from the traditional psychometric approach: (1) confirmatory approaches predominate over exploratory approaches, (2) structural analysis of items predominates over structural analysis of variables, and (3) item response theory (IRT) models predominate over factor analytic models. Thus, in the contemporary psychometric approach, confirmatory IRT models are applied to understand and measure individual differences. The intelligence construct is elaborated in confirmatory IRT models by comparing alternative models as explaining item responses. Some confirmatory IRT models include parameters to estimate the cognitive processing demands in items. These models permit items to be selected and banked by their cognitive demand features and provide results relevant to understanding what is measured by the items. Other confirmatory IRT models include parameters for person differences on the underlying processes, strategies, or knowledge structures. These models can define new types of individual differences. As will be elaborated below, parameters are included to measure qualitative differences in item responses such as relative success in various underlying cognitive operations, use of different strategies or knowledge structures, and modifiability of ability with intervention.

Psychometric Approaches to Understanding and Measuring Intelligence

Cognitive complexity level is important for measuring both aptitude and achievement in large-scale testing. Tests for standards-based assessment of mathematics, for example, often include cognitive complexity level in the test blueprint. However, little research exists on how mathematics items can be designed to vary in cognitive complexity level. In fact, determining the cognitive complexity level of items is usually based on correspondence to definitions rather than on empirically and theoretically justifiable variables that can predict item difficulty. In the current study, mathematical problem-solving items were designed for varying cognitive complexity levels based on a cognitive model of item processing. Structural variants of item models were designed to vary on two aspects of the cognitive model, the equation source and the number of subgoals. Participants were randomly assigned to test forms that contained different structural variants of the item models. Results from the linear logistic test mod...

Designing Cognitive Complexity in Mathematical Problem-Solving Items:

This paper presents a noncompensatory latent trait model, the multicomponent latent trait model for diagnosis (MLTM-D), for cognitive diagnosis. In MLTM-D, a hierarchical relationship between components and attributes is specified to be applicable to permit diagnosis at two levels. MLTM-D is a generalization of the multicomponent latent trait model (MLTM; Whitely in Psychometrika, 45:479-494, 1980; Embretson in Psychometrika, 49:175-186, 1984) to be applicable to measures of broad traits, such as achievement tests, in which component structure varies between items. Conditions for model identification are described and marginal maximum likelihood estimators are presented, along with simulation data to demonstrate parameter recovery. To illustrate how MLTM-D can be used for diagnosis, an application to a large-scale test of mathematics achievement is presented. An advantage of MLTM-D for diagnosis is that it may be more applicable to large-scale assessments with more heterogeneous items than are latent class models.

A multicomponent latent trait model for diagnosis.

Working memory capacity is an important construct in psychology because of its relationship with many higher-order cognitive abilities and psychopathologies. Working memory capacity is often measured using a type of paradigm known as complex span. Some recent work has focused on shortening the administration time of the complex span tasks, resulting in different versions of these tasks being used (Foster et al., 2015; Oswald, McAbee, Redick, & Hambrick, 2015). Variations in the complex span tasks, such as the number of set sizes, can lead to varying power to discriminate individuals at different ability levels. Thus, research findings may be inconsistent across populations due to differing appropriateness for the ability levels. The present study uses a combination of item response theory and correlational analyses to better understand the psychometric properties of the operation span, symmetry span, and rotation span. The findings show that the typical administration of these tasks, particularly the operation span, is not suitable for above average ability samples (Study 1; n = 573). When larger set sizes are added to the tasks (Study 2; n = 351), predictive validity and discriminability is improved for all complex span tasks, however the operation span is still inferior to the spatial tasks. The authors make several conclusions about which tasks and set sizes should be used depending on the intended population, and further suggest avoiding the standard-length operation span for average or higher ability populations. (PsycINFO Database Record

Susan E. Embretson

Papers

Personality Measurement Issues Viewed Through the Eyes of IRT

Psychometric Approaches to Understanding and Measuring Intelligence

Designing Cognitive Complexity in Mathematical Problem-Solving Items:

A multicomponent latent trait model for diagnosis.

What item response theory can tell us about the complex span tasks.