2 Taxonomy is always a contentious issue because the world does not come to us in neat little packages (S. Personality has been conceptualized from a variety of theoretical perspectives, and at various levels of Each of these levels has made unique contributions to our understanding of individual differences in behavior and experience. However, the number of personality traits, and scales designed to measure them, escalated without an end in sight (Goldberg, 1971). Researchers, as well as practitioners in the field of personality assessment, were faced with a bewildering array of personality scales from which to choose, with little guidance and no overall rationale at hand. What made matters worse was that scales with the same name often measure concepts that are not the same, and scales with different names often measure concepts that are quite similar. Although diversity and scientific pluralism are useful, the systematic accumulation of findings and the communication among researchers became difficult amidst the Babel of concepts and scales. Many personality researchers had hoped that they might devise the structure that would transform the Babel into a community speaking a common language. However, such an integration was not to be achieved by any one researcher or by any one theoretical perspective. As Allport once put it, " each assessor has his own pet units and uses a pet battery of diagnostic devices " (1958, p. 258). What personality psychology needed was a descriptive model, or taxonomy, of its subject matter. One of the central goals of scientific taxonomies is the definition of overarching domains within which large numbers of specific instances can be understood in a simplified way. Thus, in personality psychology, a taxonomy would permit researchers to study specified domains of personality characteristics, rather than examining separately the thousands of particular attributes that make human beings individual and unique. Moreover, a generally accepted taxonomy would greatly facilitate the accumulation and communication of empirical findings by offering a standard vocabulary, or nomenclature. After decades of research, the field is approaching consensus on a general taxonomy of personality traits, the " Big Five " personality dimensions. These dimensions do not represent a particular theoretical perspective but were derived from analyses of the natural-language terms people use to describe themselves 3 and others. Rather than replacing all previous systems, the Big Five taxonomy serves an integrative function because it can represent the various and diverse systems of personality …

The Big Five Trait taxonomy: History, measurement, and theoretical perspectives.

This personal historical article traces the development of the Big-Five factor structure, whose growing acceptance by personality researchers has profoundly influenced the scientific study of individual differences. The roots of this taxonomy lie in the lexical hypothesis and the insights of Sir Francis Galton, the prescience of L. L. Thurstone, the legacy of Raymond B. Cattell, and the seminal analyses of Tupes and Christal. Paradoxically, the present popularity of this model owes much to its many critics, each of whom tried to replace it, but failed. In reaction, there have been a number of attempts to assimilate other models into the five-factor structure. Lately, some practical implications of the emerging consensus can be seen in such contexts as personnel selection and classification.

/pdf/the-structure-of-phenotypic-personality-traits-2a2319vija.pdf

The Structure of Phenotypic Personality Traits

Pedagosko i psiholosko testiranje i procjenjivanje spadaju među najvažnije doprinose znanosti o ponasanju nasem drustvu i pružaju temeljna i znacajna poboljsanja u odnosu na ranije postupke. Iako se ne može ustvrditi da su svi testovi dovoljno usavrseni niti da su sva testiranja razborita i korisna, postoji velika kolicina informacija koje ukazuju na ucinkovitost kvalitetnih instrumenata u onim situacijama u kojima je njihovo koristenje potkrijepljeno validacijskim podacima. Pravilna upotreba testova može dovesti do boljih odluka o pojedincima i programima nego sto bi to bio slucaj bez njihovog koristenja, a također i ukazati na put za siri i pravedniji pristup obrazovanju i zaposljavanju. Međutim, losa upotreba testova može dovesti do zamjetne stete nanesene ispitanicima i drugim sudionicima u procesu donosenja odluka na temelju testovnih podataka. Cilj Standarda je promoviranje kvalitetne i eticne upotrebe testova te uspostavljanje osnovice za ocjenu kvalitete postupaka testiranja. Svrha objavljivanja Standarda je uspostavljanje kriterija za evaluaciju testova, provedbe testiranja i posljedica upotrebe testova. Iako bi evaluacija prikladnosti testa ili njegove primjene trebala ovisiti prvenstveno o strucnim misljenjima, Standardi pružaju okvir koji osigurava obuhvacanje svih relevantnih pitanja. Bilo bi poželjno da svi autori, sponzori, nakladnici i korisnici profesionalnih testova usvoje Standarde te da poticu druge da ih također prihvate.

Standards for educational and psychological testing

On robust estimation of the location parameter

Measurement and control of response bias.

We commend McShane et al. on their innovative approach to analyzing variation and covariation in replication studies and appreciate the chance to participate in this discussion of the issues. Three important challenges of empirical disciplines are replication, generalization, and integration of findings. One way to think of these three is sequential: First replicate, then generalize, and finally integrate. This ordering is the motivation for direct replications, including for large-scale replication studies of multiple effects (e.g., the Many Labs Project or MLP; Klein et al. 2014). The kind of generalization that is of interest in such studies is generalization across labs and their participant populations, using the exact same design and experimental materials in each lab. This is different from generalization across different ways of implementing a study (DeKay et al. in press; Landy et al. 2020). As a result, large-scale replication studies leave out the larger part of what generalization is. Additionally, as McShane et al. note, large-scale efforts involving multiple phenomena almost always analyze the effects separately, so nothing is learned about how the effects and the corresponding variates might vary and covary. Variates are the dependent variables (DVs) as measured per experimental condition. The contributions of large-scale efforts are thus mixed. On the positive side, we have much greater certainty about the replicability of a growing list of specific effects across labs and settings, which is nothing to sneeze at. On the negative side, however, we have learned very little about the generalizability of these effects across different implementations and methods factors, and almost nothing about how the various effects and corresponding variates might be integrated into a more coherent view of the science. For the most part, generalization and integration are still handled in qualitative reviews that organize the literature on the basis of conceptual similarities and theoretical connections. In comparison to the promising situation for replicability, the dearth of direct empirical evidence for generalizability and (especially) integration can be viewed as a failure of largescale projects to live up to their potential. But that seems too harsh, particularly for such recent endeavors. A more positive, forward-looking view is that these knowledge gaps represent a grand opportunity for advances in research design and analysis. As explained below, we believe that McShane et al.’s

The Potential of Factor Analysis for Replication, Generalization, and Integration

The effect of ignoring local item dependencies on the estimated discrimination parameter

A one-way ANOVA structure is imposed on the item difficulty and the item discrimination parameter of a two-parameter hierarchical IRT model for item effects. Bayesian estimation of the model is illustrated for the Metropolis-Hastings within Gibbs and the data augmented Gibbs procedure. The posterior of the hierarchical IRT model is explored with respect to the location of parameters and the uncertainty of these parameter estimates. The posterior correlations among parameters are shown to be due to trade-off effects among parameters either on the same parameter scales or on different parameter scales.

Exploring the posterior of a hierarchical IRT model for item effects

The replication crisis has led to a renewed discussion about the impacts of measurement quality on the precision of psychology research. High measurement quality is associated with low measurement error, yet the role of reliability in the quality of experimental research is not always well understood. In this study, we attempt to understand the role of reliability through its relationship with power while focusing on between-group designs for experimental studies. We outline a latent variable framework to investigate this nuanced relationship through equations. An under-evaluated aspect of the relationship is the variance and homogeneity of the subpopulation from which the study sample is drawn. Higher homogeneity implies a lower reliability, but yields higher power. We proceed to demonstrate the impact of this relationship between reliability and power by imitating different scenarios of large-scale replications with between-group designs. We find negative correlations between reliability and power when there are sizable differences in the latent variable variance and negligible differences in the other parameters across studies. Finally, we analyze the data from the replications of the ego depletion effect (Hagger et al., 2016) and the replications of the grammatical aspect effect (Eerland et al., 2016), each time with between-group designs, and the results align with previous findings. The applications show that a negative relationship between reliability and power is a realistic possibility with consequences for applied work. We suggest that more attention be given to the homogeneity of the subpopulation when study-specific reliability coefficients are reported in between-group studies. 

/pdf/understanding-the-role-of-subpopulations-and-reliability-in-1vjtz4z2.pdf

Paul De Boeck

Papers

The Potential of Factor Analysis for Replication, Generalization, and Integration

The effect of ignoring local item dependencies on the estimated discrimination parameter

Exploring the posterior of a hierarchical IRT model for item effects

Understanding the role of subpopulations and reliability in between-group studies

The Hierarchical Rater Thresholds Model for Multiple Raters and Multiple Items