Reliability coefficients often take the form of intraclass correlation coefficients. In this article, guidelines are given for choosing among six different forms of the intraclass correlation for reliability studies in which n target are rated by k judges. Relevant to the choice of the coefficient are the appropriate statistical model for the reliability and the application to be made of the reliability results. Confidence intervals for each of the forms are reviewed.

/pdf/intraclass-correlations-uses-in-assessing-rater-reliability-1elowl8sg3.pdf

Intraclass correlations: uses in assessing rater reliability.

In 1984, Jacobson, Follette, and Revenstorf defined clinically significant change as the extent to which therapy moves someone outside the range of the dysfunctional population or within the range of the functional population. In the present article, ways of operationalizing this definition are described, and examples are used to show how clients can be categorized on the basis of this definition. A reliable change index (RC) is also proposed to determine whether the magnitude of change for a given client is statistically reliable. The inclusion of the RC leads to a twofold criterion for clinically significant change.

/pdf/clinical-significance-a-statistical-approach-to-defining-1qbyghhgpi.pdf

Clinical significance: a statistical approach to defining meaningful change in psychotherapy research.

"With its emphasis on practical and conceptual aspects, rather than mathematics or formulas, this accessible book has established itself as the go-to resource on confirmatory factor analysis (CFA). Detailed, worked-through examples drawn from psychology, management, and sociology studies illustrate the procedures, pitfalls, and extensions of CFA methodology. The text shows how to formulate, program, and interpret CFA models using popular latent variable software packages (LISREL, Mplus, EQS, SAS/CALIS); understand the similarities and differences between CFA and exploratory factor analysis (EFA); and report results from a CFA study. It is filled with useful advice and tables that outline the procedures. The companion website offers data and program syntax files for most of the research examples, as well as links to CFA-related resources. New to This Edition *Updated throughout to incorporate important developments in latent variable modeling. *Chapter on Bayesian CFA and multilevel measurement models. *Addresses new topics (with examples): exploratory structural equation modeling, bifactor analysis, measurement invariance evaluation with categorical indicators, and a new method for scaling latent variables. *Utilizes the latest versions of major latent variable software packages"--

/pdf/confirmatory-factor-analysis-for-applied-research-4crbcxiv74.pdf

Confirmatory Factor Analysis for Applied Research

SUMMARY A generalized form of the cross-validation criterion is applied to the choice and assessment of prediction using the data-analytic concept of a prescription. The examples used to illustrate the application are drawn from the problem areas of univariate estimation, linear regression and analysis of variance.

https://rss.onlinelibrary.wiley.com/doi/pdf/10.1111/j.2517-6161.1974.tb00994.x

Cross-Validatory Choice and Assessment of Statistical Predictions

Coefficient alpha (Cronbach, 1951) is certainly one of the most important and pervasive statistics in research involving test construction and use. A review of the Social Sciences Citations Index for the literature from 1966 to 1990 revealed that Cronbach's (1951) article had been cited approximately 60 times per year and in a total of 278 different journals. In addition to the areas of psychology in which one may expect to see alpha used, such as educational, industrial, social, clinical, child, community, and abnormal psychology, this list of journals included representatives from experimental psychology, sociology, statistics, medicine, counseling, nursing, economics, political science, criminology, gerontology, broadcasting, anthropology, and accounting. In spite of its widespread use, however, there is some confusion as to the true meaning and proper interpretation of the statistic. In this article I address this confusion in two ways. First, a theoretical discussion of alpha is presented. This includes some of the many statements that have been made about alpha and an attempt to integrate these statements. Second, I take a more practical approach in which the interpretation of alpha is examined by observing the changes in alpha as the number of items and interitem correlations are manipulated.

/pdf/what-is-coefficient-alpha-an-examination-of-theory-and-22a9qx5n0t.pdf

What Is Coefficient Alpha? An Examination of Theory and Applications

This is a reprint of the orginal book released in 1968. Our primary goal in this book is to sharpen the skill, sophistication, and in- tuition of the reader in the interpretation of mental test data, and in the construction and use of mental tests both as instruments of psychological theory and as tools in the practical problems of selection, evaluation, and guidance. We seek to do this by exposing the reader to some psychologically meaningful statistical theories of mental test scores. Although this book is organized in terms of test-score theories and models, the practical applications and limitations of each model studied receive substantial emphasis, and these discussions are presented in as nontechnical a manner as we have found possible. Since this book catalogues a host of test theory models and formulas, it may serve as a reference handbook. Also, for a limited group of specialists, this book aims to provide a more rigorous foundation for further theoretical research than has heretofore been available.One aim of this book is to present statements of the assumptions, together with derivations of the implications, of a selected group of statistical models that the authors believe to be useful as guides in the practices of test construction and utilization. With few exceptions we have given a complete proof for each major result presented in the book. In many cases these proofs are simpler, more complete, and more illuminating than those originally offered. When we have omitted proofs or parts of proofs, we have generally provided a reference containing the omitted argument. We have left some proofs as exercises for the reader, but only when the general method of proof has already been demonstrated. At times we have proved only special cases of more generally stated theorems, when the general proof affords no additional insight into the problem and yet is substantially more complex mathematically.

Statistical Theories of Mental Test Scores

Statistical Theories of Mental Test Scores.

In a logical probability approach to inference, distributions on a parameter space are interpretable as representing states of knowledge, and any prevailing state of knowledge may be taken to have been arrived at from a previous state of ignorance (indifference) followed by an accumulation of prior data. In this paper an indifference procedure is introduced that requires postulating what size and what kind of samples will and will not (in a special sense) permit statistical inference and prediction—e.g., one observation from a two-parameter normal model is not (in our special sense) sufficient to permit inference about the variance but two observations are. In essence, the procedure stipulates that prior indifference distributions be improper but become proper after an appropriate minimal sample. With some limitation on the family of priors considered, this procedure permits unique specification of indifference for the more commonly encountered statistical models. Furthermore, these specification...

/pdf/a-bayesian-indifference-procedure-4h6c8c69rx.pdf

A Bayesian Indifference Procedure

A Bayesian logical probability approach is described and its advantages with respect to constructing prior distributions for Bayesian analyses are discussed. Some standard Bayesian distribution theory for categorized data is summarized. Data from an on-going experiment to compare the relative efficacy of four operative treatments for ulcer are used both to pinpoint some of the practical problems involved in clinical trials and to demonstrate the usefulness of Bayesian inference methods when facing these problems.

Melvin R. Novick

Papers

Statistical Theories of Mental Test Scores

Statistical Theories of Mental Test Scores.

A Bayesian Indifference Procedure

A Bayesian Approach to the Analysis of Data from Clinical Trials