Showing papers on "Intraclass correlation published in 1968"

Intraclass correlation coefficient as an index of reliability of median scale values for sets of stimuli rated by equal-appearing intervals.

Abstract: In scaling experiments using the method of equal-appearing intervals (Guilford, 1954), E may choose either mean ratings or median ratings as scale values. Guilford (1954, p. 204) says, \"When the frequency distrib~itions are not truncated, the mean is better. When distributions are truncated, medians are better.'' E usually desires an estimate of the reliability of his scale values. If the scale values are mean ratings and if he is willing to assume that \". . . the variance due to differences between the mean ratings is part of the error of measurement and does not represent a systematic source of variation'' (Winer, 1962, p. 128) , then an appropriate index would be the intraclass correlation coefficient for ave rages .Vh i s coefficient coilld be interpreted as follows: If the experiment were to be repeated with another random sample of . . . judges [consisting of the same number], but with the same . . . [set of stimuli], the correlation between the mean ratings obtained from the two sets of data on the same . . . [set of stimuli] would be approximarely . . . [the value obtained for the intraclass correlation coefficient] (Winer, 1962, p. 1 2 8 ) . The purpose of this note is to present a rationale for generalizing the use of the intraclass correlation coefficient for averages to experiments in which the scale values are median ratings. There is some empirical evidence that median ratings order sets of stimuli in approxinzately the same manner as do mean ratings, regardless of the forms of the distributions of these ratings for individual stimuli or the number of points on the scale. The greatest differences in the orderings of a ser of stimuli based upon mean and median ratings would probably be expected to occur when the distributions of judgments for the majority of the stimuli are skewed. When the distributions of judgments are approximately symmetrical for most of the stimuli, an ordering based upon median r a t i n g would probablv not be expected to differ systematically from one based upon mean racings. For four of the five sets of data reanalyzed by Silverman ( 1 9 6 7 ) , the majority of the distributions of judges' rating, were skewed. In the studies from which these data were taken, the number of points on the scales ranged from 7 to 11. Yet, the correlation between the mean and median ratings for each of these sets was high ( r 2 0.99). While these findings do not provide final proof that the correlation between mean and median ratings would be high for any set of experimental data, it seems likely. Thus, there is some justification for using the intraclass correlation coefficient for averages for median as well as mean ratings.