Statistical methods for assessing observer variability in clinical measures.

TLDR: This review attempts to summarise in simple terms the statistical techniques available to quantify the variation within and between observers.

Abstract: The lack of consistency ofmuch medical judgment and decision making has long been appreciated. This is now recognised as an important source of error, and attempts to quantify it have been enhanced by an increasingly sophisticated statistical toolbox at the disposal of the clinical investigator. Variability in measurement and classification may arise from two sources, (a) a lack of consistency within an individual observer (or measuring process) when carrying out successive recordings, and (b) a lack of consistency between observers. Assessing lack of consistency between observers is important for two reasons. Firstly, in any single study using more than one observer an assessment of the variation between them is essential in interpreting the data derived. Secondly, the observer variation seen in one study may be extrapolated to other studies of the same technique but using different observers-that is, the origin of the variation may be inherent in the method itself. This review attempts to summarise in simple terms the statistical techniques available to quantify the variation within and between observers.