An EULAR Study Group pilot study on reliability of simple capillaroscopic definitions to describe capillary morphology in rheumatic diseases

TLDR: In this paper, simple definitions of hairpin, tortuous or crossing were proposed for describing morphology of capillaries by rheumatologists with varying levels of expertise, and inter-rater agreement was assessed by calculation of proportion of agreement and by kappa coefficients.

Abstract: OBJECTIVE\nTo propose simple capillaroscopic definitions for interpretation of capillaroscopic morphologies and to assess inter-rater reliability.\n\n\nMETHODS\nThe simple definitions proposed were: normal--hairpin, tortuous or crossing; abnormal--not hairpin, not tortuous and not crossing; not evaluable--whenever rater undecided between normal and abnormal. Based upon an aimed kappa of 0.80 and default prevalences of normal (0.4), abnormal (0.4) and not evaluable (0.2) capillaries, 90 single capillaries were presented to three groups of raters: experienced independent raters, n = 5; attendees of the sixth EULAR capillaroscopy course, n = 34; novices after a 1-h course, n = 11. Inter-rater agreement was assessed by calculation of proportion of agreement and by kappa coefficients.\n\n\nRESULTS\nMean kappa based on 90 capillaries was 0.47 (95% CI: 0.39, 0.54) for expert raters, 0.40 (95% CI: 0.36, 0.44) for attendees and 0.46 (95% CI: 0.41, 0.52) for novices, with overall agreements of 67% (95% CI: 63, 71), 63% (95% CI: 60, 65) and 67% (95% CI: 63, 70), respectively. Comparing only normal vs the combined groups of abnormal and not evaluable capillaries did increase the kappa: 0.51 (95% CI: 0.37 ,: 0.65), 0.53 (95% CI: 0.49, 0.58) and 0.55 (95% CI: 0.49, 0.62). On the condition that the capillaries were classifiable, the mean kappa was 0.62 (95% CI: 0.50, 0.74) for expert raters (n = 65), 0.76 (95% CI: 0.69, 0.83) for attendees (n = 20) and 0.81 (95% CI: 0.74, 0.89) for novices (n = 44).\n\n\nCONCLUSION\nThis multicentre, international study showed moderate reliability of simple capillaroscopic definitions for describing morphology of capillaries by rheumatologists with varying levels of expertise. Novices were capable of distinguishing normal from abnormal capillaries by means of a 1-h training session. In future studies, the class not evaluable may be obsolete.