Intra-rater reliability

About: Intra-rater reliability is a research topic. Over the lifetime, 2073 publications have been published within this topic receiving 140968 citations.

Assessing Neonatal Pain with NIPS and COMFORT-B: Evaluation of NICU's Staff Competences

Abstract: Background Pain is considered “the 5th vital sign” that should be regularly assessed in the neonatal intensive care setting. Although over 40 pain assessment tools have been developed for neonates, their implementation in everyday practice is challenging. Epidemiological studies demonstrate that pain is still underassessed and undertreated in European NICUs. Purpose To evaluate the interrater and intrarater reliability of the NIPS and COMFORT-B scales among the tertiary NICU's staff members 4 years after their implementation in local pain guidelines with no prior dedicated training. Methods Physicians and nurses were invited to evaluate 5 video recordings of infants hospitalized in the intensive care settings, using the NIPS and COMFORT-B scales. The assessment took part twice at a 3-month interval. Interrater reliability was calculated for both scales using Kendall's W coefficient of concordance and Krippendorff's alpha coefficient. Cohen's kappa was used to assess intrarater reliability. Results 17 physicians and 19 nurses took part in the study. Interrater agreement for the COMFORT-B scale was above 0.8 for Kendall's W coefficient (p < .01) and above 0.667 for Krippendorff's alpha coefficient. Kendall's W coefficient for the NIPS scores ranged between 0.7 and 0.8 (p < .01). Krippendorff's alpha was above 0.667. Intrarater agreement for both the COMFORT-B and NIPS scales was 0.693 and 0.724, respectively. Conclusions Overall, the agreement between our staff members was moderately good for both scales. This is not enough to avoid inadequate pain assessment. More training is needed to improve NICU's staff competences in using pain scales.

Reliability of an ice hockey-specific complex test.

Abstract: Background This study assessed the intra-rater reliability of an ice hockey-specific complex test (IHCT) that reflects the intense multidirectional and intermittent efforts required in ice hockey. Methods Twenty professional players executed the IHCT twice over an interval of 2 days. Load parameters included a 10-m sprint, a 30-m transition with and without a puck, as well as slap and wrist shots. Stress parameters were lactate concentrations and heart rates. Results The intrarater reliability (ICC) ranged from 0.54 to 0.98 for load parameters, and from 0.26 to 0.87 for stress parameters. 33 % (3/9) of stress parameters and 82 % (18/22) of load parameters had an ICC > 0.75. The largest limits of agreement were 41.6 for slap shot 1 after test and 50.4 for 6-min heart rate recovery. The smallest were 0.40 (10-m sprint without a puck) and 5.36 (resting lactate). The intra-rater reliability as shown by the coefficient of variation (CV) was lower for stress than for load parameters (mean CV: 13.4 vs. 4.7). 68 % (15/22) of load parameters and 11 % (1/9) of stress parameters showed a CV below 5 %. Conclusion The IHCT is currently the only ice hockey-specific complex test with scientifically tested reliability and validity that can analyze performance under conditions similar to competition. It is suggested that coaches, fitness coaches and sports scientists explore use of this test as a valid tool to assess players’ performance and the effectiveness of training interventions.