Intra-rater reliability

About: Intra-rater reliability is a research topic. Over the lifetime, 2073 publications have been published within this topic receiving 140968 citations.

The reliability of the ITU-t p.85 standard for the evaluation of text-to-speech systems.

Abstract: An evaluation of the reliability of the ITU-T P.85 recommended standard for the evaluation of voice output systems was conducted using six English TTS systems. The P.85 standard is based on mean-opinion-score judgements of a listening panel on a number of rating scales. The study looked at how the ranking of the six systems on the scales varied across four different text genres and across two listening sessions. Rankings were also compared with a much simpler pair-comparison test across genres and listening sessions. For the ITU test a large degree of correlation was found across scales, implying that these were not really testing different aspects of the systems. There were surprisingly similar results across sessions, implying that listeners were indeed making real judgements. In comparison, the pair comparison test gave (almost) identical rankings for systems with far less variability, making statistically significant comparisons between systems possible, even across genres.

Reproducible Measurements of Muscle Characteristics Using the MyotonPRO Device: Comparison Between Individuals With and Without Paratonia.

Abstract: Background and purpose: The MyotonPRO is a portable device that measures muscle tone and biomechanical muscle properties objectively. MyotonPRO has already proven to be effective in measuring muscle properties in healthy and diseased populations. However, to the best of our knowledge, it has never been tested in individuals suffering from paratonia, a form of hypertonia frequently accompanying dementia. The aims of the present study were to (1) compare muscle tone, elasticity, and stiffness between 3 different subpopulations of young and old healthy adults and individuals with paratonia, and (2) investigate the intra- and interrater reproducibility of MyotonPRO measurements of the biceps brachii (BB) muscle in each subpopulation. Methods: MyotonPRO measurements of muscle tone, elasticity, and dynamic stiffness were carried out by 2 investigators on 2 different days over the BB muscles of 54 participants (18 healthy young adults, 20 healthy older adults, and 16 older individuals with paratonia). Muscle properties were compared between subpopulations using ANOVA/Welch and post hoc tests. Reliability (intraclass correlation coefficient) and agreement parameters (standard error of measurement and the minimal detectable change) were calculated. Results: Statistically significant differences between subpopulations were found in all parameters, except for stiffness between healthy elderly and individuals with paratonia. In the healthy subpopulations, (a) intrarater reliability was very high and intrarater agreement was good between 2 consecutive series, (b) between days intrarater reliability was low to high and intrarater agreement was variable, (c) interrater reliability was high to very high and interrater agreement was good. In individuals with paratonia, (a) intrarater reliability was moderate to high and agreement was variable between series, (b) between days intrarater reliability was poor to moderate and agreement was poor, (c) interrater reliability ranged from low to high with poor agreement. Conclusions: MyotonPRO measurements of the BB muscle showed good reproducibility in both healthy subpopulations, particularly for measurements performed within the same day. In individuals with paratonia, reliability and agreement were substantially lower. MyotonPRO can be used in clinical assessment and research. However, in individuals with paratonia, careful interpretation of results is required. Research in a larger sample of persons with paratonia at different stages of disease severity is recommended.

Interrater and intrarater reliability in the measurement of ankle joint dorsiflexion is independent of examiner experience and technique used.

Abstract: Background: Goniometric measurement is currently being used as a diagnostic and outcomes assessment tool for ankle joint dorsiflexion. Despite its common use, its interrater and intrarater reliability has been questioned. Methods: This is a prospective study examining whether the experience of the examiner or the technique used affects the interrater and intrarater reliability for measuring ankle joint dorsiflexion. Fourteen asymptomatic individuals (8 male and 6 female) with a mean age of 28.2 years (range, 23–52) were enrolled into this study. The years of clinical experience of the five examiners averaged 10.4 years (range, 0–26). Four examiners used a modified Root, Weed and Orien method of measuring ankle joint dorsiflexion. The fifth examiner utilized a nonstandardized technique. A standard goniometer was used for bilateral measurements of ankle joint dorsiflexion with the knee extended and flexed. All five examiners repeated each measurement three times during each of the three sessions, with each ...

Reliability of magnetic resonance imaging assessment of rotator cuff: the ROW study.

Abstract: Background Physiatrists encounter patients with rotator cuff disorders, and imaging is frequently an important component of their diagnostic assessment. However, there is a paucity of literature on the reliability of magnetic resonance imaging (MRI) assessment between shoulder specialists and musculoskeletal radiologists. Objective We assessed inter- and intrarater reliability of MRI characteristics of the rotator cuff. Design Cross-sectional secondary analyses in a prospective cohort study. Setting Academic tertiary care centers. Patients Subjects with shoulder pain were recruited from orthopedic and physiatry clinics. Methods Two shoulder-fellowship–trained physicians (a physiatrist and a shoulder surgeon) jointly performed a blinded composite MRI review by consensus of 31 subjects with shoulder pain. Subsequently, MRI was reviewed by one fellowship-trained musculoskeletal radiologist. Main Outcome Measurements We calculated the Cohen kappa coefficients and percentage agreement among the 2 reviews (composite review of 2 shoulder specialists versus that of the musculoskeletal radiologist). Intrarater reliability was assessed among the shoulder specialists by performing a repeated blinded composite MRI review. In addition to this repeated composite review, only one of the physiatry shoulder specialists performed an additional review. Results Interrater reliability (shoulder specialists versus musculoskeletal radiologist) was substantial for the presence or absence of tear (kappa 0.90 [95% confidence interval {CI}, 0.72-1.00]), tear thickness (kappa 0.84 [95% CI, 0.70-0.99]), longitudinal size of tear (kappa 0.75 [95% CI, 0.44-1.00]), fatty infiltration (kappa 0.62 [95% CI, 0.45-0.79]), and muscle atrophy (kappa 0.68 [95% CI, 0.50-0.86]). There was only fair interrater reliability of the transverse size of tear (kappa 0.20 [95% CI, 0.00-0.51]). The kappa for intrarater reliability was high for tear thickness (0.88 [95% CI, 0.72-1.00]), longitudinal tear size (0.61 [95% CI, 0.22-0.99]), fatty infiltration (0.89 [95% CI, 0.80,-0.98]), and muscle atrophy (0.87 [95% CI, 0.76-0.98]). Intrarater reliability for the individual shoulder specialist was similar to that of the composite reviews. Conclusions There was high interrater and intrarater reliability for most findings on shoulder MRI. Analysis of our data supports the reliability of MRI assessment by shoulder specialists for rotator cuff disorders.