Intra-rater reliability

About: Intra-rater reliability is a research topic. Over the lifetime, 2073 publications have been published within this topic receiving 140968 citations.

Facioscapulohumeral Dystrophy Natural History Study: Standardization of Testing Procedures and Reliability of Measurements

Abstract: Background and Purpose. The natural history of facioscapulohumeral muscular dystrophy (FSHD) has not been studied prospectively. Knowledge of the natural progression of any disease provides essential information for the design of clinical trials. We present a protocol for the study of the natural history of FSHD using quantitative muscle testing (QMT), manual muscle testing (MMT), and functional testing. Subjects. Thirty-two persons with FSHD (mean age=36.1 years, SD=9.6, range=17–49) and 32 age- and gender-matched volunteer controls (mean age=35.8 years, SD=8.0, range=23–50) served as subjects. Methods. Using standardized testing procedures, we examined intrarater reliability of the MMT, QMT, and functional testing measurements in both groups. We also examined interrater reliability in 7 subjects with FSHD. Eighteen muscle groups were tested for each subject using QMT and MMT. Results. Intraclass correlation coefficient (ICC) values ranged from .86 to .99 for intrarater reliability and from .86 to .99 for interrater reliability of QMT measurements. Weighted kappa values of .81 to .98 for intrarater reliability and .50 to 1.00 for interrater reliability were obtained for MMT measurements. Intrarater ICCs for various functional testing measures ranged from .60 to .97. In addition, the comparability of the two QMT machines used in the study was demonstrated by testing the same set of volunteer controls on each machine's linear force transducer (ICC=.89–.98). Conclusion and Discussion. We conclude that this standardized testing protocol produces reliable measurements of muscle strength and functional ability in subjects with FSHD.

Development of a Physical Performance and Mobility Examination

Abstract: OBJECTIVE: To develop and validate the Physical Performance and Mobility Examination (PPME), an observer-administered, performance-based instrument assessing 6 domains of physical functioning and mobility for hospitalized elderly. DESIGN: Development of a pass-fail and 3-level scoring system and training manuals for the PPME instrument for use in both clinical and research settings. Two patient samples were used to assess construct validity and interrater reliability of the PPME. A third sample was selected to assess the test-retest reliability of the instrument. SETTING/PATIENTS: (1) 146 subjects ≥65 years of age with impaired mobility admitted to Medical Units of Stanford University Hospital. (2) 352 subjects ≥65 admitted to acute Medical and Surgical Services of the Palo Alto VA Medical Center. Patient samples were obtained during hospitalization and followed until 3 months post-discharge. To study test-retest reliability, 50 additional patients, whose clinical condition was stable, were selected from both settings. METHODS: An expert panel selected 6 mobility tasks integral to daily life: bed mobility, transfer skills, multiple stands from chair, standing balance, step-up, and ambulation. Tasks were piloted with frail hospitalized subjects for appropriateness and safety. Test-retest and interrater reliability and construct validity were evaluated. Construct validity was tested using the Folstein Mini-Mental State Examination, Activities of Daily Living (ADL), Instrumental Activities of Daily Living (IADL), Geriatric Depression Scale, and modified Medical Outcomes Study Measure of Physical Functioning (MOS-PFR). Two scoring schema were developed for each task: (1) dichotomous pass-fail and (2) 3-level high pass, low pass, and fail. A summary scale was developed for each method of scoring. MAIN RESULTS: High interrater reliability and intrarater reliability were demonstrated for individual tasks. The mean percent agreement (interrater) for each pass/fail task ranged from 96 to 100% and from 90 to 100% for the 3 pairs of raters for each task using the 3-level scoring. Kappas for individual pairs of raters ranged from .80 to 1.0 for pass-fail scoring and from .75 to 1.0 for 3-level scoring (all P < 0.01). Intraclass correlation coefficients for 3-level scoring by pairs of raters ranged from .66 to 1.0. For summary scales, the mean intraclass correlation was .99 for both scoring schema. Test-retest reliability for summary scales using kappa coefficients was .99 for both pass-fail and 3-level scoring, and .99 and .98, respectively, using Pearson Product Moment Correlation. Correlations of PPME with other instruments (construct validity) suggest that the PPME adds a unique dimension of mobility beyond that measured by self-reported ADLs and physical functioning, and it is not greatly influenced by mood or mental status (r = 0.70 (ADL), r = 0.43 (IADL), r = 0.36 (MMSE), r = 0.71 (MOS-PFR), r = 0.23 (GDS)). The 3-level summary scale was sensitive to the variability in the patient population and exhibited neither ceiling nor floor effects. CONCLUSIONS: The PPME is a reliable and valid performance-based instrument measuring physical functioning and mobility in hospitalized and frail elderly.

Intrarater reliability of the functional movement screen.

Abstract: The Functional Movement Screen (FMS) is a tool that quantifies movement patterns as a way to detect performance asymmetries. Although previous study has investigated the reliability of FMS, no current research has examined intrarater reliability or how clinical experience plays a role in the reliability of this tool. In this controlled laboratory study design, repeated measures were used to investigate how experience using the FMS and clinical experience as an athletic trainer (AT) affects the intrarater reliability of FMS testing. Before the data collection, 3 individuals recruited from the university community provided signed informed consent to serve as videotaped models performing the FMS test. The participants (raters) in the study, with different levels of FMS and clinical experience, viewed each of the 3 videotaped models and rated the video models on each exercise of the FMS according to the script that was presented by one of the study investigators. A week later, the participants watched the same videos again, in a different randomized order, and rated each video model on each exercise. After the scores from the participants were collected from both sessions, the intersession scores of the FMS were examined to establish intrarater reliability of all the participants. Additionally, the intrarater reliability of different groups of clinicians and students was compared to make inferences about the influence of clinical experience as an AT along with previous experience using the FMS. The ATs with at least 6 months of experience using the FMS (ATExp group) had the strongest intrarater reliability [intraclass correlation coefficients, ICC (2,1): 0.946], followed by the AT group with moderate reliability [ICC (2,1): 0.771]. This study indicates that intrarater reliability is strong and seems to strengthen when the individuals have experience using the FMS in addition to clinical experience.

Interrater and intrarater reliability of the Assisting Hand Assessment

Abstract: OBJECTIVE. The aim of this study was to evaluate interrater and intrarater reliability for the Assisting Hand Assessment. METHOD. For interrater reliability, two designs were used: 2 occupational therapists rated the same 18 children, and 20 occupational therapists rated the same 8 children. For intrarater reliability, 20 raters each rated one child twice. Both English and Swedish versions of the instrument were used. Intraclass correlation coefficients (ICCs) and standard error of measurement (SEM) were calculated. RESULTS. ICCs for the sum score for the interrater were 0.98 (two raters) and 0.97 (20 raters) and for the intrarater 0.99. SEM was 1.5 for interrater and 1.2 for the intrarater study, which gave an error interval of ± 3 raw scores for interrater and ± 2.4 raw scores for intrarater. CONCLUSION. This study shows excellent interrater and intrarater reliability for sum scores.