Showing papers on "Intra-rater reliability published in 2002"

Reliability of measurements obtained with the Modified Ashworth Scale in the lower extremities of people with stroke

Abstract: Background and Purpose. Abnormal muscle tone is a common motor disorder following stroke, which may require rehabilitation. The Modified Ashworth Scale is a 6-point rating scale that is used to measure muscle tone. The interrater and intrarater reliability of measurements obtained with the scale remain equivocal. The purpose of this study was to investigate the reliability of measurements obtained with the scale in the lower limb of patients with stroke. Subjects. Twenty patients were tested 2 weeks after their stroke, and 12 patients were tested 12 weeks after their stroke. Methods. Gastrocnemius, soleus, and quadriceps femoris muscles on the hemiplegic side were tested. Results. Interrater reliability for 2 raters was poor, with a Kendall tau-b correlation for the combined muscle group of .062 ( P =.461). For intrarater reliability, the Kendall tau-b correlation was .567 ( P <.001). The agreement within one rater occurred mostly on the grade of 0. Discussion and Conclusion. The Modified Ashworth Scale yielded reliable measurements in the lower limb for a single examiner, and agreement was best on the grade of 0. The reliability between examiners was not good, which may bring into question the validity of measurements obtained with the scale.

Intrarater Reliability of Functional Performance Tests for Subjects With Patellofemoral Pain Syndrome.

Abstract: OBJECTIVE: Patellofemoral pain syndrome (PFPS) is a common clinical entity seen by the sports medicine specialist. The ultimate goal of rehabilitation is to return the patient to the highest functional level in the most efficient manner. Therefore, it is necessary to assess the progress of patients with PFPS using reliable functional performance tests. Our purpose was to evaluate the intrarater reliability of 5 functional performance tests in patients with PFPS. DESIGN AND SETTING: We used a test-retest reliability design in a clinic setting. SUBJECTS: Two groups of subjects were studied: those with PFPS (n = 29) and those with no known knee condition (n = 11). The PFPS group included 19 women and 10 men with a mean age of 27.6 +/- 5.3 years, height of 169.80 +/- 10.5 cm, and weight of 69.59 +/- 15.8 kg. The normal group included 7 women and 4 men with a mean age of 30.3 +/- 5.2 years, height of 169.55 +/- 9.9 cm, and weight 69.42 +/- 14.6 kg. MEASUREMENTS: The reliability of 5 functional performance tests (anteromedial lunge, step-down, single-leg press, bilateral squat, balance and reach) was assessed in 15 subjects with PFPS. Secondly, the relationship of the 5 functional tests to pain was assessed in 29 PFPS subjects using Pearson product moment correlations. The limb symmetry index (LSI) was calculated in the 29 PFPS subjects and compared with the group of 11 normal subjects. RESULTS: The 5 functional tests proved to have fair to high intrarater reliability. Intrarater reliability coefficients (ICC 3,1) ranged from.79 to.94. For the PFPS subjects, a statistical difference existed between limbs for the anteromedial lunge, step-down, single-leg press, and balance and reach. All functional tests correlated significantly with pain except for the bilateral squat; values ranged from.39 to.73. The average LSI for the PFPS group was 85%, while the average LSI for the normal subjects was 97%. CONCLUSIONS: The 5 functional tests proved to have good intrarater reliability and were related to changes in pain. Future research is needed to examine interrater reliability, validity, and sensitivity of these clinical tests.

Hip displacement in spastic cerebral palsy: repeatability of radiologic measurement.

Abstract: Radiologic measures of migration percentage (MP) and acetabular index (AI) taken from plain radiographs of the pelvis are the most commonly used tools for determining hip displacement and management options in children with cerebral palsy and spastic hip disease This study determined interrater and intrarater reliability of MP and AI on pelvic radiographs chosen to represent a wide range of age (11 months to 8 years 5 months), MP (0%-56%), and AI (9 degrees -33 degrees ) The study demonstrated that an experienced rater would be expected to measure MP on a single radiograph to within +/-58% of the true value and a change in MP between two radiographs taken at different times to within +/-83% of the true value Similarly for AI, the measurement error for a typical rater would be within +/-26 degrees on a single reading and +/-37 degrees if recording change between two occasions The authors believe that the results indicating true change are acceptable in clinical practice, provided treatment decisions are based on a series of radiographs taken at 6-month intervals, methods and training are standardized, and consistent raters are used

Reliability and accuracy of biomechanical measurements of the lower extremities

Abstract: The reliability of biomechanical measurements of the lower extremities, as they are commonly used in podiatric practice, was quantified by means of intraclass correlation coefficients (ICCs). This was done not only to evaluate interrater and intrarater reliability but also to provide an estimate for the accuracy of the measurements. The measurement protocol involved 30 asymptomatic subjects and five raters of varying experience. Each subject was measured twice by the same rater, with the retest immediately following the test. The study demonstrated that the interrater ICCs were quite low (≤0.51), except for the measurements of relaxed calcaneal stance position and forefoot varus (both 0.61 and 0.62 for left and right, respectively). However, the intrarater ICCs were relatively high (>0.8) for most raters and measurement variables. Measurement accuracy was moderate between raters. (J Am Podiatr Med Assoc 92(6): 317-326, 2002)

How reliably do rheumatologists measure shoulder movement

Abstract: Objective: To assess the intrarater and interrater reliability among rheumatologists of a standardised protocol for measurement of shoulder movements using a gravity inclinometer. Methods: After instruction, six rheumatologists independently assessed eight movements of the shoulder, including total and glenohumeral flexion, total and glenohumeral abduction, external rotation in neutral and in abduction, internal rotation in abduction and hand behind back, in random order in six patients with shoulder pain and stiffness according to a 6×6 Latin square design using a standardised protocol. These assessments were then repeated. Analysis of variance was used to partition total variability into components of variance in order to calculate intraclass correlation coefficients (ICCs). Results: The intrarater and interrater reliability of different shoulder movements varied widely. The movement of hand behind back and total shoulder flexion yielded the highest ICC scores for both intrarater reliability (0.91 and 0.83, respectively) and interrater reliability (0.80 and 0.72, respectively). Low ICC scores were found for the movements of glenohumeral abduction, external rotation in abduction, and internal rotation in abduction (intrarater ICCs 0.35, 0.43, and 0.32, respectively), and external rotation in neutral, external rotation in abduction, and internal rotation in abduction (interrater ICCs 0.29, 0.11, and 0.06, respectively). Conclusions: The measurement of shoulder movements using a standardised protocol by rheumatologists produced variable intrarater and interrater reliability. Reasonable reliability was obtained only for the movement of hand behind back and total shoulder flexion.

Reliability Improvement with Design of Experiments

Abstract: (2002). Reliability Improvement with Design of Experiments. Technometrics: Vol. 44, No. 1, pp. 93-94.

Inter and intra-rater reliability of cervical auscultation to detect aspiration in patients with dysphagia

Abstract: Objective: To measure the inter and intra-rater reliability of cervical auscultation used alone to detect aspiration in dysphagic patients.Setting: A university teaching hospital.Design: Comparison of the detection of aspiration in 16 recorded swallow sounds by five speech and language therapists on two occasions. Swallow sounds were recorded simultaneously with videofluoroscopy.Subjects: Sixteen patients referred for assessment of dysphagia with videofluoroscopy.Results: The kappa statistic for multiple raters showed fair agreement between raters (k = 0.28). There was high agreement when aspiration occurred but in non-aspirating swallows there was significant overdetection of aspiration (p < 0.001 McNemar's test). The intra-rater reliability within different individuals was widely variable (k = 0.55 (range 0.31–0.85)).Conclusions: Presented with the swallowing sounds in isolation speech and language therapists cannot reliably classify swallows into those with accompanying aspiration and those without. Th...

Assessment of the intrarater and interrater reliability of an established clinical task analysis methodology.

Abstract: Background: Task analysis may be useful for assessing how anesthesiologists alter their behavior in response to different clinical situations. In this study, the authors examined the intraobserver and interobserver reliability of an established task analysis methodology. Methods: During 20 routine anesthetic procedures, a trained observer sat in the operating room and categorized in real-time the anesthetist's activities into 38 task categories. Two weeks later, the same observer performed task analysis from videotapes obtained intraoperatively. A different observer performed task analysis from the videotapes on two separate occasions. Data were analyzed for percent of time spent on each task category, average task duration, and number of task occurrences. Rater reliability and agreement were assessed using intraclass correlation coefficients. Results: Intrarater reliability was generally good for categorization of percent time on task and task occurrence (mean intraclass correlation coefficients of 0.84-0.97). There was a comparably high concordance between real-time and video analyses. Interrater reliability was generally good for percent time and task occurrence measurements. However, the interrater reliability of the task duration metric was unsatisfactory, primarily because of the technique used to capture multitasking. Conclusions: A task analysis technique used in anesthesia research for several decades showed good intrarater reliability. Off-line analysis of videotapes is a viable alternative to real-time data collection. Acceptable interrater reliability requires the use of strict task definitions, sophisticated software, and rigorous observer training. New techniques must be developed to more accurately capture multitasking. Substantial effort is required to conduct task analyses that will have sufficient reliability for purposes of research or clinical evaluation.

Comparison of two methods used to assess first-ray mobility

Abstract: Mobility of the first-ray is associated with several common lower extremity disorders. However, the reliability and validity of clinical measurement remains unclear. In this study we examined first-ray mobility by using one hand to stabilize the lesser metatarsals while the clinician's other hand applied a displacement force to the head of the first metatarsal. The amount of mobility was graded as stiff, normal or hypermobile. We then used a well-validated mechanical device to perform similar tests and assessed validity, intrarater reliability and interrater reliability. Three clinicians having varied levels of experience graded first-ray mobility on 15 subjects. A separate investigator measured dorsal mobility with a mechanical device. Both methods of testing were repeated to assess measurement reliability. Reliability was estimated by kappa (K) statistics. Spearman correlation assessed the relationship between mobility graded manually and dorsal mobility measured by device. Manual examination intrarater K values ranged from 0.50 to 0.85, and interrater agreement from 0.09 to 0.16. Manual grading was not related (r = -0.21) to the absolute measure of total dorsal mobility made by device. This brings into question the validity and reliability of manual estimates of first-ray mobility.

Intrarater and interrater reliability of the Eaton classification of basal joint arthritis.

Abstract: The Eaton classification is a radiographic rating used to define the severity of basal joint arthritis. Despite widespread use, the intrarater and interrater reliability has never been determined. Seven men and 33 women, mean age 60 years (range, 31–88 y) were clinically diagnosed with basal joint arthritis. Forty radiographs of these patients' basal joints (21 right and 19 left) were staged according to Eaton, stages I through IV. Three hand surgeons with a Certificate of Added Qualification and 3 orthopedic surgical residents reviewed the films on 2 separate occasions at least 1 week apart. The results of intrarater and interrater reliability were assessed by using κ statistical analysis, with results classified as poor (0–.50), moderate (.51–.75), or excellent (>.75). The overall mean intrarater and interrater reliability was moderate,.657 and.529, respectively. The hand surgeons' intrarater reliability (.666) was slightly better than the residents (.648). There was a greater difference in the mean interrater reliabilities among the hand surgeons (.601) than the orthopedic residents (.487). These results are similar to those found when other orthopedic radiographic classifications have been evaluated for interrater and intrarater reliability. (J Hand Surg 2002;27A:882–885. Copyright © 2002 by the American Society for Surgery of the Hand.)

The reliability of the ITU-t p.85 standard for the evaluation of text-to-speech systems.

Abstract: An evaluation of the reliability of the ITU-T P.85 recommended standard for the evaluation of voice output systems was conducted using six English TTS systems. The P.85 standard is based on mean-opinion-score judgements of a listening panel on a number of rating scales. The study looked at how the ranking of the six systems on the scales varied across four different text genres and across two listening sessions. Rankings were also compared with a much simpler pair-comparison test across genres and listening sessions. For the ITU test a large degree of correlation was found across scales, implying that these were not really testing different aspects of the systems. There were surprisingly similar results across sessions, implying that listeners were indeed making real judgements. In comparison, the pair comparison test gave (almost) identical rankings for systems with far less variability, making statistically significant comparisons between systems possible, even across genres.

The inter-rater and intra-rater reliability of a new Canadian oral examination format in anesthesia is fair to good.

Abstract: Purpose In response to the Royal College’s request to improve the validity and reliability of oral examinations, the Examination Board in anesthesia proposed a structured oral examination format. Prior to its introduction, we studied this format in two residency programs to determine reliability of the examiners.

Infant polysomnography: reliability and validity of infant arousal assessment.

Abstract: Infant arousal scoring based on the Atlas Task Force definition of transient EEG arousal was evaluated to determine (1). whether transient arousals can be identified and assessed reliably in infants and (2). whether arousal and no-arousal epochs scored previously by trained raters can be validated reliably by independent sleep experts. Phase I for inter- and intrarater reliability scoring was based on two datasets of sleep epochs selected randomly from nocturnal polysomnograms of healthy full-term, preterm, idiopathic apparent life-threatening event cases, and siblings of Sudden Infant Death Syndrome infants of 35 to 64 weeks postconceptional age. After training, test set 1 reliability was assessed and discrepancies identified. After retraining, test set 2 was scored by the same raters to determine interrater reliability. Later, three raters from the trained group rescored test set 2 to assess inter- and intrarater reliabilities. Interrater and intrarater reliability kappa's, with 95% confidence intervals, ranged from substantial to almost perfect levels of agreement. Interrater reliabilities for spontaneous arousals were initially moderate and then substantial. During the validation phase, 315 previously scored epochs were presented to four sleep experts to rate as containing arousal or no-arousal events. Interrater expert agreements were diverse and considered as noninterpretable. Concordance in sleep experts' agreements, based on identification of the previously sampled arousal and no-arousal epochs, was used as a secondary evaluative technique. Results showed agreement by two or more experts on 86% of the Collaborative Home Infant Monitoring Evaluation Study arousal scored events. Conversely, only 1% of the Collaborative Home Infant Monitoring Evaluation Study-scored no-arousal epochs were rated as an arousal. In summary, this study presents an empirically tested model with procedures and criteria for attaining improved reliability in transient EEG arousal assessments in infants using the modified Atlas Task Force standards. With training based on specific criteria, substantial inter- and intrarater agreement in identifying infant arousals was demonstrated. Corroborative validation results were too disparate for meaningful interpretation. Alternate evaluation based on concordance agreements supports reliance on infant EEG criteria for assessment. Results mandate additional confirmatory validation studies with specific training on infant EEG arousal assessment criteria.

Improving The Reliability of Product Quality

Abstract: Based on intrducing reliability theory, the viewpoint are discussed that product quality can be improved by using reliability mangagement during design?technology?manufacture and test.

Reliability of the Modified Oswestry Disability Index for evaluating patients with low back pain

Abstract: Objective:To determined the reliability of the Oswestry Disability Index(ODI) for evaluating patients with low back pain.Method:22(female 9,male 13) subjects who were inpatients of the orthopedic words were evaluated using the modified ODI in Chinese before operation.They were divided into two groups:intrarater reliability group(n=10) and interrater reliability group(n=12).The patients answered the ODI.The test retest reliability was done in every 3 days and compared using a Pearson Correlation,individually.Result:The highly correlated coefficient of 0 953( t=8 851,P 0 001) and 0 912( t=7 008,P 0 01)were obtained in intrarater reliability group and interrater reliability group individually.Conclusion:The modified ODI in Chinese shows the superior reliability.This assessment method allows the evaluation of multiple parameters of the patient′s function. It can be used to study the clinical outcome of postoperation or rehabilitation in patients with low back pain.

Reliability and Robustness in Real-Time Control Applications

Abstract: The risk of flooding due to measurement errors, equipment and software defaults or power failures is the highest priority of wet-weather management of collection systems. It is consequently not surprising that safety and reliability issues are raised when looking at the automatic control of sewer networks for better CSO control. If not properly designed, a Real Time Control (RTC) strategy can initiate actions that threaten public safety by causing flooding and structural damages in the absence of quality measurements. For this reason, it is mandatory that all RTC schemes implemented guarantee robust and safe management for all possible degraded situations including the worst case scenario that can happen. The concept of reliability can be largely associated with system performance and especially the difference between the measured performances and those expected. Therefore, one way of guaranteeing system reliability consists in implementing a system for which degraded modes are the exception rather than the rule. To reach such an objective, reliable sensors, actuators, radios, gates, pumps recognized in the industry for their robustness and long life expectancy should be purchased. Equipment redundancy can also be used to improve the reliability of the system. Specifically, two or three water level sensors can be used at strategic locations to guarantee that a valid measurement level will always be available at these sites. Equipment redundancy enables to define data validation procedures and, therefore, to improve the quality of the information process to the decision-making system. Consequently, less uncertainties are introduced in the control strategy and better system performance can be achieved. The telecommunication architecture can also be designed to minimize the frequency of degraded situations due to breakdowns in communications. To achieve this goal, different communication mediums and multiple communication channels can be implemented. Therefore, if a communication link is lost, other channels can be used to process the information between the local control points and the central station. During degraded situations, system reliability must be guaranteed by the implementation of degraded control modes. For maximal performance, each control station will have its own set of degraded management modes in relation to the communication status and the configuration of the station. All degraded modes will be specifically designed to guarantee the protection of the citizen and of the sewer network while achieving the best possible management performance considering available information. The degraded modes will be characterized in terms of the anomaly itemized.

Component-Based Software Reliability Analysis

Abstract: New reliability models are derived to describe how test-based software reliability estimates depend on the component structure of code. The models can analyze dependent component failures. Models of this type are important for two main reasons. Firstly, they provide a quality model for software development based on component reuse. For simple software architectures, it is shown that it is feasible to re-use evidence of a component’s reliability from previous testing/usage in a different system. This technique has potential to provide extremely efficient software verification. Secondly, the new models provide a meaning for ‘reliable software design,’ making it possible to identify software designs whose reliability can be demonstrated. Traditionally, the complexity of a computation is measured in terms of the number of elemental computations used. In contrast, a statistical complexity measure is proposed to describe complexity in terms of statistical software testability. A highly complex program requires intense testing in order to justify a claim that the program achieves a given level of reliability. A low complexity program requires less testing to achieve the same claim.

Reliability Growth Management during Prototype Development

Abstract: Reliability growth is defined as positive improvement of system/equipment reliability through the systematic and permanent removal of specific failure mechanisms. Effective reliability growth is dependent upon the extent to which testing and other improvement techniques have been used during development and production to eliminate design and fabrication flaws, as well as on the rigour with which these flaws/deficiencies are analysed and corrected. In a prototype development scenario of a complex product such as a modern helicopter, reliability growth management has to consider various factors, such as the dynamic nature of design changes, usage environment, inherent reliability of constituent systems (both in-house fabricated and bought out items), quality aspects as well as users' expectations. Major factors contributing for the effective reliability growth management are continuous snag data analysis, reliability trend monitoring and design reviews. This paper presents the details of reliability growth management aspects adopted during prototype development of advanced light helicopter, designed and developed by the Hindustan Aeronautics Ltd, Bangalore. The approach followed lay greater emphasis on regular and continuous snag data analysis, monitoring of efficacy of corrective actions, establishment of an adequate failure reporting and corrective action system, proper documentation, and above all, design review by the top management for assuring reliability. It has been observed that the above management approach has yielded positive results in reducing the number of reliability problems encountered during prototype development. It also provides scope for further reliability improvement in production versions

The Orthopaedic Research Institute-Ankle Strength Testing System: inter-rater and intra-rater reliability testing.

Abstract: Achilles tendinosis is a degenerative overuse tendinopathy involving the primary ankle plantarflexors, namely the soleus and gastrocnemius muscles forming the tendo Achilles. The Orthopaedic Research Institute-Ankle Strength Testing System (ORI-ASTS) was designed to record objective measurements of force generated with a resisted ankle plantarflexion test. Testing normal subjects (n=6) was used to establish the reliability of the ORI-ASTS for measuring ankle plantarflexion force. Testing patients with Achilles tendonitis (n=5) over time and comparing Achilles tendon analogue pain scores to ankle plantarflexion force measurements was used to assess the validity of the ORI-ASTS for monitoring progression of Achilles tendinopathy. Inter-rater reliability of the ORI-ASTS with normal subjects was excellent, with single measure intraclass correlation coefficients (ICC) for right mean peak force of 0.92, left mean peak force of 0.96, right mean total force of 0.89, left mean total force of 0.91. The ORI-ASTS also had excellent intra-rater reliability for normal subjects with the following single measure ICC scores: right mean force 0.96, left mean force 0.92, right mean total force 0.97, left mean total force 0.92. The relative technical errors of measurement were calculated from these results, and ranged from 1.0% to 2.7%. Testing patients with Achilles tendinosis (n=5), and comparing analogue activity pain scores to ORI-ASTS testing demonstrated a strong negative relationship between the two parameters (Spearman Rho -0.87 to -1.0, Kendall tau b -0.82 to -1.0). The relationship was statistically significant at the p=0.01 level for two of the five patients. The ORI-ASTS shows excellent reliability for testing ankle plantarflexor force, and appears valid for objectively assessing and monitoring patients with Achilles tendinosis.

Product reliability assessment using virtual samples

Abstract: Reliability assessment for innovative products is of great interest in manufacturing enterprises. Traditionally, reliability is estimated once products or prototypes are built to go through physical tests. The latter are often time-consuming and very costly to perform. This paper suggests an alternative way to perform reliability tests based on virtual samples. It aims to reduce the number of physical tests.