Background and Purpose. Assessment of the quality of randomized controlled trials (RCTs) is common practice in systematic reviews. However, the reliability of data obtained with most quality assessment scales has not been established. This report describes 2 studies designed to investigate the reliability of data obtained with the Physiotherapy Evidence Database (PEDro) scale developed to rate the quality of RCTs evaluating physical therapist interventions. Method. In the first study, 11 raters independently rated 25 RCTs randomly selected from the PEDro database. In the second study, 2 raters rated 120 RCTs randomly selected from the PEDro database, and disagreements were resolved by a third rater; this generated a set of individual rater and consensus ratings. The process was repeated by independent raters to create a second set of individual and consensus ratings. Reliability of ratings of PEDro scale items was calculated using multirater kappas, and reliability of the total (summed) score was calculated using intraclass correlation coefficients (ICC [1,1]). Results. The kappa value for each of the 11 items ranged from .36 to .80 for individual assessors and from .50 to .79 for consensus ratings generated by groups of 2 or 3 raters. The ICC for the total score was .56 (95% confidence interval=.47–.65) for ratings by individuals, and the ICC for consensus ratings was .68 (95% confidence interval=.57–.76). Discussion and Conclusion. The reliability of ratings of PEDro scale items varied from “fair” to “substantial,” and the reliability of the total PEDro score was “fair” to “good.”

Reliability of the PEDro Scale for Rating Quality of Randomized Controlled Trials

Adequate reporting of randomized, controlled trials (RCTs) is necessary to allow accurate critical appraisal of the validity and applicability of the results. The CONSORT (Consolidated Standards of Reporting Trials) Statement, a 22-item checklist and flow diagram, is intended to address this problem by improving the reporting of RCTs. However, some specific issues that apply to trials of nonpharmacologic treatments (for example, surgery, technical interventions, devices, rehabilitation, psychotherapy, and behavioral intervention) are not specifically addressed in the CONSORT Statement. Furthermore, considerable evidence suggests that the reporting of nonpharmacologic trials still needs improvement. Therefore, the CONSORT group developed an extension of the CONSORT Statement for trials assessing nonpharmacologic treatments. A consensus meeting of 33 experts was organized in Paris, France, in February 2006, to develop an extension of the CONSORT Statement for trials of nonpharmacologic treatments. The participants extended 11 items from the CONSORT Statement, added 1 item, and developed a modified flow diagram. To allow adequate understanding and implementation of the CONSORT extension, the CONSORT group developed this elaboration and explanation document from a review of the literature to provide examples of adequate reporting. This extension, in conjunction with the main CONSORT Statement and other CONSORT extensions, should help to improve the reporting of RCTs performed in this field.

/pdf/extending-the-consort-statement-to-randomized-trials-of-39t9d1gijb.pdf

Extending the CONSORT statement to randomized trials of nonpharmacologic treatment: explanation and elaboration.

Background: The impact of a recurrent defect on the outcome after rotator cuff repair has been controversial. The purpose of this study was to evaluate the functional and anatomic results after arthroscopic repair of large and massive rotator cuff tears with use of ultrasound as an imaging modality to determine the postoperative integrity of the repair.

Methods: Eighteen patients who had complete arthroscopic repair of a tear measuring >2 cm in the transverse dimension were evaluated at a minimum of twelve months after surgery and again at two years after surgery. The evaluation consisted of a standardized history and physical examination as well as calculation of the preoperative and postoperative shoulder scores according to the system of the American Shoulder and Elbow Surgeons. The strength of both shoulders was quantitated postoperatively with use of a portable dynamometer. Ultrasound studies were performed with use of an established and validated protocol at a minimum of twelve months after surgery.

Results: Recurrent tears were seen in seventeen of the eighteen patients. Despite the absence of healing at twelve months after surgery, thirteen patients had an American Shoulder and Elbow Surgeons score of ≥90 points. Sixteen patients had an improvement in the functional outcome score, which increased from an average of 48.3 to 84.6 points. Sixteen patients had a decrease in pain, and twelve had no pain. Although eight patients had preoperative forward elevation to <95°, all eighteen regained motion above shoulder level and had an average of 152° of elevation. At the second evaluation, a minimum of twenty-four months after surgery, the average score, according to the system of the American Shoulder and Elbow Surgeons, had decreased to 79.9 points; only nine patients had a score of ≥90 points, and six patients had a score of ≤79 points. The average forward elevation decreased to 142°.

Conclusions: Arthroscopic repair of large and massive rotator cuff tears led to a high percentage of recurrent defects. The minimum twelve-month evaluation showed excellent pain relief and improvement in the ability to perform activities of daily living despite the high rate of recurrent defects; however, at a minimum follow-up of two years, the results deteriorated with only twelve patients who had an American Shoulder and Elbow Surgeons score of ≥80.

Level of Evidence: Therapeutic study, Level IV (case series [no, or historical, control group]). See Instructions to Authors for a complete description of levels of evidence.

The Outcome and Repair Integrity of Completely Arthroscopically Repaired Large and Massive Rotator Cuff Tears

As the name suggests, evidence-based medicine (EBM), is about finding evidence and using that evidence to make clinical decisions. A cornerstone of EBM is the hierarchical system of classifying evidence. This hierarchy is known as the levels of evidence. Physicians are encouraged to find the highest level of evidence to answer clinical questions. Several papers published in Plastic Surgery journals concerning EBM topics have touched on this subject. 1–6 Specifically, previous papers have discussed the lack of higher level evidence in PRS and need to improve the evidence published in the journal. Before that can be accomplished, it is important to understand the history behind the levels and how they should be interpreted. This paper will focus on the origin of levels of evidence, their relevance to the EBM movement and the implications for the field of plastic surgery as well as the everyday practice of plastic surgery.

/pdf/the-levels-of-evidence-and-their-role-in-evidence-based-1m9l6ykje6.pdf

The Levels of Evidence and their role in Evidence-Based Medicine

Measuring the whole or the parts?: Validity, reliability, and responsiveness of the disabilities of the arm, shoulder and hand outcome measure in different regions of the upper extremity

A standardized method for the assessment of shoulder function

Assessing the reliability and responsiveness of 5 shoulder questionnaires.

Measures of both generic and disease-specific health status are being developed and used with increasing frequency for the appraisal of musculoskeletal conditions. The purpose of this study was to compare prospectively the validity of five questionnaires in the assessment of function of the shoulder. Ninety subjects who had various problems related to the shoulder agreed to enter the study. All of the subjects completed a questionnaire package that included the Shoulder Pain and Disability Index, the Simple Shoulder Test, the Subjective Shoulder Rating Scale, the Modified American Shoulder and Elbow Surgeons Shoulder Patient Self-Evaluation Form, and the Shoulder Severity Index as well as a measure of generic health status (the acute version of the Short Form 36 [SF-36]) and two questions that asked the patient to rate the severity of the problem and his or her over-all health. Frequency distributions were created and compared among questionnaires. Spearman rank correlations were calculated to compare the questionnaires with each other and with other assessments. One-way analysis of variance was used to determine the ability of the questionnaires to discriminate between self-rated severity of the problem and over-all health. The frequency distributions were similar among the five shoulder questionnaires, but those of the five shoulder questionnaires differed from that of the SF-36. The correlations were good (0.73 ≤ r ≤ 0.80) among all of the five shoulder questionnaires except the Subjective Shoulder Rating Scale; they were lower with the Subjective Shoulder Rating Scale and the physical function dimension of the SF-36 (0.12 ≤ r ≤ 0.60). The shoulder questionnaires discriminated between levels of severity (p < 0.0001) but not between levels of over-all health (0.10 ≤ p ≤ 0.86). In this concurrent comparison of measures of shoulder-specific outcome in the same subjects, the shoulder questionnaires performed similarly, both in describing function of the shoulder and in discriminating between levels of severity. The shoulder questionnaires performed differently than the SF-36, which confirms the need to use both disease-specific and generic health-status measures to evaluate patients who have a problem related to the shoulder.

Measuring Function of the Shoulder. A Cross-Sectional Comparison of Five Questionnaires*

A standardized method for assessment of elbow function

Current elbow-scoring systems are based on the observer-derived assessment of a variety of clinical and functional criteria, which are scored separately and then aggregated. The aggregate score then is assigned a categorical ranking that ranges from excellent to poor. The developers of different elbow-scoring systems have chosen different outcome criteria, assigned different weights to each criterion, and accorded different ranges of values to each categorical ranking. Five different elbow-scoring systems (the Mayo elbow-performance index and the systems of Broberg and Morrey, Ewald et al., The Hospital for Special Surgery, and Pritchard) were used to evaluate the same group of patients. The validity of the scoring systems was determined with use of visual-analog scales for the assessment of pain and function, patient and physician-derived ratings of the severity of impairment of the elbow, and two functional questionnaires completed by the patient (the Disabilities of the Arm, Shoulder and Hand questionnaire and the Modified American Shoulder and Elbow Surgeons patient self-evaluation form). The study sample consisted of sixty-nine patients who had sought treatment at one of two tertiary referral clinics because of problems related to the elbow. Pearson product-moment correlation coefficients were used to compare the raw aggregate scores, and kappa statistics were used to determine the level of agreement among the categorical rankings (excellent, good, fair, and poor). Examination of the five scoring systems revealed a remarkable lack of concordance with regard to the aspects of elbow function that were assessed. Good correlation was observed when the systems were compared on the basis of raw scores (Pearson product-moment correlation coefficients, 0.79 to 0.90), but only slight-to-moderate correlation was noted when the systems were compared on the basis of categorical rankings (quadratic weighted kappa coefficients, 0.18 to 0.49). Validity testing showed the system of Ewald et al. and the Mayo elbow-performance index to be the most discriminating, the system of Pritchard to be the least discriminating, and the system of The Hospital for Special Surgery and the system of Broberg and Morrey to be intermediate. The scores determined with the elbow-scoring systems demonstrated only moderate correlation with the score for function on the visual analog scale (Pearson product-moment correlation coefficients, 0.44 to 0.66), whereas those derived from the functional questionnaires completed by the patient demonstrated moderate-to-good correlation with the score for function (Pearson product-moment correlation coefficients, 0.72 and 0.80).

Robin R. Richards

Papers

A standardized method for the assessment of shoulder function

Assessing the reliability and responsiveness of 5 shoulder questionnaires.

Measuring Function of the Shoulder. A Cross-Sectional Comparison of Five Questionnaires*

A standardized method for assessment of elbow function

Validity of Observer-Based Aggregate Scoring Systems as Descriptors of Elbow Pain, Function, and Disability*